Scrapers Intro
This document explains how JobFront gathers data from employer job boards.
Overview
JobFront scrapes job listings directly from employer career pages and ATS-hosted job boards — not from job aggregators. For each tracked company, we store the canonical URL to the job posting (e.g., https://jobs.lever.co/acme or https://acme.wd5.myworkdayjobs.com/careers). Our scraping workers visit those URLs on a recurring schedule, extract all active job listings, and update our database accordingly.
This approach means we reflect changes to a company's job board as quickly as our scraping cadence allows - between every 8 minutes and every 24 hours, depending on the job board.
Scraping Cadence & SLA
Our nominal SLA is to scrape every job board once every 24 hours.
In practice:
A continuous pool of workers processes job boards from a queue.
Each job board is emitted into the queue based on when it was last successfully scraped.
Workers process the queue continuously.
Under normal load, most job boards are scraped within 12–24 hours of their last successful scrape.
Some high-priority job boards may be prioritized to run more frequently, up to every 8 minutes.
Discovery & URL Verification
What Is Discovery?
Employer job board urls occasionally change — a company may change their ATS provider, migrate to a new domain, or update their career site URL. When our scraper detects that a job board has likely changed, it sends the source to our discovery pipeline to verify and update the URL.
Discovery Flow
Detection trigger: After retries are exhausted and a zero-job result is confirmed, the source is flagged for rediscovery
Format detection: The discovery system visits the company's known URL and inspects where the browser actually lands (after any redirects). It then checks that current URL against our database of known ATS patterns to identify the platform.
Known ATS match: If the URL matches a known ATS format (e.g., Greenhouse, Workday, Lever), we extract the new canonical job board URL and update our record automatically.
Common format search: If the ATS format has changed entirely, the discovery system attempts to find the company's new job board by searching known ATS patterns for that company domain.
Custom scraper regeneration: For companies with fully custom career pages (not a recognized ATS), the discovery system may trigger a new custom scraper build.
Resumption: Once a valid, verified URL is confirmed, the source re-enters the normal scraping queue.
Supported ATS Platforms
As of April 2026, JobFront actively scrapes job boards hosted on the following platforms. At this stage we're adding 10's of thousands of new sources every month, so this list is very dynamic.
Enterprise & Large ATS
Workday
ADP
Paylocity
iCIMS
SAP SuccessFactors
Taleo (Oracle)
Oracle Recruiting Cloud
Dayforce HCM
Cornerstone OnDemand (CSOD)
UKG (rec.pro.ukg.net)
UKG / UltiPro (legacy)
Avature
Phenom (PhenomPeople)
Eightfold
Jibeapply
BrassRing (IBM Kenexa)
Infor Cloudsuite
SilkRoad
SelectMinds
Mid-Market ATS
Greenhouse
Lever
Ashby
Workable
SmartRecruiters
BambooHR
ApplyToJob
Jobvite
Paycom
SAAS HR
Recruiting by Paycor
Hireology
HRM Direct
Teamtailor
Join
Zoho Recruit
Workstream
Rippling
Recruitee
Personio
Pinpoint
Breezy HR
CareerPlug
ApplicantPro
Hirebridge
SMB & Niche ATS
Comeet (Spark Hire Recruit)
Freshteam
Gem
Gusto
Harri
RecruiterBox / Trakstar Hire
TriNet Hire
Collage
CATS One
Cadient Talent
Ceipal
Careers Page
Careers With Us
Cegid HR (DigitalRecruiters)
Dover
EasyApply (GetHired)
Factorial HR
Fountain
GovernmentJobs (NEOGOV)
Hiring Today
Inhire
Instahyre
JobAppNetwork
JobApsCloud
JobDiva
JobLinkApply
JobScore
Keka
Kula
Now Hiring
Paradox
Polymer
PrismHR Hiring
AppOne
Apploi
Gupy
Nowhiring
International ATS
Welcome to the Jungle
France / Europe
TalentSoft (Cegid)
France / Europe
TalentView (Kelio)
France
Personio
Germany / Europe
Join
Germany / Europe
Gupy
Brazil
InHire
Brazil
Instahyre
India
Keka
India
Zoho Recruit
Global (India-origin)
Specialty / Hourly-Work Platforms
Workstream
QSR / hourly
Fountain
Hourly / high-volume
Harri
Hospitality / F&B
CareerPlug
Hourly / franchise
JobAppNetwork
QSR / franchise
Public Sector
GovernmentJobs (NEOGOV)
JobApsCloud
DE Jobs
Non-Traditional Career Pages
Notion
Companies hosting job listings on Notion pages
Y Combinator Work at a Startup
YC-backed startups
Wellfound (AngelList Talent)
Startups
Custom Scrapers
Increasingly our data corpus is filled with custom-generated scrapers for non-ATS-based sites (directly gathering data from employer careers pages and websites). We have a sophisticated system that allows us to build 1000's of custom scrapers every day, and maintain those scrapers at scale to protect against changing careers pages and urls.
Last updated