# Scrapers Intro

This document explains how JobFront gathers data from employer job boards.

***

### Overview

JobFront scrapes job listings **directly from employer career pages and ATS-hosted job boards** — not from job aggregators. For each tracked company, we store the canonical URL to the job posting (e.g., `https://jobs.lever.co/acme` or `https://acme.wd5.myworkdayjobs.com/careers`). Our scraping workers visit those URLs on a recurring schedule, extract all active job listings, and update our database accordingly.

This approach means we reflect changes to a company's job board as quickly as our scraping cadence allows - between every 8 minutes and every 24 hours, depending on the job board.

***

### Scraping Cadence & SLA

**Our nominal SLA is to scrape every job board once every 24 hours.**

In practice:

* A continuous pool of workers processes job boards from a queue.
* Each job board is emitted into the queue based on when it was last successfully scraped.
* Workers process the queue continuously.
* Under normal load, most job boards are scraped **within 12–24 hours** of their last successful scrape.
* Some high-priority job boards may be prioritized to run more frequently, up to every 8 minutes.

***

### Discovery & URL Verification

#### What Is Discovery?

Employer job board urls occasionally change — a company may change their ATS provider, migrate to a new domain, or update their career site URL. When our scraper detects that a job board has likely changed, it sends the source to our **discovery pipeline** to verify and update the URL.

#### Discovery Flow

1. **Detection trigger**: After retries are exhausted and a zero-job result is confirmed, the source is flagged for rediscovery
2. **Format detection**: The discovery system visits the company's known URL and inspects where the browser actually lands (after any redirects). It then checks that current URL against our database of known ATS patterns to identify the platform.
3. **Known ATS match**: If the URL matches a known ATS format (e.g., Greenhouse, Workday, Lever), we extract the new canonical job board URL and update our record automatically.
4. **Common format search**: If the ATS format has changed entirely, the discovery system attempts to find the company's new job board by searching known ATS patterns for that company domain.
5. **Custom scraper regeneration**: For companies with fully custom career pages (not a recognized ATS), the discovery system may trigger a new custom scraper build.
6. **Resumption**: Once a valid, verified URL is confirmed, the source re-enters the normal scraping queue.

***

### Supported ATS Platforms

As of **April 2026**, JobFront actively scrapes job boards hosted on the following platforms. At this stage we're adding 10's of thousands of new sources every month, so this list is very dynamic.

#### Enterprise & Large ATS

| Platform                    |
| --------------------------- |
| Workday                     |
| ADP                         |
| Paylocity                   |
| iCIMS                       |
| SAP SuccessFactors          |
| Taleo (Oracle)              |
| Oracle Recruiting Cloud     |
| Dayforce HCM                |
| Cornerstone OnDemand (CSOD) |
| UKG (rec.pro.ukg.net)       |
| UKG / UltiPro (legacy)      |
| Avature                     |
| Phenom (PhenomPeople)       |
| Eightfold                   |
| Jibeapply                   |
| BrassRing (IBM Kenexa)      |
| Infor Cloudsuite            |
| SilkRoad                    |
| SelectMinds                 |

#### Mid-Market ATS

| Platform             |
| -------------------- |
| Greenhouse           |
| Lever                |
| Ashby                |
| Workable             |
| SmartRecruiters      |
| BambooHR             |
| ApplyToJob           |
| Jobvite              |
| Paycom               |
| SAAS HR              |
| Recruiting by Paycor |
| Hireology            |
| HRM Direct           |
| Teamtailor           |
| Join                 |
| Zoho Recruit         |
| Workstream           |
| Rippling             |
| Recruitee            |
| Personio             |
| Pinpoint             |
| Breezy HR            |
| CareerPlug           |
| ApplicantPro         |
| Hirebridge           |

#### SMB & Niche ATS

| Platform                     |
| ---------------------------- |
| Comeet (Spark Hire Recruit)  |
| Freshteam                    |
| Gem                          |
| Gusto                        |
| Harri                        |
| RecruiterBox / Trakstar Hire |
| TriNet Hire                  |
| Collage                      |
| CATS One                     |
| Cadient Talent               |
| Ceipal                       |
| Careers Page                 |
| Careers With Us              |
| Cegid HR (DigitalRecruiters) |
| Dover                        |
| EasyApply (GetHired)         |
| Factorial HR                 |
| Fountain                     |
| GovernmentJobs (NEOGOV)      |
| Hiring Today                 |
| Inhire                       |
| Instahyre                    |
| JobAppNetwork                |
| JobApsCloud                  |
| JobDiva                      |
| JobLinkApply                 |
| JobScore                     |
| Keka                         |
| Kula                         |
| Now Hiring                   |
| Paradox                      |
| Polymer                      |
| PrismHR Hiring               |
| AppOne                       |
| Apploi                       |
| Gupy                         |
| Nowhiring                    |

#### International ATS

| Platform              | Primary Region        |
| --------------------- | --------------------- |
| Welcome to the Jungle | France / Europe       |
| TalentSoft (Cegid)    | France / Europe       |
| TalentView (Kelio)    | France                |
| Personio              | Germany / Europe      |
| Join                  | Germany / Europe      |
| Gupy                  | Brazil                |
| InHire                | Brazil                |
| Instahyre             | India                 |
| Keka                  | India                 |
| Zoho Recruit          | Global (India-origin) |

#### Specialty / Hourly-Work Platforms

| Platform      | Focus                |
| ------------- | -------------------- |
| Workstream    | QSR / hourly         |
| Fountain      | Hourly / high-volume |
| Harri         | Hospitality / F\&B   |
| CareerPlug    | Hourly / franchise   |
| JobAppNetwork | QSR / franchise      |

#### Public Sector

| Platform                |
| ----------------------- |
| GovernmentJobs (NEOGOV) |
| JobApsCloud             |
| DE Jobs                 |

#### Non-Traditional Career Pages

| Platform                       | Notes                                          |
| ------------------------------ | ---------------------------------------------- |
| Notion                         | Companies hosting job listings on Notion pages |
| Y Combinator Work at a Startup | YC-backed startups                             |
| Wellfound (AngelList Talent)   | Startups                                       |

#### Custom Scrapers

Increasingly our data corpus is filled with custom-generated scrapers for non-ATS-based sites (directly gathering data from employer careers pages and websites). We have a sophisticated system that allows us to build 1000's of custom scrapers every day, and maintain those scrapers at scale to protect against changing careers pages and urls.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.jobfront.io/jobs-data-platform/scrapers-intro.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.