Which Public Datasets Best Track AI-Related Changes in U.S. Job Postings?

If you want to track how AI is changing the U.S. labor market, the most timely signal available is not employment data. It’s job postings. Employment numbers tell you what happened months ago. Job postings tell you what employers are planning right now — which skills they’re requesting, which roles they’re creating, and which roles they’re quietly not filling.

But not all job-posting data is the same. The field is dominated by a handful of commercial datasets, each with different coverage, different AI-detection methods, and different blind spots. If you’re trying to build a picture of AI’s labor-market impact — for research, for reporting, or for your own career decisions — you need to know which dataset answers which question, and where each one falls short.

Here’s the landscape, based on what the researchers and institutions who actually use these data have published.

The Three Datasets That Matter Most

The U.S. job-posting data market is effectively an oligopoly. Three providers — Lightcast, Indeed Hiring Lab, and Revelio Labs — supply the vast majority of the data used in public research, Federal Reserve analysis, and academic work. Each has a different origin story, a different coverage model, and a different way of identifying “AI” in a posting.

1. Lightcast (formerly Burning Glass Technologies)

What it is: The most widely used dataset in academic and Federal Reserve research on AI and labor markets. Lightcast aggregates job ads from thousands of online job boards, company career pages, and aggregators, capturing what the Atlanta Fed describes as the “near-universe of online job postings”. The company uses natural language processing to parse job postings and extract structured data: employer, title, industry, location, experience requirements, education, and — crucially — a detailed skills taxonomy.

How it identifies AI jobs: Lightcast’s “AI and ML” skill category contains 173 unique skills related to AI and machine learning, spanning general terms (“Artificial Intelligence”), specific concepts (“Neural Networks”), programming packages (“PyTorch”), and applications (“ChatGPT”). A job posting is classified as an “AI job” if it lists at least one skill from this category.

Why researchers use it: Lightcast data goes back to 2010, which means you can track the evolution of AI skill demand across the entire generative AI boom — and before it. The Federal Reserve Banks of Atlanta and San Francisco have both published analyses using Lightcast data to document how AI skill demand has changed over time. The New York Fed used Lightcast job-posting data combined with Anthropic’s task-level AI exposure metric to test whether AI-exposed occupations saw disproportionate hiring declines after ChatGPT’s release.

What it’s good for: Long-run trends, skill-level analysis, and regional comparisons. If you want to know whether AI skill demand is growing in marketing versus finance versus legal, Lightcast has the taxonomy to answer that. Recent data shows the share of marketing manager postings requiring AI skills rose from 8% to 34% over 18 months — a 325% increase — while financial analyst postings rose from 11% to 39%.

Limitations:

• Coverage bias. Online postings represent roughly 60–70% of all vacancies by some estimates, and they skew toward higher-skill, white-collar jobs.

• Classification errors. A 2025 SSRN paper comparing Lightcast’s skills-taxonomy method with LLM-based classification found that Lightcast-based methods “often misclassify postings,” limiting their strategic usefulness. A job that mentions “ChatGPT” in passing may be counted the same as a job that requires deep ML expertise.

• Cost. Lightcast data is commercial. Full access is expensive, which limits who can reproduce or extend findings.

2. Indeed Hiring Lab

What it is: Indeed’s research arm publishes a public AI Tracker — a monthly CSV updated from Indeed’s job-posting data, which covers a very large share of the U.S. online job market. Unlike Lightcast, Indeed’s data is partially public: the AI Tracker CSV is available on GitHub, making it one of the few sources you can analyze without a commercial license.

How it identifies AI jobs: Indeed uses a keyword-based approach, scanning job descriptions for AI-related terms from a list of nearly 600 keywords. This is broader and less precise than Lightcast’s skills taxonomy, but it captures a wider range of AI mentions — including postings where AI is mentioned as a secondary responsibility or a preferred qualification rather than a core requirement.

Why it matters: Indeed’s data shows the share of U.S. job postings mentioning AI has grown from 3.44% in August 2025 to 6.28% by July 2026 — a nearly doubling in a single year. This is one of the most timely public indicators of AI’s spread across job categories.

Indeed’s research has also revealed something important about who is adopting AI hiring: nearly 90% of all AI-related job postings come from just 1% of companies, and only about 1 in 20 firms had any AI-related posting at all as of late 2025. Half of the top 1% of firms by posting volume have adopted AI in their hiring, while few smaller firms have.

What it’s good for: Real-time tracking of AI mentions across the economy, firm-size analysis, and international comparisons (Indeed operates in multiple countries). The keyword approach also captures the “long tail” of AI references — roles where AI is a peripheral skill rather than the core job.

Limitations:

• Keyword breadth. A posting that mentions “AI” in a list of preferred qualifications is counted the same as one where AI is the primary role. Indeed itself has noted that some AI mentions “lack clear context”.

• No skills taxonomy. Unlike Lightcast, Indeed doesn’t parse skills into categories, so you can’t easily distinguish between an AI engineer and a marketing manager who happens to mention AI.

• Coverage. While Indeed is large, it’s still a single job board. Postings that appear only on company career pages or niche boards may be missed.

3. Revelio Labs

What it is: The most comprehensive commercial dataset on the market, by Revelio’s own measure. Revelio’s COSMOS dataset sources from 1 million+ company websites, all major job boards, and staffing firm job boards, with over 5 billion current and historic postings covering 7 million+ companies. Unlike Lightcast and Indeed, Revelio also integrates employee profiles, layoff notices (WARN filings), and employee sentiment data, allowing firm-level linking of postings to actual workforce changes.

How it identifies AI jobs: Revelio measures AI exposure at the activity level, not the occupation level. Each work activity in Revelio’s proprietary taxonomy is assigned an AI exposure score (0–1) by an LLM that evaluates how much of the activity current generative AI can perform. Occupation-level exposure is then aggregated from the activities that occupation performs. The measure is benchmarked against two established external measures — GPTs-are-GPTs (Eloundou et al. 2023) and Felten et al. (2023) — which correlate strongly with Revelio’s.

Why it matters: Revelio’s methodology is the most transparent about its limitations. The company publishes a detailed appendix explaining coverage biases, reporting lags, and sampling weights. It also corrects for the fact that online professional profiles skew toward white-collar workers in large cities — a nurse with a 25% chance of appearing online counts for four workers, while a software engineer with a 90% chance counts for roughly 1.1.

What it’s good for: Firm-level analysis, linking hiring demand to actual employment and layoff outcomes, and methodological transparency. If you want to know whether a specific company’s AI hiring is translating into headcount changes, Revelio is the only dataset that can answer that.

Limitations:

• Commercial-only access. Unlike Indeed’s AI Tracker, Revelio’s data is not publicly available. The company does publish a monthly AI Labor Market Tracker report, but the underlying data requires a license.

• Proprietary exposure metric. The activity-level AI exposure score is built with an undisclosed LLM and taxonomy, which limits reproducibility.

• Reporting lag. Revelio corrects for the lag in profile updates with a nowcasting model, but the most recent months remain the least reliable.

Comparison Table

What the Data Actually Shows — and What It Doesn’t

The most important finding from comparing these datasets is not the AI-job numbers themselves. It’s how different the signals are depending on the method.

The New York Fed used Lightcast data and Anthropic’s task-level exposure metric to test whether AI-exposed occupations saw hiring declines after ChatGPT’s release. Their conclusion: “the evidence from job postings provides little indication of a distinct AI-driven decline in labor demand”. Less than 10% of workers and vacancies are in occupations with high AI exposure, and 40% of workers are in jobs with zero measured AI exposure.

But Indeed’s data shows AI mentions in postings nearly doubling in a single year — from 3.44% to 6.28%. And Revelio’s data links posting changes to actual layoff notices and headcount data, which Lightcast and Indeed cannot do.

The tension between these signals is not a contradiction. It’s a measurement gap. “AI-exposed occupation” (Lightcast + Anthropic) and “posting mentions AI” (Indeed) are measuring different things. One measures whether a job’s tasks could be affected by AI. The other measures whether employers are talking about AI in their hiring. Neither directly measures whether AI is actually changing hiring outcomes.

Methodology Matters More Than You Think

The biggest mistake you can make with job-posting data is treating any of these measures as a direct proxy for “AI is changing jobs.”

Postings measure demand, not employment. A posting is an intention. It may never be filled. It may be filled by an internal candidate. It may be posted for compliance reasons. Revelio explicitly notes that postings “measure hiring demand rather than existing employment, and so serve as an early indicator of changes in labor demand” — not a measure of actual employment change.

AI exposure ≠ AI adoption. The New York Fed’s measure of AI exposure is derived from task-level analysis of what AI could do, combined with observed AI usage patterns. It does not measure whether a given firm has actually adopted AI or whether it has changed its hiring because of it. As the New York Fed writes: “A job being exposed to AI may not translate into reduced hiring or increased layoffs for the occupation as a whole”.

Keyword mentions are not skill requirements. A posting that says “familiarity with AI tools a plus” is not the same as a posting that says “requires 5+ years of ML engineering experience.” Indeed’s own analysis found that only about half of AI-related job descriptions mentioned building new AI tools or directly using AI models through prompts. The rest were vague, contextual, or aspirational.

Coverage bias is real and directionally important. Online postings over-represent white-collar, high-skill, and urban jobs. If you’re trying to understand AI’s impact on blue-collar work, retail, or rural labor markets, postings data will systematically undercount what you’re looking for.

A Practical Guide: Which Dataset for Which Question

What I’m Still Trying to Figure Out

I don’t have this solved. Here’s what I’m wrestling with:

1. How do you validate these datasets against each other? Lightcast says AI job postings are growing. Indeed says AI mentions are growing. Revelio says AI exposure is concentrated. But none of them measure the same thing. Is there a way to triangulate?

2. What about the jobs that aren’t posted online? The 30–40% of vacancies that never appear on a job board are invisible to all three datasets. If those vacancies are concentrated in sectors with different AI exposure, the aggregate picture is incomplete.

3. How do you separate AI talk from AI work? A posting that mentions “AI” in a preferred-qualifications line is not the same as a posting for an AI engineer. Lightcast’s skills taxonomy gets closer, but the SSRN paper shows it still misclassifies.

4. When does posting data become leading indicator vs. noise? The New York Fed found little evidence of AI-driven hiring decline. Indeed found AI mentions nearly doubling. At what point does the volume of AI mentions translate into changes in actual hiring behavior?

5. What’s the public-data alternative? All three major datasets are commercial (though Indeed’s tracker is partially public). Is there a fully public, reproducible dataset that tracks AI in U.S. job postings? The Kaggle AI/ML postings dataset is a snapshot, not a longitudinal series. The BLS JOLTS data tracks openings and hires but not AI content.

6. What am I missing? This is the big one. What dataset or method am I not aware of?