📈 economics

Who Uses AI? Platform Selection and the Measurement of Occupational AI Exposure

This paper demonstrates that using AI platform logs to measure occupational AI exposure introduces significant selection bias due to non-representative user bases, causing employment effect estimates to vary drastically across platforms and channels, which can be substantially corrected by reweighting the data to match official workforce demographics.

Original authors: Michelle Yin, Burhan Ogut

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michelle Yin, Burhan Ogut

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Echo Chamber" Problem

Imagine you want to know how much AI is changing the American workforce. To do this, researchers look at the chat logs of people using AI tools like ChatGPT, Claude, or Microsoft Copilot. They assume: "If people in a certain job are chatting a lot with AI, that job must be heavily exposed to AI."

This paper argues that this method is like trying to understand the entire population of a city by only interviewing people who live in the most expensive, high-tech neighborhood.

The researchers, Michelle Yin and Burhan Ogut, show that the people using these AI platforms are not a random sample of all workers. They are a specific, skewed group. Because of this, the data tells us more about who is using the app than about which jobs are actually being changed by AI.

The Core Analogy: The "Gourmet Food" Survey

Imagine you want to know how much gourmet food the average American eats.

The Flawed Method: You go to a high-end, Michelin-star restaurant and ask the diners, "How often do you eat gourmet meals?"
The Result: You get a report saying, "Wow, Americans eat gourmet food every single day!"
The Reality: You missed the millions of people who eat at fast-food joints, cook at home, or can't afford fancy restaurants. Your data is biased because you only looked at the "gourmet" crowd.

In this paper:

The "Gourmet Crowd" = The users of AI platforms (mostly computer scientists, office workers, and tech-savvy professionals).
The "Fast-Food Crowd" = Truck drivers, food preparers, construction workers, and service staff.
The Mistake: Researchers are using the "gourmet" data to make conclusions about the "whole city" (the entire workforce).

What the Researchers Found

The authors ran a series of tests to see what happens when you change the "source" of your data. Here are their main discoveries:

1. The Results Flip-Flop
If you use data from Claude (Consumer), you might conclude that AI is hurting employment in certain jobs. But if you switch to data from Claude (Enterprise/Business) or Microsoft Copilot, the results can completely flip, suggesting AI is helping or having no effect.

Analogy: It's like asking a group of coffee lovers if coffee is good for you (they say yes), then asking a group of tea drinkers if coffee is good for you (they say no). If you mix the two groups or switch between them, your answer changes wildly, even though the "coffee" itself hasn't changed.

2. The "Who" Matters More Than the "What"
The paper shows that the biggest factor changing the results isn't how capable the AI is, but which occupations are using it.

Computer and Math jobs make up only 3.4% of all US jobs, but they generate 32% of all conversations on consumer AI platforms.
Food Preparation jobs make up 8.8% of all US jobs, but they generate only 0.7% of the conversations.
The Takeaway: The data is screaming about the jobs that are already tech-heavy, while whispering about the jobs that are not.

3. The "Reweighting" Fix
The authors developed a mathematical "correction" (like a filter) to fix this bias. They took the AI chat data and forced it to look like the real US workforce (matching the actual number of truck drivers, nurses, and teachers).

The Result: When they applied this fix, the scary predictions about AI destroying jobs shrunk dramatically (by 42% to 93%).
Analogy: It's like taking a photo of a room full of billionaires and then digitally adding thousands of regular people to the photo. Suddenly, the "average wealth" of the room drops to a realistic number.

Why This Matters for Policy

The paper warns that if governments or companies use these uncorrected AI chat logs to decide who needs retraining or help, they will make a huge mistake.

The Current Plan: If you follow the raw data, you would send billions of dollars to help Computer Programmers and Financial Analysts (because the data says they are the ones using AI the most).
The Reality: You might be ignoring Cashiers, Truck Drivers, and Food Prep Workers. These groups might be at high risk of AI disruption, but because they aren't chatting with AI tools yet, the data says they are "safe."

The Bottom Line

The paper does not say AI isn't changing jobs. It says our measuring stick is broken.

Platform Data is great for seeing what AI can do and how tech-savvy people are using it.
Platform Data is terrible for predicting how AI will affect the average worker because it misses the workers who aren't using the tools yet.

The Solution: Researchers and policymakers must stop treating AI chat logs as a perfect mirror of the workforce. They need to "reweight" the data to include the people who aren't on the platform, or else they will be solving the wrong problems for the wrong people.

Technical Summary: Who Uses AI? Platform Selection and the Measurement of Occupational AI Exposure

1. The Problem: Non-Classical Measurement Error in Platform-Derived Exposure

The empirical literature on artificial intelligence (AI) and labor markets has increasingly shifted from measuring potential AI capability (e.g., expert ratings, patent matching) to measuring observed AI use via conversation logs from platforms like Anthropic's Claude, OpenAI's ChatGPT, and Microsoft Copilot. While this moves measurement closer to revealed behavior, the authors argue that these logs constitute a non-probability sample.

The core problem is that platform-derived exposure scores ( $E_{o,p}$ ) conflate two distinct components:

True Occupational Exposure ( $E_o$ ): The share of an occupation's task bundle where AI can substitute for or complement worker input.
Platform Selection ( $\psi_{o,p}$ ): The degree to which an occupation is over- or under-represented in the platform's user base relative to the general workforce.

Because the user base of AI platforms is systematically skewed toward high-skill, high-wage occupations (e.g., Computer and Mathematical) and away from service and manual labor occupations, the resulting exposure measure inherits this selection bias. The authors formalize this as non-classical measurement error, where the error term is correlated with the true variable of interest. Consequently, regression coefficients using these measures do not identify the structural employment elasticity of AI exposure; instead, they reflect a mixture of true exposure and the specific occupational composition of the platform's users.

2. Methodology

Conceptual Framework

The authors define the platform-derived proxy as:
$E_{o,p} = \psi_{o,p} E_o + \eta_{o,p} + u_{o,p}$
Where:

$\psi_{o,p}$ is the between-occupation selection parameter (ratio of platform conversation share to workforce employment share).
$\eta_{o,p}$ is the within-occupation task-selection residual (differences in task performance conditional on being on the platform).
$u_{o,p}$ is classical noise.

Because $\psi_{o,p}$ is observed (via platform logs and Bureau of Labor Statistics employment shares), the authors propose a workforce reweighting procedure to remove the between-occupation component:
$\tilde{E}_{o,p} = \frac{E_{o,p}}{\psi_{o,p}}$
Under a maintained ordering condition (that within-occupation selection biases coefficients in the same direction as between-occupation selection), the baseline estimate and the reweighted estimate define a partial-identification interval for the structural employment elasticity.

Empirical Design

The study employs a difference-in-differences (DiD) specification on a 2015–2024 American Community Survey (ACS) panel of 13.1 million person-year observations at the six-digit Standard Occupational Classification (SOC) level.

Treatment: Interaction of a post-2022 indicator with ten different platform-derived exposure measures.
Variants: The ten measures include:
- Three variants of the Massenkoff and McCrory (2026) composite (baseline, workforce-reweighted, and Wave 5 AEI weights).
- Five waves of Anthropic Claude.ai consumer data (Dec 2024–Feb 2026).
- Two waves of Anthropic enterprise API data.
- Microsoft Copilot applicability scores (raw and reweighted).
Controls: Demographics, state, year, and six-digit SOC fixed effects.

Diagnostic Tests

Cross-Platform Dispersion: Estimating the same DiD specification with different platform inputs to observe coefficient instability.
Within-Vendor Channel Substitution: Comparing consumer vs. enterprise channels of the same vendor (Anthropic) to isolate user-base composition effects, holding model and rubric constant.
Event Studies: Testing for pre-trends to distinguish measurement bias from genuine treatment effect heterogeneity.
Cross-Domain Outcomes: Analyzing correlations between platform rankings and non-labor outcomes (e.g., health, education) to demonstrate that platform visibility correlates with socioeconomic status.

3. Key Results

Instability of Coefficients

Changing only the platform input while holding the empirical design fixed causes substantial variation in the estimated employment effects:

Magnitude: The post-2022 employment coefficient varies by a factor of 1.9 across the ten variants.
Sign Reversal: Within the same vendor (Anthropic), switching from the consumer channel to the enterprise channel reverses the sign of the employment coefficient at every observed release. The consumer channel yields negative, significant coefficients, while the enterprise channel yields positive, insignificant coefficients.
Temporal Instability: Even within the same channel (Claude.ai consumer), the coefficient magnitude changes significantly across waves (e.g., from -0.12 to -0.22 percentage points) as the user base composition shifts due to product launches (e.g., Claude Code).

Effect of Reweighting

Workforce reweighting significantly attenuates the estimated effects, confirming that the baseline estimates are driven largely by platform selection:

Anthropic Composite: Reweighting attenuates the employment coefficient by 93%, rendering it statistically indistinguishable from zero.
Microsoft Copilot: As Copilot's user base is closer to workforce demographics, reweighting attenuates the coefficient by 42%, though it remains significant.
Cross-Occupation Regression: In the Massenkoff and McCrory (2026) specification linking exposure to BLS employment projections, reweighting reduces the estimated negative association from -0.70 to -0.38 percentage points, implying 45% of the published magnitude disappears when platform composition is replaced with workforce composition.

Partial Identification Bounds

Using the baseline and reweighted estimates as endpoints, the authors construct a partial-identification interval for the structural employment elasticity. For the Massenkoff and McCrory composite, the interval suggests that a one-standard-deviation increase in exposure is associated with a projected decadal employment growth decline between 0.38 and 0.70 percentage points. The width of this interval (0.32 percentage points) represents the contribution of observable between-occupation platform selection.

Distributional Incidence

Platform visibility is strongly correlated with socioeconomic advantage:

Occupations overrepresented on platforms (e.g., Computer and Mathematical) have higher wages and education levels.
Occupations underrepresented (e.g., Food Preparation, Transportation) have lower wages and higher rates of disability.
Policy Simulation: A hypothetical $10 billion retraining fund allocated based on platform exposure would direct 39% of funds to high-wage, high-education occupations that a workforce-weighted allocation would not identify as exposed.

4. Contributions and Significance

The paper makes four primary contributions to the literature:

Identification of a Specific Measurement Error: It formalizes the non-classical measurement error inherent in platform-derived AI exposure, showing that the error is selected on the same occupation-level task structure the measure intends to capture.
Empirical Diagnostics: It provides evidence that platform source, product channel, and release wave fundamentally alter downstream coefficients, even when regression designs are fixed.
Correction Procedure: It introduces a workforce-reweighting procedure and partial-identification bounds that convert this instability into an estimable component, allowing researchers to bound the structural elasticity.
Distributional Analysis: It demonstrates that platform selection correlates with wages, education, and occupational advantage, implying that exposure rankings based on logs may systematically underweight economically vulnerable populations where risk is less visible in platform data.

Claims on Significance

The authors do not claim that platform logs are uninformative. Instead, they argue that platform data are highly informative about task intensity among active users but less informative about non-users, workers weakly represented on the platform, and tasks displaced from the observed occupation-platform pair.

The paper concludes that usage-based measures recover augmentation among observed users more directly than substitution in the workforce. For policy applications (e.g., retraining, career advising), the relevant object is exposure among the workforce at risk, not exposure among current platform users. The authors advocate that applied work using platform-derived exposure should:

Report workforce-reweighted counterparts.
Test sensitivity to platform source.
Distinguish between task capability, worker adoption, and use conditional on adoption.

Without these distinctions, variation in measured exposure may reflect platform adoption patterns across occupations rather than variation in underlying AI applicability.