🤖 AI

Algorithmic Monocultures in Hiring

This paper analyzes a novel dataset of 4 million job applications screened by a single algorithm vendor and finds that such algorithmic monocultures create homogeneous rejection outcomes that disproportionately and adversely impact Black and Asian applicants, often necessitating wide application strategies to secure human review.

Original authors: Rishi Bommasani, Sarah H. Bana, Kathleen A. Creel, Dan Jurafsky, Percy Liang

Published 2026-05-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Rishi Bommasani, Sarah H. Bana, Kathleen A. Creel, Dan Jurafsky, Percy Liang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "One-Size-Fits-All" Gatekeeper

Imagine you are looking for a job. In the past, you might have sent your resume to 20 different companies. Each company had its own hiring manager, its own way of reading your resume, and its own unique taste in what makes a good employee. If one manager didn't like you, another might have loved you. You had many different doors to knock on.

Today, however, many companies don't hire alone. They all hire the same "gatekeeper."

This paper studies a specific hiring company called pymetrics. They build software that uses games to test job applicants. Instead of reading resumes, the software watches how you play these games and gives you a score. If the score is high, you move on to a human. If it's low, you are rejected immediately.

The problem? Over 60% of the biggest companies in the U.S. use the exact same gatekeeper.

The authors call this an "Algorithmic Monoculture." Think of it like a garden where every farmer uses the exact same type of seed and the exact same watering can. If that seed is bad for a certain type of soil, every farmer's garden will fail in the same way. In hiring, if the algorithm has a bias, it rejects the same people at every single company they apply to.

The Study: What They Found

The researchers got access to a massive, secret database containing 4 million job applications from 3.3 million people across 156 different companies. They looked at what happened when these people played the games.

Here are the three main things they discovered:

1. The "Bad Luck" of Race (Adverse Impact)

The researchers checked if the games treated different racial groups fairly. They looked at the results for every single job position separately, not just the average.

The Finding: They found that for Black applicants, about 26% of the applications they sent were to jobs where the algorithm was statistically likely to reject them unfairly. For Asian applicants, it was about 15%.
The Analogy: Imagine a basketball hoop that is slightly tilted. If you are tall, you might still make the shot. But if you are a specific height, the tilt makes it impossible to score. The researchers found that for certain racial groups, the "tilt" in the algorithm made it much harder to get a "pass" score, even though the game itself didn't ask about race.

2. The "Systemic Rejection" (Being Blackballed Everywhere)

This is the most surprising part. The authors asked: What happens if a person applies to many different jobs?

In a normal world, if you apply to 10 jobs, you might get rejected by 8 and accepted by 2. You have a chance. But because so many companies use the same algorithm, the results are identical.

The Finding: If an applicant applies to 10 different positions, 4% of them get rejected by all 10.
The Analogy: Imagine you are a musician trying to get a gig. You play for 10 different bands.
- Normal World: Band A hates your style, but Band B loves it. You get a job.
- Monoculture World: All 10 bands use the same "Music Judge" robot. The robot decides you aren't a fit. You get rejected by all 10 bands instantly. You are "systemically rejected." You aren't just unlucky; the system has closed every door for you at the same time.

The researchers found that this "all-or-nothing" rejection happens much more often than it would if every company were making its own independent decision.

3. The "Super-Applicant" Simulation

The researchers wanted to know: Can applicants fix this by applying to more jobs?

They used the computer's ability to run "what-if" scenarios. They simulated what would happen if an applicant applied to every single job available in the database.

The Finding: Even in this extreme simulation, no one was rejected by every single model. Everyone had at least one job where they would have been recommended.
The Reality Check: However, in the real world, people can't apply to every job. They have to be selective. The simulation showed that to have a 99.9% chance of getting at least one recommendation, a person would need to apply to 25 different positions.
The Analogy: To find a single "yes" in this system, you have to knock on 25 doors. In a normal system, you might only need to knock on 10. The algorithmic monoculture forces job seekers to work much harder just to get a single chance.

Why This Matters

The paper argues that when many companies rely on the same few vendors for hiring, they create a bottleneck.

It hides bias: If you look at the average of all companies, the numbers might look okay. But when you look at specific jobs, the bias is clear.
It creates "dead ends": It's not just that you might not get one job; it's that you might be locked out of the entire labor market because the same algorithm rejected you everywhere you looked.
It's hard to fix: Because the data is hidden inside these private companies, it's very hard for researchers or regulators to see this happening until it's too late.

The Authors' Recommendations

The paper suggests a few ways to fix this:

Check each job, not just the average: Regulators should look at how the algorithm treats people for each specific job, not just the total number of hires across a whole company.
Watch the "Monoculture": Agencies should monitor how many companies are using the same vendor. If too many rely on the same tool, it creates a risk for the whole economy.
Let researchers in: The only way to know if these systems are fair is to let independent scientists look at the data. The authors suggest laws should force these companies to share data with researchers, similar to how social media platforms are now required to share data with scientists.

Summary

This paper reveals that by all using the same "hiring robot," companies have accidentally created a system where the same people get rejected everywhere they apply. It's like having a single, biased bouncer at every club in the city; if you get turned away by him, you can't get into any club, no matter how many you try.

Technical Summary: Algorithmic Monocultures in Hiring

Problem Statement

The paper addresses the phenomenon of algorithmic monoculture in hiring, defined as a state where many decision-makers (employers) rely on the same or similar algorithms to screen job applicants. While over 90% of U.S. employers use hiring algorithms, a significant concentration exists around a few third-party vendors (e.g., HireVue, pymetrics). The authors hypothesize that this reliance on a single vendor creates homogeneous outcomes: the same individuals and racial groups face rejection across multiple employers, leading to systemic exclusion from the labor market.

Existing research on algorithmic hiring often suffers from data limitations:

Single-employer studies cannot track if rejection at Firm A predicts rejection at Firm B.
Correspondence studies (sending fake resumes) cannot observe the underlying decision-making process of the algorithms.
Static audits cannot observe deployment-time outcomes across the real labor market.

Consequently, there is a lack of empirical evidence regarding whether algorithmic monoculture leads to correlated rejections and adverse impact at a systemic level.

Methodology

The authors conducted the first large-scale empirical study observing deployed algorithmic hiring decisions across multiple employers using a single vendor.

Data Acquisition

Source: A novel dataset from pymetrics, a talent assessment platform.
Scope: 4,197,168 job applications submitted by 3,372,132 unique applicants to 1,746 distinct positions across 156 employers.
Timeframe: December 2018 to December 2022.
Process: Applicants play 12–16 online games measuring cognitive traits. pymetrics trains binary classifiers (recommend/do not recommend) based on gameplay features of current employees (positive examples) versus random profiles (negative examples).
Demographics: 40.2% of applicants self-reported race (16.8% Asian, 14.2% White, 3.6% Black, 3.0% Hispanic). The study focuses on self-reported data and does not impute race.

Analytical Framework

The study employs three primary lenses:

Adverse Impact Analysis:
- The authors apply the U.S. Equal Employment Opportunity Commission (EEOC) "four-fifths" rule (impact ratio < 0.8) and statistical significance testing (z-test).
- Crucial Distinction: Unlike prior vendor reports that aggregate data across all positions, this study disaggregates analysis per position (1,746 separate models), aligning with U.S. legal standards that operationalize discrimination on a per-job basis.
Systemic Rejection Analysis:
- The authors define systemic rejection as an applicant being rejected from all positions to which they apply.
- They introduce a Baseline of Independence: A theoretical model calculating the probability of systemic rejection if employers made statistically independent decisions. This is computed using a Poisson-Binomial distribution based on individual position selection rates.
- Comparison: Observed systemic rejection rates in the pymetrics data are compared against this baseline and against data from a large correspondence study (Kline et al., 2022) involving 108 Fortune 500 firms.
Counterfactual Simulation:
- Leveraging the deterministic replicability of the algorithms (the same gameplay features yield the same score), the authors simulated outcomes for 1,000 applicants if they had applied to every available position (495 models).
- They also modeled an "intermediate regime" where applicants apply to a "connected set" of models (models that share applicants with the ones the user actually applied to) to estimate how many applications are needed to guarantee a recommendation.

Key Results

1. Adverse Impact at the Position Level

While aggregate data across all applications showed no significant adverse impact (consistent with prior vendor claims), disaggregation revealed significant disparities:

Black Applicants: 10.62% of positions demonstrated adverse impact against Black applicants. 30.70% of Black applicants applied to at least one such position, and 25.87% of all applications submitted by Black applicants were to models with adverse impact.
Asian Applicants: 5.32% of positions demonstrated adverse impact against Asian applicants, affecting 18.53% of Asian applicants and 14.74% of their applications.
Occupational Variation: Adverse impact was particularly pronounced in Architecture and Engineering roles for both Black and Asian applicants.

2. Systemic Rejection and Homogeneity

Observed vs. Baseline: The observed rate of systemic rejection (being rejected from all $k$ positions applied to) significantly exceeded the baseline of independence ( $\chi^2 = 18,481, p < 0.001$ ).
Specific Rates: 4% of applicants who applied to 10 positions were rejected from all of them. This rate is higher than expected by chance.
Comparison to Non-Algo Hiring: In contrast, the Kline et al. (2022) study of 108 firms using mixed human/algorithmic screening showed observed systemic rejection rates that closely matched the baseline of independence. This suggests the excess homogeneity is a distinctive feature of centralized algorithmic assessment.
Mechanism: 42 pymetrics models are shared across different companies. While total monoculture (same model for all) is rare, partial monoculture (shared models across many employers) creates correlated outcomes.

3. Simulation of Applicant Behavior

Universal Recommendation: In the simulation where applicants applied to every position, no applicant was rejected by every model; the "worst" applicant was recommended by 52 models (11%).
Intermediate Regime: To achieve a systemic rejection rate below 0.1% (99.9% chance of at least one recommendation), applicants would need to submit 25 applications under the algorithmic monoculture scenario. Under the baseline of independence, only 10 applications would be required.
Implication: Applicants must apply significantly more broadly to overcome the homogenizing effect of algorithmic monoculture.

Key Contributions

First Cross-Employer Empirical Study: The paper provides the first large-scale dataset observing real algorithmic outcomes for the same applicants across multiple employers, overcoming the data silos of previous research.
Disaggregated Adverse Impact: It demonstrates that aggregating hiring data masks significant per-position adverse impact, challenging the validity of vendor-level aggregate reporting for compliance.
Evidence of Systemic Rejection: It empirically validates the theory that algorithmic monoculture leads to "algorithmic blackballing," where applicants are systematically rejected across the labor market due to correlated model behaviors, a phenomenon not observed in non-monocultural screening.
Methodological Innovation: The paper introduces a "baseline of independence" to quantify the marginal increase in systemic rejection due to algorithmic correlation and utilizes deterministic simulation to generate counterfactual labor market outcomes.

Significance and Claims

The authors claim that algorithmic monoculture fundamentally alters labor market dynamics by creating systemic exclusion.

Structural Harm: The reliance on a single vendor creates a bottleneck where a single algorithmic decision can cascade into rejection across multiple employers, disproportionately affecting specific racial groups and individuals.
Policy Implications: The findings suggest that current regulatory frameworks (e.g., NYC Local Law 144) may be insufficient if they rely on aggregate data or fail to account for position-level adverse impact and cross-employer correlations.
Research Access: The paper argues that independent research is critical to illuminate opaque hiring systems. It highlights the difficulty of acquiring such data and suggests that policy (similar to the EU Digital Services Act) may be necessary to mandate researcher access to centralized hiring datasets.

The authors remain modest regarding the generalizability of their findings to all algorithmic hiring, noting that their data is specific to pymetrics' game-based approach and that they lack external measures of applicant quality or model validity. However, they assert that the observed patterns of adverse impact and systemic rejection are sufficient to warrant regulatory scrutiny and further independent inquiry.