AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

Technical Summary: AI Self-preferencing in Algorithmic Hiring

Problem Definition
As Large Language Models (LLMs) become ubiquitous in both content generation and evaluation, a novel form of algorithmic bias has emerged: AI self-preferencing. While prior computer science research has identified that LLMs tend to favor content they generated themselves over human-written or alternative model outputs, the real-world operational implications of this bias remain unquantified. This paper investigates this phenomenon within the context of algorithmic hiring, a high-stakes domain where employers increasingly use LLMs to screen resumes that applicants may have also drafted or refined using similar AI tools. The core problem is whether LLM evaluators systematically favor resumes generated by their own architecture, potentially distorting hiring outcomes independent of candidate qualifications.

Methodology
The authors employ a large-scale, controlled resume correspondence experiment to empirically measure self-preference bias.

• Dataset: The study utilizes a dataset of 2,245 anonymized, human-written resumes sourced from a professional platform (LiveCareer.com) prior to the widespread adoption of generative AI.
• Experimental Design: For each human-written resume, the authors generate counterfactual versions using a diverse set of state-of-the-art LLMs (including GPT-4o, GPT-4-turbo, LLaMA 3.3-70B, Mistral-7B, Qwen 2.5-72B, and DeepSeek-V3). Crucially, only the executive summary section is regenerated; all other factual content (experience, education, skills) remains identical to isolate the effect of the text's source.
• Evaluation Protocol: The LLMs act as evaluators in a pairwise comparison task. They are presented with two executive summaries describing the same candidate: one generated by the evaluator itself and one generated by a human or an alternative LLM. The evaluator must select the "stronger" resume.
• Bias Measurement: The study quantifies bias using two fairness metrics adapted from algorithmic fairness literature:
  1. Statistical Parity: The difference in selection rates between self-generated and non-self-generated resumes.
  2. Equal Opportunity: The difference in selection rates conditional on content quality. This is measured via conditional logistic regression (controlling for linguistic features) and human annotation (where annotators identify the objectively higher-quality summary).
• Simulation: To assess labor market impact, the authors simulate hiring pipelines across 24 occupations, modeling a scenario where a pool of candidates (half human-written, half AI-generated) competes for a fixed number of interview slots.
• Mitigation Testing: The study evaluates two intervention strategies: System Prompting (instructing models to ignore source origin) and Majority Voting Ensembles (combining the target evaluator with smaller models exhibiting lower self-recognition).

Key Results

• LLM-vs-Human Bias: There is strong, consistent evidence that LLMs prefer their own outputs. Across most models, the self-preference bias ranges from 67% to 82% after controlling for content quality. Larger models (e.g., GPT-4o, DeepSeek-V3) exhibit the strongest bias, with GPT-4o showing an 81.9% preference for its own summaries over human-written ones even when the human version is judged superior by human annotators.
• LLM-vs-LLM Bias: Self-preference is more heterogeneous when comparing different LLMs. DeepSeek-V3 shows the strongest bias in this setting (preferring its own output by 69% over LLaMA 3.3-70B), whereas GPT-4o and LLaMA 3.3-70B do not display consistent preferences against other models.
• Operational Impact: In simulated hiring pipelines, candidates using the same LLM as the evaluator are 23% to 60% more likely to be shortlisted than equally qualified applicants submitting human-written resumes. This disadvantage is most severe in business-related fields (sales, accounting, finance) and less pronounced in sectors like agriculture and arts.
• Mitigation Efficacy: Both proposed interventions significantly reduce bias.
  • System Prompting reduced self-preference by 17% to 62% (relative decrease).
  • Majority Voting Ensembles proved highly effective, reducing bias by 50% to 71% across tested models. For instance, GPT-4o's bias dropped from 82% to 30% using majority voting.

Significance and Claims
The paper claims to provide the first empirical evidence that AI self-preferencing can distort candidate evaluations in algorithmic hiring. Its primary contributions are:

1. Identification of a New Bias Mechanism: It establishes that self-preference is an interactional bias arising endogenously from AI-AI interactions, distinct from traditional demographic biases (e.g., race, gender). This challenges existing fairness frameworks that focus solely on protected attributes.
2. Quantification of Operational Risk: The study demonstrates that this bias is not merely a theoretical artifact but has tangible consequences, creating a "lock-in" effect where applicants using specific AI tools gain unwarranted advantages, potentially reducing diversity and amplifying inequities in hiring.
3. Practical Mitigation: The authors show that this bias is not immutable. Simple, low-cost design interventions (prompt engineering and ensemble methods) can substantially mitigate the distortion without requiring model retraining or weight modification.
4. Policy Implications: The findings call for expanded AI fairness frameworks in business operations that account for biases arising from AI systems evaluating content they could have produced. The authors suggest that regulators and organizations must consider transparency regarding AI usage in screening and incorporate self-preference metrics into third-party audits.

The paper concludes that while AI-assisted decision-making offers efficiency, the dual role of LLMs as both generators and evaluators introduces a structural risk that requires specific governance and technical safeguards to ensure equitable outcomes.