Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring

1. Problem Statement

While extensive research has documented that Large Language Models (LLMs) exhibit name-based bias in hiring outcomes (e.g., callback rates or direct salary recommendations), there is a critical gap in understanding how this bias manifests in intermediate artifacts within multi-stage hiring pipelines. Modern hiring systems often use LLMs to generate candidate summaries or competency assessments, which are then fed to human recruiters or downstream AI systems.

The paper addresses the following questions:

• Do LLMs introduce bias when summarizing resumes, even if the factual content remains stable?
• How does name-conditioned bias propagate through these intermediate summaries to affect downstream hiring decisions?
• Is this bias detectable via standard aggregate fairness audits, or does it hide in subtle, non-directional variations?

2. Methodology

The authors conducted a large-scale, controlled experiment using synthetic data to isolate causal effects of names while controlling for resume content.

A. Data Curation

• Synthetic Resumes: The team constructed 1,073 unique resumes across 232 job titles using standardized O*NET task statements and the OpenResume dataset scaffold. This ensured that all candidates had identical educational backgrounds, job trajectories, and skill sets, differing only in the specific task bullet points sampled from a controlled pool.
• Job Postings: Real-world job postings were scraped from Indeed, LinkedIn, and ZipRecruiter and semantically matched to the resumes.
• Name Perturbations: Each resume was paired with 8 intersectional race-gender name variants (White/Black/Hispanic/Asian $\times$ Male/Female), totaling nearly one million (982,656) LLM-generated summaries.
• Models: Four LLMs were tested: GPT-4o-mini (proprietary), Qwen2.5-32B, Llama-3.1-8B, and Gemma-9B (open-source).

B. Experimental Design

• Output Structure: LLMs were prompted to generate a 4-sentence summary:
  • S1–S3: Factual, resume-grounded descriptions of experience.
  • S4: An evaluative sentence explaining alignment with the target job.
• Counterfactual Analysis: For every resume-job pair, the model generated 8 summaries differing only in the applicant's name. This allowed for clean attribution of variation to name conditioning.
• Decomposition: The analysis separated factual content (S1–S3) from evaluative framing (S4).

C. Evaluation Metrics

• Coarse-Grained: Sentence length, lexical overlap (Jaccard similarity), and sentiment valence (VADER).
• Component-Level:
  • Factuality: Measured via MiniCheck (entailment probability against the source resume).
  • Macro-Category: A fine-tuned RoBERTa classifier categorized tasks into Analytical, Managerial, Operational, or Social.
  • Evaluative Framing: Measured via Subjectivity (TextBlob) and Agency (Language Agency Classifier).
• Hiring Simulation: A downstream LLM judge scored the summaries on Competence, Agency, and Fit (1–10 scale) to measure decision volatility.

3. Key Contributions

1. Localization of Bias: The paper demonstrates that name-conditioned bias in LLM hiring primarily arises from evaluative framing (S4) rather than factual hallucination.
2. Tail-Distributed Instability: The bias is not a uniform shift in average scores but is concentrated in the distributional tails. While average metrics appear stable, specific name pairs trigger extreme, non-directional instability.
3. Transformation of Bias Type: The study reveals that evaluative summarization transforms directional bias (e.g., White names consistently scoring higher) into symmetric instability (arbitrary fluctuations where no group is consistently advantaged). This "algorithmic arbitrariness" evades standard group-level fairness audits.
4. Framework for Instance-Level Auditing: The authors propose a component-level decomposition framework that enables targeted auditing of intermediate artifacts, moving beyond monolithic outcome assessments.

4. Key Results

A. Stability of Factual Content

• Factuality: Sentences S1–S3 remained highly factual and grounded in the resume across all name variants.
• Lexical Overlap: While there was a slight decrease in lexical overlap for later sentences (S3) under name swaps, the magnitude was negligible.
• Macro-Structure: The distribution of task categories (Analytical, Managerial, etc.) showed no significant differences across demographic groups.

B. Instability in Evaluative Framing (S4)

• Amplification in Open-Source Models: Open-source models (Llama, Gemma, Qwen) exhibited significantly higher amplification of Subjectivity and Agency in S4 compared to GPT-4o-mini.
• Tail Concentration: The instability was concentrated in the tails of the distribution. For certain name pairs (particularly Hispanic and Asian-coded names), the Across/Within ratio (comparing name-swapped variance to inference noise) was significantly elevated (up to 1.9x for Gemma).
• Non-Directional: The shifts were often symmetric; a candidate might receive a high agency score with one name and a low score with another, without a consistent group-level advantage.

C. Downstream Hiring Simulation

• Elimination of Directional Bias: When judges evaluated only S4 (the evaluative sentence), the directional racial bias seen in direct resume evaluation disappeared (no significant group differences in mean scores).
• Introduction of Volatility: However, S4-only evaluation doubled the within-group score ranges and tripled the rate of large disagreements (≥2 points) compared to full resume evaluation.
• Anchoring Effect: Even when judges saw the full summary (S1–S4), the instability in S4 "anchored" the final decision. Groups with high S4 agency variation showed 15.2% of cases with score ranges ≥2, double the baseline of 7.2%.
• Decision Flips: At moderate screening thresholds, the "flip rate" (where a candidate is hired with one name but rejected with another) was significantly higher for S4-only and Full evaluations compared to Resume-only.

5. Significance and Implications

• The "Invisible" Harm: Traditional fairness audits that check for group-level disparities (e.g., "Do Black candidates score lower on average?") may fail to detect this type of bias. The harm here is procedural unfairness and algorithmic arbitrariness, where outcomes depend on random or arbitrary name signals rather than merit.
• Pipeline Risks: As LLMs become integral to multi-stage hiring pipelines, the "symmetric instability" generated in intermediate summaries can compound, leading to a high volume of arbitrary exclusions that are difficult to trace back to the demographic signal.
• Mitigation Strategies: The authors suggest component-level monitoring (separating factual extraction from evaluative synthesis) and tail-aware triage (flagging cases with extreme framing scores for human review) as more effective interventions than retraining models to shift mean scores.
• Open-Source vs. Proprietary: The findings highlight that open-source models may be more susceptible to these framing instabilities than proprietary models like GPT-4o-mini, suggesting a need for stricter auditing of open-weight models in high-stakes domains.

In conclusion, the paper argues that the danger of LLMs in hiring is not just in what they say (factual errors), but in how they frame it (evaluative instability), creating a new class of bias that is subtle, non-directional, and highly disruptive to fair decision-making.