AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions

Technical Summary: AI Alignment Amplifies the Role of Race, Gender, and Disability in Hiring Decisions

Problem Statement
As language models (LMs) increasingly participate in consequential decision-making, including hiring, a critical question remains: do these systems reproduce historical human patterns of discrimination or generate new ones? While human labor markets exhibit well-documented biases against women, racial minorities (specifically Black candidates), and people with disabilities, it is unclear whether LMs inherit these biases from pre-training data or if post-training alignment processes reshape them. Existing literature suggests AI might replicate human cognitive biases, but the specific mechanisms and magnitudes of demographic effects in LMs compared to human employers are not fully understood.

Methodology
The authors conducted a large-scale audit study involving 27 language models (19 instruction-tuned and 8 pre-trained base models) across 177 occupations, representing nearly half of U.S. employment.

• Experimental Design: The study utilized a pairwise matching design. For each occupation, candidate profiles were generated by sampling independently from occupation-specific distributions of skills, education, work experience, and GPA. These profiles were converted into natural-language resumes.
• Demographic Counterfactuals: Each resume was replicated into eight variants representing all combinations of gender (male/female), race (White/Black), and disability status (disabled/non-disabled). All productivity-relevant attributes were held constant; only the demographic information block varied.
• Task: Models were prompted to choose the better candidate between two randomly paired profiles with differing qualifications and demographics. This design isolates the causal effect of demographic identity on hiring decisions.
• Statistical Analysis: The authors employed logistic regression on the log-odds of selecting one candidate over another, regressing the outcome on demographic differences and qualification differences. They utilized:
  • Matched Pairs: Nine sets of models sharing the same architecture but differing in alignment (pre-trained vs. instruction-tuned) to isolate the causal effect of alignment.
  • Meta-Analysis: Random-effects meta-analysis to aggregate per-model estimates and compare them against human benchmarks derived from a meta-analysis of 361 correspondence experiments.
  • Mechanism Testing: Specifications for "differential returns" (testing if qualifications yield different returns based on group) and "information asymmetry" (testing if the absence of signals disproportionately harms specific groups).

Key Contributions

1. Quantification of Demographic Effects: The study provides the first large-scale, cross-occupation quantification of how LMs treat female, Black, and disabled candidates relative to their counterparts, holding qualifications fixed.
2. Causal Role of Alignment: By comparing matched pre-trained and instruction-tuned models, the paper isolates post-training alignment as the primary driver of demographic effects, demonstrating that alignment significantly amplifies existing biases rather than merely inheriting them.
3. Divergence from Human Patterns: The research establishes that LMs do not simply replicate human discrimination; they reverse the direction of racial discrimination, attenuate the disability penalty, and amplify the female advantage compared to human employers.
4. Mechanism Identification: The paper identifies that LMs operate via statistical discrimination mechanisms (differential returns and information asymmetry) but that alignment alters the magnitude and direction of these effects asymmetrically across groups.

Results

• Demographic Effects: Across instruction-tuned models, female and Black candidates receive hiring advantages, while disabled candidates face a penalty.
  • Magnitude: The female advantage corresponds to roughly a 12% increase in hiring odds, and the Black advantage to an 8% increase. The disability penalty corresponds to a 6% decrease. These effects are comparable to six months to one year of additional education.
  • Alignment Amplification: Relative to pre-trained models, alignment amplifies the female advantage by 325% and the Black advantage by 330%. Conversely, it worsens the disability penalty by 171%.
• Comparison with Human Discrimination:
  • Race: Human employers show a 32% penalty for Black candidates; LMs reverse this, showing a ~9% advantage.
  • Disability: Human employers show a 41% penalty; LMs attenuate this to roughly 5% but do not reverse it.
  • Gender: Human employers show a slight 4% advantage for women; LMs amplify this to 13%.
• Mechanisms:
  • Differential Returns: LMs reward the same qualifications more generously for marginalized groups (higher returns to general skills for all three groups; higher returns to work experience for female and Black candidates, but a penalty for disabled candidates).
  • Information Asymmetry: When qualification signals (work experience) are absent, LMs impose additional penalties on marginalized groups. This penalty is largest for disabled candidates. Alignment causally amplifies these information-asymmetry penalties for all three groups.
• Occupational Variation: Female and Black advantages are larger in higher-wage occupations. The disability penalty is most severe in occupations where disabled workers are most prevalent.

Significance and Claims
The paper claims that AI does not merely reproduce historical human discrimination but produces a fundamentally different pattern of results. The authors argue that the role of race, gender, and disability in AI-driven hiring is comparable to significant educational differentials (6–12 months of education).

Crucially, the study challenges the assumption underlying current regulations (such as the EU AI Act and NYC Local Law 144) that AI will simply replicate historical biases. Instead, the findings suggest that post-training alignment actively reshapes the relative treatment of marginalized groups. The authors conclude that the policy question is no longer whether AI discriminates against marginalized groups, but how alignment reshapes the relative treatment across them, noting that while alignment benefits female and Black candidates in this context, it exacerbates disadvantages for disabled candidates.