🤖 AI

Explanation Fairness in Large Language Models: An Empirical Analysis of Disparities in How LLMs Justify Decisions Across Demographic Groups

This paper introduces the Explanation Fairness Taxonomy (EFT) and presents empirical evidence that large language models systematically generate explanations of varying quality, tone, and sophistication across demographic groups, revealing that while prompting can mitigate decision-linked disparities, stylistic inequalities persist due to pre-training distributions.

Original authors: Gautam Veldanda

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gautam Veldanda

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a hiring manager, a doctor, or a loan officer. You have to make a tough decision: reject a candidate, deny a loan, or turn away a patient. Now, imagine you have a super-smart robot assistant (a Large Language Model, or LLM) that helps you make these choices.

This paper asks a very specific question: If the robot makes the exact same decision for two different people, does it treat them equally when it explains why?

The researchers found that the answer is often no. Even when the decision is identical, the robot's explanation changes based on the person's name, which signals their race, gender, religion, or age.

Here is a breakdown of the paper's findings using simple analogies:

1. The "Two-Story" Problem

Imagine two people, Alex and Jordan, apply for the same job. They have the exact same resume. The robot decides to reject both.

To Alex (a stereotypical White male name): The robot writes a long, polite, detailed letter. It says, "After careful review, we feel you lack specific leadership experience in cross-functional teams. We recommend you gain this skill and reapply." It sounds like a helpful coach.
To Jordan (a stereotypical Black female name): The robot writes a short, blunt note. It says, "This candidate does not meet current requirements." It sounds like a cold door slam.

The decision (rejection) was the same, but the explanation was unfair. The paper calls this Explanation Fairness.

2. The "Fairness Ruler" (The Taxonomy)

The authors created a new measuring stick called the Explanation Fairness Taxonomy (EFT). Think of it as a ruler with five different sides to measure how "unfair" an explanation is:

Verbosity (Length): Does the robot write a novel for one person and a tweet for another?
Sentiment (Tone): Is the tone warm and respectful for one group, but cold and dismissive for another?
Hedging (Certainty): Does the robot sound confident ("You are not qualified") for one group, but wishy-washy ("You might not be qualified") for another?
Decision-Link (Faithfulness): If the decision changes (e.g., from reject to accept), does the explanation change logically? Or does the robot just give the same generic excuse regardless of the outcome?
Lexical Complexity (Vocabulary): Does the robot use simple, easy words for one group, but confusing, fancy jargon for another?

3. The Experiment: A "Name Swap" Test

The researchers tested this on 5 different AI models (like GPT-4, Claude, LLaMA, etc.) across 4 serious areas: hiring, medical triage, credit checks, and legal judgments.

They used a "name swap" trick. They took a scenario (e.g., "Candidate X is rejected") and ran it through the AI 80 times, changing only the name to signal different demographics (e.g., changing "John Smith" to "Jamal Washington" or "Priya Patel").

The Result: The AI models were consistently unfair.

The "Bad" Actor: One model, Qwen3, was the worst offender. It was 5.9 times more likely to give a short, lazy explanation to one group and a long, detailed one to another, compared to a better model like LLaMA.
The "Good" Actors: Models like GPT-4.1 and LLaMA 3.3 were much fairer, though not perfect.
The Pattern: The unfairness wasn't the same for everyone.
- Muslim-named applicants often got shorter, less detailed explanations than Christian-named applicants.
- Female applicants often got longer explanations than males, but the tone wasn't necessarily nicer.
- Older applicants often got shorter explanations than younger ones.

4. The "Magic Prompt" Myth

The researchers tried to fix this by giving the AI a "magic instruction" (a prompt) before it started writing. They told it: "Ignore names and be fair," or "Give everyone a detailed, respectful explanation."

The Surprise: The magic didn't work.

The instructions successfully fixed the "Decision-Link" issue (making the explanations match the decision better).
But, the instructions failed completely to fix the tone, length, or vocabulary issues. The AI still wrote short, cold notes for some groups and long, warm ones for others, even after being told to be fair.

The Takeaway: The paper suggests that these unfair styles are "baked into" the AI's brain during its initial training (like a habit learned in childhood). You can't fix a deep-seated habit just by telling the person to "be nice" in a single conversation; you have to retrain them from scratch.

5. Why Should We Care? (The "Right to Explain")

The paper points out a looming legal deadline. By August 2026, a new law in the EU (the AI Act) will require high-risk AI systems (like those used in hiring or loans) to explain their decisions.

The paper warns: Just having an explanation isn't enough.
If a bank gives a loan rejection to a minority applicant with a 3-sentence, cold explanation, but gives a 3-paragraph, helpful explanation to a majority applicant, the bank might be technically following the law (they gave an explanation) but violating the spirit of the law (the explanation wasn't equal quality).

Summary

This paper proves that AI models often act like a two-faced judge: they might give the same verdict to everyone, but they treat people differently when they hand down the reasoning.

The Problem: The AI's "voice" changes based on who is listening.
The Cause: It's likely built into the model's training data, not just a simple mistake.
The Fix: Telling the AI to "be fair" in a prompt doesn't work. We need to pick better AI models (some are naturally fairer) and potentially retrain them to fix these deep-seated biases.

Technical Summary: Explanation Fairness in Large Language Models

1. Problem Statement

While extensive research exists on decision-level fairness in Large Language Models (LLMs)—focusing on whether protected attributes correlate with adverse outcomes—there is a critical gap regarding explanation fairness. This paper defines explanation fairness as the systematic variation in the quality, depth, tone, and linguistic sophistication of justifications provided by LLMs across different demographic groups, even when the underlying decision is identical.

The authors argue that a model can produce balanced decisions while explaining them unequally (e.g., offering elaborate, respectful justifications to one group and curt, dismissive ones to another). This disparity is distinct from decision fairness and holds significant regulatory urgency, particularly in light of the EU AI Act (Article 13), which mandates that high-risk AI systems provide transparent justifications sufficient for individuals to understand and contest decisions. If explanation quality is systematically lower for minority groups, compliance with the "right to explanation" is compromised.

2. Methodology

2.1 Experimental Design

The study employs a controlled empirical design using name substitution within prompt templates, a standard methodology in correspondence-style audit research.

Domains: Four consequential decision domains were selected: Hiring, Medical Triage, Credit Assessment, and Legal Judgment.
Templates: 80 prompt templates were constructed (20 per domain). In each pair, the scenario content (qualifications, circumstances, decision outcome) is held constant, while the demographic attribute (name) is varied to represent different groups (Gender, Race, Age, Religion, and Intersectional identities).
Models: Five LLMs were evaluated: GPT-4.1, Claude Sonnet, LLaMA 3.3 70B, GPT-OSS 120B, and Qwen3 32B.
Data Volume: Up to 400 prompt pairs were evaluated across all models. For the Decision-Linked Explanation metric, a contrastive explanation was elicited for each prompt (justifying the opposite decision in the same scenario) to compute faithfulness proxies.

2.2 The Explanation Fairness Taxonomy (EFT)

The paper introduces the Explanation Fairness Taxonomy (EFT), a pragmatic framework comprising five formally defined, operationalizable dimensions. These dimensions are measurable via black-box metrics without access to model internals:

Verbosity Disparity: Measured by word count and elaboration depth (number of semantically distinct reasons identified via sentence embedding clustering).
Sentiment Disparity: Measured by the VADER compound score to assess emotional valence (positive, negative, neutral).
Epistemic Hedging Disparity: Measured by the Hedging Density Score (HDS), a weighted lexicon-based metric quantifying the frequency of uncertainty markers (e.g., "might," "possibly," "uncertain").
Decision-Linked Explanation Disparity: Measured by the Explanation Faithfulness Proxy (EFP). This heuristic calculates the structural dissimilarity (via cosine distance of embeddings) between an explanation for a decision ( $d$ ) and an explanation for the opposite decision ( $\neg d$ ) in the same scenario. A lower EFP indicates the explanation is invariant to the decision, suggesting a lack of genuine reasoning.
Lexical Complexity Disparity: Measured by Flesch-Kincaid Grade Level (FKGL), Type-Token Ratio (TTR), and domain-term density.

2.3 Statistical Analysis

Disparities were quantified using absolute differences between demographic pairs. Statistical significance was tested using the one-sample Wilcoxon signed-rank test, with effect sizes calculated via Cohen's $d$ . Multiple comparisons were corrected using the Benjamini-Hochberg (BH) procedure.

3. Key Contributions

Operational Taxonomy: The definition of the EFT, providing five distinct, measurable dimensions of explanation-level disparity.
Novel Metrics: The introduction of HDS and EFP as practical, black-box heuristics for auditing explanation fairness without requiring model weights or internal access.
Empirical Benchmark: A large-scale study across 80 prompt templates, 4 domains, and 5 LLMs, with results showing systematic disparities across all metrics.
Mitigation Analysis: An evaluation of two prompting interventions ("Blind" and "Fairness" instructions) on the highest-disparity models to determine if deployment-level instructions can resolve these issues.

4. Key Results

4.1 Existence of Disparities (RQ1)

All eight EFT metrics (including sub-metrics for lexical complexity) showed statistically significant disparities across all prompt pairs ( $p_{BH} < 10^{-62}$ ). Effect sizes (Cohen's $d$ ) ranged from small to large:

Large Effects: Lexical Complexity (FKGL, $d=0.856$ ) and Epistemic Hedging (HDS, $d=0.810$ ).
Medium Effects: Elaboration depth, Sentiment, TTR, Domain Density, and EFP ( $d \approx 0.57–0.77$ ).
Small Effects: Verbosity (Word Count, $d=0.457$ ).
The distribution of disparities was right-skewed, meaning while average disparities were moderate, specific demographic pairings exhibited extreme inequality.

4.2 Demographic Axes (RQ2)

Disparities varied by demographic axis but did not follow a single uniform pattern of harm:

Religion: Showed the highest mean disparity ( $d=0.822$ ), particularly in elaboration depth and lexical complexity, though the sample size for this axis was small ( $n=40$ ).
Directionality: The direction of disparity was not uniform. For example, Muslim-named applicants received significantly shorter and less elaborate explanations than Christian-named applicants. Conversely, female applicants received longer and more elaborate explanations than male applicants, though this did not correlate with more positive sentiment.

4.3 Model Variance (RQ3)

Model choice was strongly associated with the magnitude of disparity:

Qwen3 32B exhibited the highest disparities, with verbosity disparities 5.9 times larger than LLaMA 3.3 70B. It frequently produced minimalist outputs for one demographic group while generating detailed responses for another in identical scenarios.
GPT-OSS 120B showed the highest disparity in the EFP proxy, particularly in the legal domain, suggesting its explanations were less sensitive to decision changes for certain groups.
GPT-4.1 and LLaMA 3.3 70B were the most equitable models, ranking 4th and 5th, though they still exhibited statistically significant disparities.

4.4 Mitigation via Prompting (RQ4)

Two prompting interventions ("Blind" and "Fairness") were tested on Qwen3 32B and GPT-OSS 120B:

EFP Proxy: Both interventions resulted in significant reductions (78–95%) in EFP disparity. However, the authors note this is ambiguous; it could indicate more consistent decision-sensitivity or simply uniformly generic explanations.
Stylistic Dimensions: Neither intervention produced significant reductions in verbosity, sentiment, hedging, or lexical complexity. In some cases, disparities increased (though not significantly).
Conclusion: Prompting alone is insufficient to resolve stylistic explanation inequalities.

5. Significance and Claims

The paper claims that explanation fairness is a distinct, measurable, and legally consequential dimension of AI safety, separate from decision fairness.

Regulatory Implications: The findings suggest that organizations may satisfy the "letter" of the EU AI Act (providing an explanation) while failing its "spirit" (providing an explanation of equal quality and accessibility). The EFT offers a reproducible, black-box auditing framework to detect these asymmetries prior to the 2026 enforcement window.
Deployment Guidance: The study identifies model selection as the highest-leverage intervention. Practitioners are advised that simple prompting strategies cannot resolve stylistic disparities; instead, addressing these gaps likely requires training-level interventions (e.g., contrastively balanced data or targeted fine-tuning).
Limitations: The authors explicitly state that EFP is a heuristic proxy requiring human validation, that the study relies on automated metrics without human-rated ground truth, and that the name-substitution design captures correlational signals rather than causal attribution to specific demographic attributes.

In summary, the paper establishes that LLMs systematically justify decisions differently based on demographic framing, that these disparities are robust across models and domains, and that they cannot be easily fixed via deployment-level prompting, necessitating a shift toward training-level remediation and rigorous automated auditing.