🤖 AI

A Scalable Entity-Based Framework for Auditing Bias in LLMs

This paper introduces a scalable, entity-based framework for auditing LLM bias that leverages synthetic data to conduct the largest study to date (1.9 billion data points), revealing systematic disparities such as political and geographic favoritism that persist despite instruction tuning and are amplified by increased model scale.

Original authors: Akram Elbouanani, Aboubacar Tuo, Adrian Popescu

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Akram Elbouanani, Aboubacar Tuo, Adrian Popescu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read librarian named "The Model." This librarian has read almost everything ever written on the internet. Now, imagine you want to test if this librarian is fair when talking about different people, countries, or companies.

The problem is, if you just ask the librarian, "Is Country X a good place?" they might give you an answer based on what they've read in the news, which could be biased. If you ask, "Is Politician Y honest?" they might lean on their personal opinions formed from years of reading.

This paper introduces a clever new way to test the librarian's fairness without tricking them or asking them direct opinion questions. Here is how they did it, explained simply:

1. The "Name-Swap" Game

Instead of asking the librarian for their opinion, the researchers created a game. They wrote hundreds of thousands of short stories (sentences) where the facts clearly pointed to one answer, but the name of the person or place was the only thing that changed.

The Setup: Imagine a sentence that says, "The leader of [Name] signed a treaty that everyone agreed was fair."
The Test: The sentence structure is identical, but they swap [Name] with "France," "North Korea," "Germany," or "Syria."
The Logic: Since the sentence says the treaty was "fair," the answer should be the same no matter who the leader is. If the librarian suddenly says "France is good" but "North Korea is bad" for the exact same sentence, that proves the librarian is judging based on the name, not the facts.

2. The "Fake News" Factory (Synthetic Data)

To test this on a massive scale, the researchers didn't just use real news articles (which are messy and hard to control). They built a "factory" that generated 1.9 billion fake sentences.

Think of this like a video game where you can spawn thousands of identical rooms, but you just change the painting on the wall. This allowed them to test the librarian on a scale never seen before, covering politicians, countries, and companies in English, Russian, and Chinese.

The Big Discovery: They checked if these "fake" sentences worked like real ones. They found that the librarian's bias in the fake sentences matched their bias in real news perfectly. This means their "factory" is a valid way to test fairness without needing millions of real-world examples.

3. What the Librarian Actually Said (The Results)

When they ran the game with 1.9 billion data points, they found some very consistent patterns. The librarian wasn't neutral; they had strong "pre-judgments" based on names:

Politics: The librarian liked politicians on the Left and disliked those on the Far-Right. They also tended to rate female politicians slightly higher than male ones.
Geography: The librarian had a strong "West vs. Rest" bias. Wealthy Western countries (like Sweden or Switzerland) got high scores. Countries in the Global South (like Syria or North Korea) got very low scores, even when the sentence was positive.
Business: Western companies got a pass. Companies in the defense (weapons) and pharmaceutical (medicine) sectors got penalized, likely because the librarian associated them with controversy in their training data.

4. Does Size or Language Matter?

The researchers tested if making the librarian "smarter" (bigger models) or speaking a different language helped.

Bigger is Not Better: Surprisingly, the bigger, more powerful models were actually more biased. They had stronger opinions than the smaller ones.
Language Doesn't Fix It: You might think asking the librarian in Russian or Chinese would make them fairer to Russian or Chinese entities. It didn't. Even when asked in their native language, the librarian still favored Western entities. It seems the librarian's "brain" was built mostly on English data, and that English bias sticks no matter what language you speak to them.
Instructions Help a Little: If you tell the librarian, "Be neutral and follow the rules" (Instruction Tuning), they become slightly less biased, but they don't stop being biased. They just tone down their opinions a bit.

5. Why This Matters

The paper concludes that these large language models are like mirrors reflecting the world's existing inequalities. If you use them to make decisions about who gets a loan, which news to show, or how to assess a country's risk, they might unfairly penalize certain groups just because of their names, not because of the actual evidence.

The authors built a "bias detector" tool (which they made public) so that before anyone uses these AI models for important jobs, they can run this "Name-Swap" game to see if the model is fair.

In short: The paper shows that even when you give an AI all the facts, it often still judges you based on your name, your country, or your company's industry, and making the AI bigger or speaking to it in different languages doesn't fix this problem.

Technical Summary: A Scalable Entity-Based Framework for Auditing Bias in LLMs

Problem Statement

Existing approaches to bias evaluation in Large Language Models (LLMs) face a fundamental trade-off between ecological validity and statistical control. Traditional methods either rely on artificial, template-based prompts that poorly reflect real-world usage (e.g., political compasses, CrowS-Pairs) or utilize naturalistic tasks that lack the scale and rigor necessary to isolate specific variables. Consequently, bias is often treated as a stable property rather than a context-dependent phenomenon, limiting the external validity of evaluations. Furthermore, auditing bias in real-world data is challenging due to the difficulty of systematically controlling confounding variables such as topic, syntax, and narrative context. There is a critical need for audit methods that maintain task fidelity while enabling the isolation of entity-specific biases to ensure reliability in high-stakes domains like politics, geopolitics, and economics.

Methodology

The authors propose a scalable, entity-based framework designed to reconcile statistical rigor with ecological validity. The core principle is robustness: a model's prediction for a given text should remain invariant when the named entity is swapped, provided the text contains sufficient evidence to determine the correct label independently of the entity.

1. Framework Components

Entity Sets: The study focuses on three socially significant domains:
- Politics: 984 politicians categorized by ideological alignment (Far-Left to Far-Right) and gender.
- Geopolitics: 1,200 countries and territories.
- Economics: 1,200 companies across 10 industries, diversified by region and ownership.
Synthetic Data Generation: To achieve scale while maintaining control, the authors use GPT-5.1 to generate $\sim$ 1,000 sentence templates per task. These templates are "evidence-light" regarding entity-specific facts but "semantically determinate" for the target label. For example, a sentence describing a funding collapse implies a "Not Credible" label regardless of whether the entity is Sweden or Algeria. This design ensures that any shift in prediction is attributable solely to the entity name.
Tasks: The framework evaluates 12 downstream tasks across the three domains, including sentiment analysis, leadership evaluation, financial risk assessment, and international law breach detection.
Bias Metric: Bias is quantified as the systematic deviation in model predictions induced by a target entity relative to a population baseline. Scores are normalized and aggregated across tasks to produce a global bias score ( $\Delta(e_i)$ ) for each entity. Positive scores indicate a preference, while negative scores indicate a penalty.

2. Validation Strategy

To address concerns regarding the validity of synthetic data, the authors compare bias measurements derived from their synthetic templates against five real-world benchmarks (MAD-TSC, P-Stance, FinEntity, LIAR). They mask target entities in real data, generate corresponding synthetic corpora, and compute the Pearson correlation between the bias scores.

3. Experimental Scope

The audit covers 1.9 billion data points across:

16 Models: Including LLaMA, Qwen, Aya, and Mistral families, varying in size (7B–72B), architecture, and instruction-tuning status.
3 Languages: English, Russian, and Chinese.
4 Prompting Strategies: Zero-shot and few-shot with textual and numerical labels.

Key Results

1. Validation of Synthetic Proxy

The study confirms that synthetic data serves as a valid proxy for auditing bias. High Pearson correlations (0.74–0.98) were observed between bias scores derived from synthetic templates and real-world benchmarks across different models, languages, and datasets. This indicates that the bias signal depends on how models associate entities with tasks rather than surface-level wording.

2. General Bias Trends

Politics: Models consistently penalize Right-Wing and Far-Right politicians while favoring Left-Wing and Center-Left figures. A significant gender divide exists, with female politicians receiving higher positive scores than males.
Geopolitics: A stark polarization exists between Western industrialized nations (positive bias) and the Global South (negative bias). Wealthier countries (high GDP quantiles) are favored over poorer ones. Notably, Israel exhibits a polarity inversion: positive in general sentiment but negative in International Law Breach detection.
Economics: Western companies are favored over those from the Global South. Sector-specific biases are evident: the retail sector is favored, while defense, pharmaceutical, and real estate sectors face systematic penalties.

3. Structural Drivers of Bias

Language: While the direction of bias remains stable across languages, the magnitude is highest in English. Prompts in Chinese or Russian do not mitigate Western-aligned preferences; in some cases, native-language prompts yield even more negative scores for entities like Russia and China, suggesting biases are inherited from English-dominated training signals rather than being culturally adaptive.
Model Scale: Larger models exhibit stronger bias magnitudes than smaller counterparts.
Instruction Tuning: Instruction tuning reduces the overall magnitude of bias but does not alter the direction of effects. The underlying priors persist, merely constrained in expression.
Prompting: Bias structures are robust to variations in prompting (zero-shot vs. few-shot, text vs. numbers), though few-shot and numerical labels slightly increase bias magnitude.

4. Performance Correlation

The paper notes that bias is associated with degraded performance. Entities with extreme bias scores (either highly positive or negative) tend to have lower F1-scores compared to entities with neutral bias, suggesting that reliance on entity priors rather than textual evidence leads to suboptimal reasoning.

Significance and Claims

The paper claims to present the largest bias audit to date, providing a systematic, large-scale analysis that moves beyond anecdotal observations. Its primary contributions are:

Methodological Innovation: It introduces a scalable framework that uses synthetic data to isolate entity bias with statistical rigor while maintaining ecological validity, validated against real-world benchmarks.
Empirical Evidence: It demonstrates that bias is not an artifact of specific models or prompts but a pervasive structural issue across architectures, languages, and tasks.
Critical Findings: The study highlights that current LLMs project a Western-centric view onto global tasks, penalizing non-Western entities and specific political/industrial sectors regardless of the linguistic context or model alignment.
Practical Utility: The authors position the framework as a necessary pre-deployment audit tool for high-stakes applications (e.g., risk assessment, legal analysis), arguing that hiding these systemic failures ensures they remain unaddressed in deployed systems.

The authors conclude that addressing these limitations requires interventions in dataset design and curation to disentangle descriptive information from embedded reputational and normative stereotypes, rather than relying solely on instruction tuning to mitigate surface-level bias.