🤖 AI

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

StereoTales is a multilingual framework and dataset comprising over 650,000 stories across 10 languages that reveals how open-ended LLM generation systematically produces culturally adaptive, harmful stereotypes regardless of model size, while demonstrating strong alignment between human and AI judgments of bias.

Original authors: Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao, Stefano Palminteri, Bazire Houssin, Benoît Malézieux, Matteo Dora

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao, Stefano Palminteri, Bazire Houssin, Benoît Malézieux, Matteo Dora

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have 23 different "storytellers" (AI models) from various tech companies. You ask each of them to write a short story about a character, but you only give them one specific detail about that character to start with—like "the character is a retired teacher" or "the character is a young immigrant."

The paper, StereoTales, is a massive experiment to see what other details these storytellers invent on their own. Do they accidentally fill in the blanks with harmful stereotypes?

Here is the breakdown of what they found, using simple analogies:

1. The Experiment: A "Fill-in-the-Blanks" Game

The researchers didn't just ask the AI "Is this sentence biased?" (which is like a multiple-choice test). Instead, they let the AI write free-form stories.

The Setup: They created prompts in 10 different languages (English, French, Arabic, Chinese, etc.) and covered 79 different character traits (age, job, religion, income, etc.).
The Volume: They generated over 650,000 stories.
The Goal: To see if the AI, when left to its own devices, would automatically link certain groups to negative traits (e.g., linking "immigrant" with "poor" or "woman" with "secretary") without being told to do so.

2. Finding #1: The "Bad Habits" Are Everywhere

The Metaphor: Imagine a room full of 23 different chefs. You ask them all to cook a meal. You might expect the "fancy" chefs to be perfect and the "simple" ones to make mistakes.
The Reality: The paper found that every single chef added a pinch of harmful spice to their dish, regardless of how famous or expensive they were.

Universal: From the biggest, most powerful models to the smaller, open-source ones, they all produced harmful stereotypes.
Shared: It wasn't just one "bad apple." The same harmful links appeared across almost all the different companies. It's as if they all learned the same bad habits from the same library of books they were trained on.
Size Doesn't Matter: Making the AI "smarter" or "bigger" didn't stop it from doing this. In fact, the most capable models sometimes produced more harmful links, not fewer.

3. Finding #2: The "Cultural Chameleon" Effect

The Metaphor: Imagine a chameleon that changes its colors not just based on the branch it sits on, but based on the language you speak to it.
The Reality: The AI doesn't just have one set of biases that it translates into different languages. Instead, it adapts its biases to fit the culture of the language you use.

Local Stereotypes: If you ask in English, the AI might stereotype a specific group common in the US. If you ask in Spanish, it might stereotype a different group common in Latin America.
The Danger: This means testing an AI only in English is like checking a chameleon only on a green leaf. You miss all the other colors it shows up on red or blue leaves. The paper argues that an AI might look "safe" in English but be very biased when you switch to another language.

4. Finding #3: The AI Can Judge Itself (But It's a Bit Blind)

The Metaphor: The researchers asked the AI to look at its own stories and say, "Is this harmful?" and compared that to a panel of 247 human judges.
The Reality: The AI and the humans generally agreed (about 62% alignment), but the AI had some funny blind spots.

Over-Caution on Some Things: The AI was very strict about "Gender" and "Sexual Orientation." It knew these were sensitive topics and tried hard not to be biased there.
Blind Spots on Others: The AI was surprisingly lax about things like money, education, age, and religion. It didn't think linking "poor people" to "illiteracy" was as harmful as humans did.
The Takeaway: The AI is good at spotting the stereotypes we talk about (like gender), but it misses the ones we don't talk about as much (like class or education).

5. The Big Picture

The paper concludes that we can't just check if an AI is "fair" by asking it a few multiple-choice questions in English.

Bias is hidden in the stories: It shows up when the AI is writing freely, not just when it's answering a quiz.
Bias is cultural: An AI might be safe in one language but dangerous in another.
Bias is structural: It's not a glitch in one model; it's a feature of how these models are built and trained.

In short: The AI storytellers are like mirrors. If you speak to them in English, they show you American biases. If you speak to them in Arabic or Hindi, they show you local biases from those cultures. And unfortunately, almost every mirror they hold up reflects some harmful stereotypes, no matter how "smart" the mirror claims to be.

Technical Summary: StereoTales

Problem Statement

Existing research on social bias in Large Language Models (LLMs) is predominantly constrained by three limitations:

Format: Most benchmarks rely on tightly controlled settings (multiple-choice, template infilling, or question-answering) that test bias recognition rather than bias generation. This fails to reflect the open-ended, free-form generation typical of real-world LLM usage.
Scope: Studies are overwhelmingly English-centric and US-focused, often limiting socio-demographic attributes to gender and race based on US Census categories. This ignores how biases manifest across different languages and cultural contexts.
Granularity: Prior work often scores individual outputs or focuses on small, open-weight models, leaving the behavior of frontier models and the distributional emergence of biases (patterns visible only across hundreds of generations) largely unexplored.

Consequently, biases that emerge only in open-ended generation, across diverse languages, and in recent frontier models remain uncharted.

Methodology

The authors introduce StereoTales, a multilingual dataset and evaluation pipeline designed to systematically discover social bias in open-ended LLM generation.

Dataset Construction

Scale: The dataset comprises approximately 650,000 stories generated by 23 frontier LLMs from major providers (OpenAI, Anthropic, Google, Mistral, Alibaba, etc.).
Languages: Generation occurs in 10 languages (English, French, Spanish, Italian, Portuguese, Dutch, Ukrainian, Arabic, Hindi, and Chinese). Crucially, prompts, attribute values, and scenarios were manually translated by native speakers to avoid semantic drift and cultural flattening associated with automatic translation.
Prompt Design: Prompts are constructed via a Cartesian product of:
- 19 Demographic Attributes: Covering identity, socio-economic status, and life situations (e.g., age, gender, religion, education, income, immigration status).
- 79 Attribute Values: Closed sets of admissible values for each attribute.
- 36 Narrative Scenarios: Diverse situations (e.g., job interviews, family dinners, health crises) serving as narrative seeds.
Generation Protocol: Each prompt fixes a single attribute value for the protagonist while leaving others unspecified. The model generates a ~200-word story.

Analysis Pipeline

Attribute Extraction: An ensemble of three LLM extractors (GPT-5-nano, Gemini 2.5 Flash Lite, Gemini 3.1 Flash Lite) extracts the socio-demographic profile of the protagonist from each story across all 19 dimensions.
Statistical Association Discovery: The pipeline builds contingency tables for co-occurrences between the prompt-imposed attribute (Base) and the extracted attributes.
- Attribute Level: Fisher's exact test (Monte Carlo estimation) determines if a distribution depends on the base attribute. Effect size is measured via bias-corrected Cramér's V.
- Value Level: One-sided Fisher tests identify specific value pairs driving the association. Effect size is measured by Lift ( $P(B=b|A=a)/P(B=b)$ ). Only pairs with $Lift \ge 2$ and statistical significance (Benjamini-Yekutieli corrected) are retained.
Harmfulness Judgment: To avoid injecting author biases, harmfulness is determined by two independent evaluators:
- Human Study: 247 UK-based participants rated 1,580 unique associations on a 5-point Likert scale for harmfulness and frequency in the real world.
- LLM Self-Evaluation: The same 23 models that generated the stories were prompted to rate the harmfulness of the same associations.

Key Contributions

StereoTales Dataset: A large-scale, multilingual resource containing ~650k stories, attribute annotations, and harmfulness ratings, covering 10 languages and 23 models.
Distributional Bias Discovery: A methodology that moves beyond sample-level classification to identify systematic, statistically significant associations across the distribution of model outputs.
Cross-Model and Cross-Language Evaluation: The first study to jointly evaluate bias universality across 23 frontier models and the cultural adaptability of biases across 10 languages.
LLM-as-Judge Calibration: A large-scale calibration of LLM self-evaluation against human judgment in the domain of stereotype harmfulness.

Key Results

1. Pervasiveness and Universality of Harmful Stereotypes

Ubiquity: Every model evaluated emits consequential harmful stereotypes in open-ended generation, regardless of size or capability.
Shared Patterns: Harmful associations are largely shared across providers rather than being isolated misbehaviors. Approximately 25% of harmful associations appear in at least half of the models, and a significant portion is produced by every single model.
Capability Independence: There is no detectable reduction in harmful associations with increased model size. Surprisingly, a weak positive correlation exists between model capability (ELO) and the number of harmful associations, suggesting that current alignment recipes do not effectively mitigate these biases.

2. Language-Specificity and Cultural Adaptation

Cultural Mirroring: Models do not transfer a static, English-dominant set of biases. Instead, they act as "cultural chameleons," adapting stereotypes to the cultural frame evoked by the prompt language.
Local vs. Global: Roughly half of harmful emissions are confined to at most two languages.
Target Shifting: The prompt language shifts which groups are targeted. Models tend to reduce harmful associations about the "unmarked" (majority) identity of the prompt language while increasing harms against locally salient protected minorities.
Regional Clusters: Harmful biases cluster by cultural region (e.g., a strong West-European block and a weaker Latin-American block), indicating that training data and cultural context, rather than just language family, drive bias profiles.

3. Alignment Between Human and LLM Judgments

Broad Alignment: LLM judgments are broadly aligned with human ratings (Spearman $\rho = 0.62$ ).
Systematic Blind Spots: Disagreements are systematic and shared across providers. Models consistently underestimate the harm of stereotypes anchored in socio-economic status, age, religion, immigration, and education. Conversely, they overestimate harm regarding gender and gender alignment—the very axes that have historically been the focus of safety alignment.
Conservative Bias: LLMs are generally more conservative than humans, using the extreme "5" rating significantly less often.

Significance and Claims

The paper claims that monolingual fairness benchmarks substantially underestimate the harm a model can emit when prompted in another language. Because biases adapt culturally to the prompt language, a model deemed "safe" in English may emit distinct, locally harmful stereotypes in other languages.

Furthermore, the study challenges the notion that bias recognition benchmarks (which frontier models often saturate) imply that bias generation is solved. The results indicate that harmful association generation is a structural property of current LLMs, likely inherited from shared pretraining data and alignment recipes, rather than a failure of scale or capability.

Finally, the paper posits that while LLMs can serve as complementary evaluators for bias, their self-judgment is fragile and exhibits systematic blind spots on socio-economic and ideological attributes, making them an insufficient replacement for human evaluation in these specific domains.

The authors release the full dataset, code, and annotation platform to enable the community to extend this work to new languages, attribute taxonomies, and normative reference panels.