🤖 AI

Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

This paper introduces INCLUDE, a multilingual benchmark evaluating Indian-centric socio-cultural biases across six languages, revealing that current English-centric safety alignment in LLMs creates a critical cross-lingual safety gap where non-English outputs, particularly in Bengali and closed-source models, exhibit significantly higher bias than their English counterparts.

Original authors: Namya Bhatnagar

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Namya Bhatnagar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where the voice of a computer assistant is as fluent and fair in a rural village in India as it is in a bustling office in London. For years, the technology behind these assistants has been trained primarily on English text, learning to recognize harmful stereotypes and refuse offensive requests through a process called safety alignment. This training acts like a filter, designed to stop the machine from repeating biases about race, religion, or social status. However, a critical question has remained unanswered: does this filter work just as well when the user speaks Hindi, Bengali, or a mix of English and Hindi? If the filter only understands English, then millions of people speaking other languages are left exposed to the very biases the technology was meant to prevent. This is the core concern of a new study that investigates whether the promise of safe artificial intelligence holds true across the diverse linguistic landscape of India.

The researchers, led by Namya Bhatnagar, set out to measure this gap by creating a specialized test called INCLUDE. This benchmark was designed to see how well different artificial intelligence models handle culturally specific stereotypes when prompted in six different languages: English, Hindi, Bengali, Marathi, Tamil, and Hinglish, which is a common mix of Hindi and English used in daily conversation. To build this test, the team first defined six areas where bias often appears, such as caste, religion, and region. They then gathered a panel of twenty experts, including teachers and university faculty, to review and refine thousands of questions. These questions were carefully crafted to sound natural, asking the models to complete sentences in ways that would reveal if they were leaning toward harmful stereotypes or rejecting them. The team translated these prompts into the five Indian languages, ensuring that the meaning and the intensity of the stereotypes remained consistent across all versions.

The study evaluated ten different artificial intelligence models, splitting them into two groups: those that are open for anyone to inspect and modify, and those that are closed systems owned by large technology companies. When the researchers tested the open-source models, they found a clear and troubling pattern. The models were significantly more likely to produce biased, stereotypical answers when the prompts were written in Indian languages compared to English. Among the Indian languages, Bengali triggered the highest level of bias, followed closely by Hindi. Surprisingly, English prompts actually resulted in the lowest bias scores for these open models, suggesting that the safety filters are working well for English speakers but failing for everyone else. This creates an illusion of safety where a user might feel protected in English but is left vulnerable in their native tongue.

The results for the closed-source models, which are often used to power the search and retrieval functions of voice assistants, told a different and even more complex story. In this group, the pattern flipped completely. The models showed the strongest bias associations when the prompts were in English, while the Indian languages showed lower bias scores. This reversal suggests that the source of the problem is not just one thing. For the open models, the issue appears to be that the safety training was done mostly in English, leaving other languages unguarded. For the closed models, the bias seems to come from the massive amount of English text they were originally trained on, which has embedded strong stereotypes into their core understanding of the world.

One of the most significant findings concerned Hinglish, the code-mixed language used by millions of people in India. The study found that Hinglish did not get the safety benefits of English. Even though it contains English words, the models treated it statistically the same as the other low-resource Indian languages, grouping it with Hindi and Tamil rather than with English. This means that users who rely on this mixed language for voice interaction are not receiving the same level of protection as those speaking pure English. The researchers also noted that while the bias scores varied, the differences were not random; they were consistent enough to be considered a real, measurable failure in how these systems are built.

The study concludes that the safety of artificial intelligence is not uniform across languages. It reveals that the current methods of training these models create a divide where English speakers are protected from harmful stereotypes, while speakers of other languages are not. The findings suggest that simply translating safety rules from English is not enough; the very structure of how these models learn and how they are aligned needs to change to account for the nuances of different cultures and languages. By identifying these gaps, the researchers hope to provide a roadmap for developers to build systems that are truly fair for everyone, regardless of the language they speak. The work highlights that achieving true fairness in artificial intelligence requires looking beyond the dominant languages and addressing the specific cultural contexts in which these technologies are deployed.

Technical Summary: Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

Problem Statement
Current safety alignment training for Large Language Models (LLMs) is heavily English-centric, creating a "Safety Alignment Illusion." While models may pass rigorous safety audits in English, they frequently fail to extend these protections to low-resource and code-mixed languages. This gap is particularly critical for spoken language technologies (e.g., voice assistants) deployed in linguistically diverse regions like India. Users with limited literacy or those relying on voice interaction due to the cumbersome nature of non-Latin scripts are disproportionately exposed to stereotype-reinforcing outputs and harmful biases when interacting in regional languages or code-mixed registers like Hinglish. Existing benchmarks (e.g., StereoSet, CrowS-Pairs) are English-focused, while prior Indian-centric studies have been limited in linguistic scope or lack evaluation of code-mixed languages.

Methodology
To address this gap, the author introduces INCLUDE (Indian Cultural Lens for Understanding and Detecting Embedded Biases), a multilingual evaluation benchmark designed to quantify Indian-centric socio-cultural biases.

Dataset Construction: INCLUDE comprises 2,604 prompts spanning six languages: English, Hindi, Bengali, Marathi, Tamil, and Hinglish (Hindi-English code-mix). The prompts target six bias axes: Caste, Intersectionality, Region, Religion, Socioeconomic Status (SES), and Spoken Language, covering 27 identity groups.
Validation: A 20-person Expert Panel (teachers, faculty, native speakers) iteratively validated bias segments for stereotypical accuracy. Translations into five low-resource languages were performed via an automated GPT-4o pipeline and rigorously validated by bilingual annotators using a 5-point Likert scale and back-translation. Inter-annotator agreement was assessed using weighted Cohen's $\kappa$ , ranging from "Fair" (Bengali) to "Almost perfect" (Hinglish, Tamil).
Evaluation Metrics:
- Open-Source Models (6 models: TinyLlama, Qwen, MistralAI, DeepSeek, GPT-2, Gemma): Evaluated using conditional log-likelihood scoring. The bias score ( $\delta_i$ ) is calculated as the difference in loss between anti-stereotypical and stereotypical completions. Positive scores indicate a preference for stereotypes.
- Closed-Source Models (4 embedding models: Voyage-3.5, Voyage-3-Large, OpenAI text-embedding-3-large, Cohere embed-multilingual-v3.0): Evaluated using the Word Embedding Association Test (WEAT) cosine similarity. This measures the association strength between target identity vectors and stereotypical vs. anti-stereotypical attribute vectors.
Statistical Analysis: Due to the repeated measures nature of the data (same prompts across languages), standard ANOVA was avoided. Instead, the study employed linear mixed-effects models and non-parametric Friedman tests for overall group differences, followed by Wilcoxon signed-rank tests with Holm correction for pairwise comparisons.

Key Contributions

Exposure of Asymmetry in Open-Source Models: The study demonstrates that all five low-resource Indian languages elicit significantly higher bias than English in open-source models. This confirms that post-training safety alignments (e.g., Constitutional AI) are disproportionately effective for English, leaving regional language speakers without meaningful algorithmic protection.
Discovery of the "English Reversal" in Closed-Source Models: In contrast to open-source models, closed-source embedding models exhibit a reversal where English prompts elicit the strongest stereotype associations. This suggests that pretraining corpus composition (the dominance of English text) introduces a distinct bias pathway that operates independently of, and often contrary to, post-training alignment.
Characterization of Hinglish: The benchmark reveals that Hinglish, despite its lexical similarity to English, is statistically indistinguishable from low-resource Indian languages in terms of bias profiles. It inherits the full bias risk of low-resource languages rather than the safety benefits of English, highlighting that code-mixing does not automatically confer safety alignment.

Results

Open-Source Findings: Bengali yielded the highest average bias score (ABS = 1.46), followed by Hindi (1.41) and English (1.40). Statistical analysis confirmed that English differs significantly from all other languages, eliciting lower bias. Bengali was uniquely distinct from Hindi within the Indian cluster.
Closed-Source Findings: A significant language-level variation was observed (stronger than in open-source models). English targets showed significantly higher bias scores than Indian languages, particularly Bengali. This confirms that embedding spaces encode stronger stereotype-attribute associations for English targets in the Indian context.
Hinglish Specifics: Hinglish had the lowest Average Bias Score (ABS) among all languages in open-source models but remained statistically grouped with the Indian languages rather than English. This indicates that while the English lexicon triggers some safety mechanisms, the code-mixed syntactic structure retains the bias patterns of low-resource languages.

Significance and Claims
The paper claims that INCLUDE provides a rigorous framework for detecting cross-lingual safety gaps that are currently invisible to English-centric audits. The findings challenge the assumption that safety alignment is language-agnostic, revealing two distinct mechanisms of bias:

Post-training alignment gaps in generative models that fail to protect low-resource languages.
Pretraining corpus biases in embedding models that encode stronger stereotypes for English targets.

The author asserts that these findings are critical for the development of equitable voice assistants and spoken language technologies in India. By demonstrating that code-mixed interactions (Hinglish) do not inherit English safety benefits, the work argues for the necessity of culturally informed evaluation frameworks. The study positions INCLUDE as a step toward democratizing access to fair AI, offering a practical tool for researchers and developers to identify and mitigate biases that reinforce societal inequalities in non-Western contexts. The paper concludes modestly, noting that future work should extend these evaluations to multimodal systems, diverse global contexts, and other code-mixed languages, and suggests that professional human translation could further strengthen validity.

Technical Summary: Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

More like this