💬 NLP

LLM Evaluators are Biased across Languages

This paper demonstrates that multilingual LLM evaluators exhibit a consistent, statistically significant bias where lower-resource languages receive higher scores and pass safety filters more easily than higher-resource languages, a critical flaw that remains undetected by standard pairwise accuracy metrics despite the evaluators' high agreement on relative rankings.

Original authors: Ej Zhou, Lucas Resck, Zheng Hui, Anna Korhonen

Published 2026-07-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ej Zhou, Lucas Resck, Zheng Hui, Anna Korhonen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where every time you ask a question to a super-smart robot, it answers you in your own language. These robots, called Large Language Models (LLMs), are like digital encyclopedias that can write stories, solve math problems, and chat about anything. But how do we know if they are doing a good job? We need a referee. In the world of AI, these referees are called "evaluators." They are either other robots trained to give scores or human-like prompts that ask the AI to grade itself. Usually, we check if these referees are fair by seeing if they can correctly pick the "better" answer between two options, kind of like a judge deciding which of two paintings is prettier. This is called "pairwise accuracy." If the referee picks the right winner 90% of the time, we assume they are doing a great job and that their scoring is fair no matter what language is being spoken.

However, there's a hidden problem with this assumption. Just because a referee can tell you which of two paintings is better doesn't mean they are using the same ruler to measure them. In the real world, a 4-star restaurant in Tokyo might be considered a culinary masterpiece, while a 4-star restaurant in New York might be seen as just "okay." The stars mean different things depending on where you are. This paper, titled "LLM Evaluators are Biased across Languages," dives into whether our AI referees suffer from the same cultural confusion. The researchers wanted to know: if an AI gives a perfect answer in English, does it give that same answer the exact same score if the answer is translated into Hindi, Farsi, or Ukrainian? Or does the language itself change the score, even if the meaning stays exactly the same?

The team set up a massive experiment to find out. They took the exact same set of instructions and responses and translated them into 23 different languages, ranging from widely spoken ones like English and Spanish to others like Hebrew and Hindi. They then asked eight different types of AI evaluators—some that are just prompted to give a score, and others that are specially trained to give a "reward" number—to grade these identical answers.

The results were a shock. The evaluators were not using the same ruler. Even though the content was semantically identical (meaning it meant the exact same thing), the scores shifted wildly depending on the language. The bias was huge: on a scale of 1 to 5, the difference between the highest-scoring languages and the lowest-scoring ones was about 0.5 points. That's a massive gap in the world of grading.

Here is the twist that makes this so tricky: the evaluators still looked perfect on paper. When the researchers checked the "pairwise accuracy" (did the referee pick the better answer?), the scores were incredibly high, often above 90%. The referees were consistently good at saying "Answer A is better than Answer B." But they were terrible at being consistent about how good Answer A actually was. It's like a judge who always correctly picks the winner of a race but gives the winner 100 points if they run in French and only 50 points if they run in German.

The paper discovered a clear pattern: the evaluators were being surprisingly generous to lower-resource languages (languages with less data available online, like Hindi or Hebrew) and stricter with high-resource languages like English. It's as if the AI judges think, "I'm not 100% sure what a perfect answer looks like in this language, so I'll just give it a high score to be safe." Conversely, for English, they felt very confident and strict, holding the answers to a higher standard.

This isn't just a numbers game; it has real-world consequences. Many AI systems use a "global threshold" to decide what is safe to show to users. For example, if a safety filter is set to block anything with a score below a certain number, the bias means that harmful content in lower-resource languages is much more likely to slip through the cracks. The paper found that under a single global rule, the acceptance rate for content could vary by up to 43 percentage points between languages. In one specific example, a harmful question in English was correctly blocked, but the exact same question in Ukrainian was allowed to pass because the evaluator gave it a higher score.

The researchers also investigated why this happens. They suspected it might be because the AI is less certain about lower-resource languages. They found that when the AI was less confident (measured by how surprised it was by the text), it did tend to give higher scores. However, they proved that uncertainty wasn't the whole story. Even after accounting for how confused the AI was, the language itself still predicted the score. This means the bias is a deep, structural issue in how these models are built, not just a simple case of "I don't know this language well."

The paper concludes that we cannot simply trust raw scores from these AI evaluators across different languages. While we can fix some of the problem by adjusting the scores for each language, this creates a new vulnerability: if someone mixes languages in a single prompt (code-switching), the system might get confused about which language rule to apply and let harmful content slip through. The authors suggest that instead of just checking if evaluators can pick the "winner," we need to build systems that are calibrated to be fair and consistent across all languages, ensuring that a 5-star answer in Hindi is truly as valuable as a 5-star answer in English.

Technical Summary: LLM Evaluators are Biased across Languages

1. Problem Statement

Large Language Model (LLM) evaluators, encompassing both prompted "LLM-as-a-Judge" systems and trained reward models, are standard tools for automated benchmarking, safety auditing, and Reinforcement Learning from Human Feedback (RLHF). These systems are routinely validated using pairwise accuracy, which measures whether an evaluator correctly ranks a preferred response over a rejected one.

The prevailing assumption in multilingual settings is that high pairwise accuracy implies language-neutral scoring: semantically identical content should receive the same absolute score regardless of the language in which it is expressed. This paper challenges that assumption, demonstrating that LLM evaluators exhibit systematic, language-dependent shifts in pointwise scores. Consequently, evaluators that appear robust under pairwise metrics induce substantial disparities in threshold-based decisions, particularly disadvantaging or mis-calibrating lower-resource languages.

2. Methodology

Datasets and Scope

The authors conducted experiments across 23 languages using semantically identical instruction–response pairs. The data was drawn from RewardBench and its human-validated multilingual extension, M-RewardBench. These benchmarks cover four distinct categories: Chat, Chat-Hard, Safety, and Reasoning, ensuring the findings span heterogeneous task formats and domains. All data in M-RewardBench was professionally translated and human-validated to preserve semantic parallelism, avoiding machine translation artifacts.

Evaluators

The study evaluated eight distinct models representing different architectures and training paradigms:

LLM-as-a-Judge (4 models): Aya Expanse 32B, Qwen 2.5 72B, LLaMA 3.1 70B Instruct, and M-Prometheus 14B. These were prompted with a standard 1–5 Likert-scale rubric.
Reward Models (4 models): URM-LLaMA-3.1-8B, BTRM-Qwen-2-7B, Skywork-Reward-Gemma-2-27B, and Skywork-Reward-LLaMA-3.1-8B-v0.2. These output continuous scalar scores without explicit rubrics.

Experimental Design

Scoring Shift Analysis: The authors compared mean pointwise scores across languages for identical content.
Pairwise vs. Absolute Validation: They analyzed the relationship between pairwise accuracy and acceptance rates under a fixed global decision threshold.
Mechanism Investigation: To understand why lower-resource languages received higher scores, the authors modeled the relationship between model uncertainty (measured via Negative Log-Likelihood, NLL) and reward scores. They employed nested Ordinary Least Squares (OLS) regressions to decompose score variance into components explained by uncertainty, language identity, and their interaction.
Attack Surface Analysis: They tested the vulnerability of per-language thresholding strategies against code-switched prompts.

3. Key Results

Systematic Scoring Bias

Directional Shift: Evaluators assign significantly different absolute scores to semantically identical content based on language. The bias is substantial, amounting to a ~0.4–0.5 point shift on a 1–5 scale.
Resource Correlation: Contrary to the expectation that models perform worse in lower-resource languages, evaluators score lower-resource languages more generously. There is a strong negative correlation between language resource level (training data availability) and the strictness of scoring (Spearman $\rho = -0.81$ for reward models). Languages like Hindi, Hebrew, and Arabic receive the highest scores, while English, Italian, and French receive the lowest.
Cross-Model Consistency: This bias is not an artifact of a single model. It persists across all eight evaluators, including frontier models (GPT-4.1-mini, Qwen3-32B), and is consistent across different architectures (LLaMA, Qwen, Gemma) and training paradigms.

The Blind Spot of Pairwise Accuracy

High Accuracy, Divergent Outcomes: Evaluators maintain uniformly high pairwise accuracy (>90%) across all languages. However, under a fixed global decision threshold (e.g., the 50th percentile), acceptance rates vary by up to 43 percentage points across languages.
Safety Implications: This discrepancy means harmful content in lower-resource languages is significantly more likely to pass safety filters than identical content in high-resource languages. For example, a rejected English response might score -4.51 (filtered), while its Ukrainian counterpart scores +3.57 (passed) despite identical semantic content.

Mechanism: Uncertainty vs. Structural Bias

Uncertainty Correlation: There is a strong positive correlation between model uncertainty (higher NLL) and reward scores. Models tend to give higher scores when they are less confident about the content.
Uncertainty is Not the Sole Cause: While uncertainty explains part of the variance, language identity remains a significant predictor even after controlling for NLL.
Structural Misalignment: Within-language regressions show that the relationship between instance difficulty (NLL) and score is inconsistent and domain-dependent (often negative in Chat tasks). This confirms the bias is a structural, language-level misalignment rather than a simple reaction to content difficulty.

Vulnerability of Mitigation Strategies

Code-Switching Attacks: A proposed mitigation—using per-language thresholds—requires reliable Language Identification (LID). The authors demonstrate that code-switched prompts (e.g., an English wrapper around Hindi content) can defeat LID tools. If the system misidentifies the language as English, it applies a lenient English threshold to content that the model actually scores against a stricter Hindi baseline, artificially inflating acceptance rates from 50% to 75%.

4. Contributions

Language-Dependent Scoring Shifts: The paper demonstrates that LLM evaluators exhibit systematic, cross-model-consistent bias in pointwise scores, where lower-resource languages are scored more generously. This bias persists in frontier judges.
Blind Spot of Pairwise Accuracy: It establishes that pairwise accuracy is structurally blind to these shifts, masking severe disparities in acceptance rates under global thresholds. It highlights that per-language thresholds are vulnerable to code-switched inputs.
Mechanism Analysis: The study investigates the "why," finding that while model uncertainty correlates with the bias, language identity remains predictive after controlling for uncertainty. The effect cannot be reduced to instance-level difficulty alone.

5. Significance and Claims

The authors argue that the current reliance on pairwise accuracy as the primary validation metric for multilingual evaluators is insufficient. The findings suggest that uncalibrated cross-lingual biases pose a risk to the deployment of lower-resource languages in safety auditing and RLHF.

Practical Implication: Treating raw evaluator scores as comparable across languages without calibration is dangerous. While per-language offset corrections can reduce acceptance-rate gaps by ~60%, they do not eliminate the structural variance (language $\times$ item interactions) and introduce new attack surfaces via LID failures.
Future Direction: The paper calls for a shift from post-hoc correction to principled solutions in training, such as language-aware calibration of reward models, normalization of language-conditioned scores during RLHF, and evaluation protocols that explicitly test cross-lingual score consistency alongside pairwise accuracy.

The work concludes that the community must recognize multilingual bias in LLM evaluators as a structural issue, moving beyond relative ranking metrics to ensure fair and reliable absolute scoring across languages.