💬 NLP

Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity

This paper introduces the VOIR DIRE benchmark to demonstrate that MLLM-as-a-Judge systems suffer from distinct calibration and orientation failures when evaluating culturally ambiguous content, revealing that model biases toward specific cultural norms persist even after correcting for scale compression and cannot be fully resolved by persona prompting or in-context demonstrations.

Original authors: Daniel Lee, Harsh Sharma, Eunkyu Park, Pranav Narayanan Venkit, Jeonghwan Kim, Kah Mun Chia, Andreas Vlachos, Shafiq Joty

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel Lee, Harsh Sharma, Eunkyu Park, Pranav Narayanan Venkit, Jeonghwan Kim, Kah Mun Chia, Andreas Vlachos, Shafiq Joty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a super-smart robot judge to grade student essays or rate photos. Usually, we check if this robot agrees with human teachers to see if it's doing a good job. But this paper asks a tricky question: Which human teachers is the robot agreeing with?

The authors, Daniel Lee and his team, built a special test called VOIR DIRE (named after the legal process of questioning jurors to find hidden biases). They wanted to see if these AI judges have a hidden "cultural bias" when they look at pictures, specifically comparing how they see things through an American lens versus a Mainland Chinese lens.

Here is the story of what they found, explained simply:

1. The Setup: Two Different Worlds

The researchers created 626 pairs of images. Each pair showed the same "concept" (like a celebratory meal, a fashion outfit, or a building) but styled in two different ways: one very American, one very Chinese.

They asked two groups of human experts to rate these images:

Group A: American experts.
Group B: Chinese experts.

The Twist: The experts agreed with each other within their own groups (they were consistent), but they disagreed strongly with each other on the same images.

Example: An image of a convenience store meal might be rated "low quality" by Americans but "fresh and reliable" by Chinese experts. Both groups are "right" based on their own cultural rules, but the numbers are totally different.

2. The Problem: The Robot Judge is "Stuck"

When the AI judges looked at these images, they didn't act like a neutral referee. They made two specific mistakes:

Mistake A: The "Positivity Floor" (The Rubber Band)

Imagine a ruler where the bottom numbers (1 and 2) represent "bad" or "low quality."

The Human Reality: Americans often rate certain Chinese cultural items as "bad" (giving them 1s or 2s).
The Robot's Problem: The AI judges refuse to use the bottom of the ruler. They almost never give a 1 or a 2. They get stuck at a "floor" of about 3.
The Result: Because the AI won't give low scores, it accidentally agrees with the Chinese experts (who rated those items higher) and disagrees with the American experts (who rated them low). The AI isn't "choosing" China; it's just too polite to say "this is bad."

Mistake B: The "Default Setting" (The Compass)

Even when the researchers tried to fix the "politeness" issue by telling the AI, "Pretend you are an American," the AI didn't fully switch gears.

It's like telling a GPS, "Drive to New York," but the car keeps drifting slightly toward Chicago because its internal compass is stuck pointing North.
The AI has a default cultural orientation. Even when told to be American, it still leans slightly toward Chinese cultural norms when judging Chinese images. This "drift" couldn't be fixed by just changing the instructions.

3. The Experiments: Trying to Fix the Bias

The team tried various tricks to see if they could make the AI a fairer judge:

Changing the Persona: They told the AI, "You are a typical American" or "You are a typical Chinese person."
- Result: It helped a little, but the AI still didn't fully switch. It couldn't learn to give low scores to things Americans dislike, even when told to be American.
Showing Examples (In-Context Learning): They showed the AI examples of how humans rated similar pictures before asking it to judge.
- Result: This actually made things worse. Instead of learning to use the full scale (1 to 5), the AI just started giving even higher scores (4s and 5s). It didn't learn to be fair; it just learned to be more enthusiastic.

4. The Big Takeaway

The paper concludes that you cannot just look at one "agreement score" to see if an AI judge is good.

The Old Way: "This AI agrees with humans 80% of the time." (But which humans? Americans? Chinese? Both?)
The New Way: You have to report two scores: "How well does it agree with Americans?" and "How well does it agree with Chinese people?"

If an AI agrees with Americans but disagrees with Chinese people (or vice versa), that isn't a mistake in the data—it's a property of the AI. It reveals which cultural "lens" the AI is wearing.

The Bottom Line

AI judges are like jurors who haven't been fully screened. They have hidden cultural biases that make them "agree" with one group of people while silently disagreeing with another. The paper argues that we need to stop pretending these judges are neutral and start measuring exactly whose culture they are representing.

Key Metaphor:
Think of the AI as a thermometer that is stuck at "Room Temperature."

If you put it in a freezer (a culture that rates things low), it won't drop to "Freezing." It stays at "Room Temp."
If you put it in an oven (a culture that rates things high), it might go up a bit, but it won't get as hot as it should.
The paper says: "Don't just say the thermometer is 'accurate' because it matches the oven. Tell us it's broken because it refuses to measure the cold."

Technical Summary: Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity

Problem Statement

The "LLM-as-a-Judge" and "MLLM-as-a-Judge" paradigms are increasingly used to evaluate open-ended generation tasks where reference-based metrics are insufficient. These systems are conventionally validated by measuring their agreement with human annotations. However, this validation approach contains a critical flaw: it does not specify whose judgments the model approximates. In preference-sensitive domains (e.g., plausibility, appropriateness, desirability), human judgments are not monolithic but are culturally situated. When a judge model is evaluated against a culturally heterogeneous pool, the resulting "agreement" score silently conditions on the specific cultural composition of that pool.

Prior work has documented cultural biases in model generation (e.g., misrepresentation in text-to-image systems), but less attention has been paid to how cultural biases shape model evaluation. This is structurally consequential because judge outputs often serve as training signals for Reinforcement Learning from AI Feedback (RLAIF) or as ground truth for leaderboards. If a judge possesses an unreported cultural lean, it propagates that bias into future training cycles and evaluation metrics.

Methodology: The VOIR DIRE Benchmark

To investigate this, the authors introduce VOIR DIRE, a multimodal benchmark designed to quantify the divergence between MLLM-as-a-Judge alignment and culturally distinct human reference pools under preference-sensitive ambiguity.

Benchmark Construction

Dataset Composition: VOIR DIRE consists of 626 culturally paired image–prompt items. Each item pairs a U.S.-aligned concept with a mainland Chinese-aligned concept sharing the same referent (e.g., a celebratory meal in both contexts).
Categories: The items span three categories: Food and Beverage, Fashion and Human Characteristics, and Architecture and Symbols.
Image Sourcing: Images are generated using Gemini 3 Pro Image with matched parameters (aspect ratio, lighting, composition) to ensure that visual differences are strictly cultural cues rather than aesthetic or compositional variances. A subset of 30 naturalistic photo pairs was also created to verify that findings are not artifacts of synthetic imagery.
Prompt Design: Prompts were authored by bicultural expert annotators to elicit evaluative judgments (not factual lookups) that are plausibly contestable across cultures.
Human Annotation: Ten bicultural-fluent annotators (five U.S., five mainland Chinese) rated items on four dimensions:
1. Q1 (Evaluative Response): A 5-point ordinal scale rating the image against the prompt.
2. Q2 (Cultural Alignment): Rating the item on a 5-point scale from Chinese to American.
3. Q3 (Subjectivity): Rating the subjectivity of the prompt.
4. Q4 (Cross-cultural Recognizability): Rating how likely someone from the other culture would understand the rationale.

Experimental Setup

The authors evaluated six MLLMs (three U.S.-origin: Claude Opus 4.7, GPT-5.5, Llama 4 Scout; three China-origin: GLM-5, Kimi K2.6, Qwen-3-VL-Flash) under ten prompt conditions:

Two neutral conditions (English and Simplified Chinese).
Eight persona conditions crossing persona detail (basic vs. advanced), persona culture (American vs. Chinese), and prompt language.

Additionally, four In-Context Learning (ICL) conditions were tested, varying the source of demonstration examples (U.S. vs. Chinese annotators) and the source of demonstration imagery (American vs. Chinese).

Validation of Cultural Ambiguity

The benchmark validates that items are within-pool reliable (Krippendorff's $\alpha$ = 0.86 for U.S., 0.74 for China) but cross-pool divergent (Pearson $r$ = -0.12 between U.S. and Chinese consensus on Q1). Crucially, the disagreement is over evaluation (Q1), not recognition (Q2), confirming that the pools agree on the content but disagree on its normative value.

Key Results

1. Positivity-Floor Calibration Failure

Models exhibit a "positivity floor," rarely producing ratings of 1 or 2 (only 6–16% of cells). On items where the U.S. consensus is low (≤2), models fail to assign low ratings, instead clustering around a floor of ~3.

Mechanism: This floor mechanically validates the more permissive cultural reading. In this corpus, the Chinese pool is more permissive on contested items; thus, the model's inability to use the lower end of the scale results in a systematic alignment with the Chinese consensus on Chinese imagery.
Asymmetry: The bias is localized. On American imagery, models show a slight U.S. lean. On Chinese imagery, the bias flips and widens, showing a strong lean toward Chinese humans.

2. Orientation Failure (The Residual)

Even when calibration is partially recovered (e.g., via persona prompting), a cultural orientation residual persists.

Persona Prompts: While persona prompts (e.g., "You are an American") shift the judge's tilt in the expected direction, they do not fully correct the calibration failure. On Chinese imagery, five of six models remain below the midpoint of the cultural axis even under U.S. persona conditions.
ICL Demonstrations: Reference-pool demonstrations do not restore calibration. Instead, they widen the rating window upward (increasing 5s) without restoring the use of the low end (1s and 2s). Furthermore, ICL deepens the orientation residual rather than correcting it; adding demonstrations makes the judge more aligned with the demonstration pool's cultural norm, trading alignment with one pool for misalignment with the other.

3. Model Origin as a Covariate

Model origin (U.S. vs. China) adds a small, additive tilt ( $\sim$ 0.10 MAE) that is approximately invariant under demonstration.

Not Pure Competence: A "competence" hypothesis (that Chinese-origin models simply know Chinese culture better) would predict symmetry (CN models align with CN humans, US models with US humans). The data shows no such symmetry: all six systems lean toward Chinese humans on Chinese imagery, regardless of origin.
Hierarchy of Influence: The variance in judge behavior is dominated by the item ( $R^2$ = 0.57), followed by persona configuration, with model origin contributing the least.

Contributions and Significance

The paper makes three primary contributions:

VOIR DIRE Benchmark: A multimodal benchmark of 626 culturally paired items that establishes cultural ambiguity as a property of the items, validated by within-pool reliability and cross-pool divergence.
Decomposition of Bias: The identification of two distinct failure modes in MLLM-as-a-Judge:
- Calibration Failure: The inability to produce low-end ratings (positivity floor).
- Orientation Failure: The default to a specific cultural norm that persists even after calibration attempts.
Methodological Recommendations:
- R1 (Multi-pool Reporting): Alignment should be reported against each reference pool separately. A single "agreement with humans" score is a category error in culturally heterogeneous settings. Cross-pool divergence should be treated as a property of the judge, not noise.
- R2 (Origin as Covariate): Model origin should be reported alongside model identity in evaluations, as it introduces a systematic, albeit small, cultural tilt.

Conclusion

The study concludes that current MLLM-as-a-Judge systems fail to capture the full spectrum of human preference under cultural ambiguity due to a positivity-floor calibration failure and a persistent orientation failure. Neither persona prompting nor in-context demonstrations can fully resolve these issues; in fact, demonstrations often exacerbate the orientation residual. The authors argue that evaluation frameworks must shift from seeking a single "best" alignment score to reporting a profile of alignment across multiple cultural reference pools, treating divergence as a fundamental characteristic of the judge model.