Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judge

This study reveals that LLMs used as automated judges exhibit significant scoring inconsistencies across different models, temperatures, and repeated runs, challenging their reliability for enterprise workflows and highlighting the need for robust monitoring and hybrid evaluation strategies.

The Gist

Imagine you have hired five different super-tasters to judge a new batch of cookies. You give them all the exact same cookie and ask them to rate it on a scale of 0 to 10 for three things: Taste, Freshness, and How Full the Box Is.

You expect that if you ask the same taster to judge the same cookie ten times in a row, they will give you the same score every time. You also expect that if you ask five different tasters to judge the same cookie, their scores should be somewhat similar.

This paper is basically a report card on what happens when you actually try this experiment with AI models (the "tasters") instead of humans. The author, Fiona Lau, discovered some very surprising and slightly worrying things.

Here is the breakdown in simple terms:

1. The "Ghost in the Machine" Problem (Inconsistency)

Even when you tell the AI, "Be as serious and boring as possible" (which is what setting the temperature to 0 means in AI terms), the AI still acts like a moody artist.

• The Analogy: Imagine asking a friend, "How much do you like this pizza?" ten times in a row while they are staring at the same slice. You'd expect them to say "8/10" every time. But with these AIs, they might say "8/10" the first time, "6/10" the second time, and "9/10" the third time, even though nothing changed.
• The Finding: The study found that no model was truly consistent. They gave different scores for the exact same answer, over and over again. This is a big problem for businesses that rely on these scores to make decisions.

2. The "Strict vs. Generous" Judges (Different Models, Different Scores)

The study tested five famous AI models (from OpenAI, Google, and Anthropic). They are like five different judges on a TV competition show.

• The Analogy: Think of Judge A as a grumpy food critic who hates everything and gives low scores. Judge B is a super-nice grandma who thinks everything is delicious and gives high scores.
• The Finding:
  • Google's Gemini was the "Grandma." It was very generous, especially with the "Completeness" score (did the answer cover everything?).
  • Anthropic's Claude was the "Grumpy Critic." It was much stricter.
  • The Result: If you ask the same question to these different models, you could get a "Pass" from one and a "Fail" from the other. This means the choice of which AI you use changes the outcome of your business logic.

3. The "Completeness" Trap

The study looked at three specific criteria: Relevance (did it answer the question?), Accuracy (is it true?), and Completeness (did it miss anything?).

• The Analogy: Imagine a student answering a math problem.
  • Relevance: They wrote about math. (Good)
  • Accuracy: The math is correct. (Good)
  • Completeness: They forgot to write the final "Therefore, x = 5." (Bad)
• The Finding: The AI models were the worst at judging Completeness. They couldn't agree on whether an answer was "full" or "missing pieces." It was like asking five people if a glass is "half full," and they all give you different answers. This is dangerous because in business, "completeness" often determines if a customer gets a refund or if a ticket gets escalated.

4. The "Temperature" Knob (Does turning it down help?)

In AI, "Temperature" is a dial that controls how random the AI is.

• High Temperature (1.0): The AI is creative, wild, and takes risks.

• Low Temperature (0.0): The AI is supposed to be robotic, logical, and repeatable.

• The Analogy: It's like telling a jazz musician to "play the same note exactly the same way every time."

• The Finding:

  • For OpenAI (GPT) and Google (Gemini), turning the dial down to 0 helped a lot. They became much more consistent.
  • For Anthropic (Claude), turning the dial down barely helped. They were still inconsistent even when told to be robotic.
  • The Lesson: You can't just "fix" inconsistency by turning a setting to zero. It depends entirely on which AI you are using.

Why Should You Care? (The Real-World Impact)

You might think, "So the AI gives a slightly different score? Who cares?"

Here is why it matters:
Imagine a bank uses an AI to decide if a loan application is "Good" or "Bad."

• Scenario A: The AI gives a score of 85 (Approved).
• Scenario B: The same AI, looking at the same application 10 minutes later, gives a score of 45 (Rejected).

If this happens, the bank is being unfair. One customer gets a loan, and another identical customer gets rejected, just because the AI had a "bad day" or a "random thought."

The Bottom Line

The paper concludes that we cannot blindly trust AI to be a "Judge" yet.

• Don't trust the score: A score of 80 today might be a 60 tomorrow.
• Watch the "Completeness": This is the most unstable metric.
• Mix it up: If you need reliability, don't rely on just one AI. You might need a "hybrid" system where a human double-checks the AI, or you use multiple AIs to vote on the answer.

In short: AI is a powerful tool, but right now, it's a bit like a weather forecast that changes its mind every time you ask it the same question. We need to learn how to live with that uncertainty before we let it make big decisions for us.

Technical Summary

Here is a detailed technical summary of the paper "Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judge."

1. Problem Statement

The paper addresses a critical gap in the deployment of LLM-as-a-Judge systems within enterprise environments. While Large Language Models (LLMs) are increasingly used to automate the evaluation of other LLM outputs or user-generated content (e.g., for routing, triage, and quality control), their scoring consistency remains poorly understood.

• The Core Issue: Existing research focuses on accuracy, bias, and alignment with human preferences, but few studies examine the stability of numerical scores assigned to identical inputs.
• The Risk: In production pipelines, if an identical input receives different scores across repeated runs or different model families, it leads to inconsistent downstream actions (e.g., a query being routed differently, a customer receiving different service levels, or a system gating decision flipping).
• The Gap: Prior work on non-determinism mostly focused on multiple-choice tasks or selection-based judgments. This study targets score-based judgment (assigning a 0–1 value to qualitative criteria), which is prevalent in enterprise RAG (Retrieval-Augmented Generation) systems but lacks empirical stability analysis.

2. Methodology

The study employs a systematic empirical evaluation using a real-world enterprise dataset.

• Dataset: Real question–answer pairs sourced from an internal enterprise RAG system. The data spans diverse categories and authentic user intents. (Note: The dataset is proprietary and not publicly released to prevent benchmark contamination and protect privacy).
• Evaluation Criteria: Three specific metrics were used to score the answers:
  1. Relevance: Does the answer address the question?
  2. Accuracy: Is the information factually correct?
  3. Completeness: Does the answer fully cover the necessary information? (This is highlighted as crucial for assessing RAG retrieval sufficiency).
• Models Evaluated: Five widely used models across three major families:
  • OpenAI: GPT-4o, GPT-4o-mini
  • Google: Gemini-2.5-Flash
  • Anthropic: Claude-Haiku-4.5, Claude-Sonnet-4.5
• Experimental Design:
  • Temperature Settings: Each model was run at Temperature = 0 (theoretically deterministic) and Temperature = 1 (stochastic) to isolate the effect of sampling randomness.
  • Repetition: Each model evaluated the same set of questions 10 times per temperature setting.
  • Prompting: A fixed, structured prompt (Appendix 1) instructed models to provide a score (0–1) and a textual justification for each criterion.

3. Key Contributions

• Empirical Evidence of Non-Determinism in Scoring: The study provides concrete data showing that even under "deterministic" settings (Temperature=0), LLMs produce varying numerical scores for identical inputs.
• Metric-Specific Variability: It identifies that Completeness is the most volatile metric, exhibiting significantly higher variance than Relevance or Accuracy across all models.
• Cross-Model Divergence: It demonstrates that different model families interpret the same criteria differently. For example, one model might rate an answer as "Complete" while another rates it "Partial" based on differing interpretations of what constitutes a "follow-up suggestion."
• Temperature Inefficacy: It challenges the assumption that lowering temperature guarantees stability, showing that the effect is highly model-dependent.

4. Key Results

RQ1: Within-Model Consistency (Repeated Runs)

• No True Determinism: Even at Temperature=0, models exhibited score variability.
• Metric Variance: Completeness showed the highest standard deviation. Relevance and Accuracy were more stable but still inconsistent.
• Model Performance:
  • Gemini: Showed the most dramatic improvement in stability when moving from Temp=1 to Temp=0 (variance dropped to near zero in many cases).
  • Anthropic (Claude/Sonnet): Showed minimal improvement at Temp=0; in some cases (e.g., Claude on Completeness), variability remained high or even increased.
  • OpenAI (GPT): Showed moderate improvement with lower temperatures.

RQ2: Cross-Model Consistency (Different Models, Same Input)

• Systematic Bias: Models differ in "strictness" and "interpretation."
  • Gemini tended to be more generous with Completeness scores.
  • Anthropic models were stricter, often assigning near-zero scores to answers that OpenAI and Gemini rated positively, citing misalignment with user intent.
• Case Study (Question 21): For a specific question, Gemini rated an answer as "1.0" (Complete) because it explained a limitation well. GPT-4 rated it "0.5" (Partial) because it lacked follow-up suggestions. This divergence highlights how model choice dictates workflow outcomes.

RQ3: Impact of Temperature

• Uneven Effects: Lowering temperature reduces variance for GPT and Gemini models but has limited or inconsistent effects for Anthropic models.
• Stability vs. Direction: Temperature changes affect the stability (variance) of scores but not necessarily the mean score (tendency). A model might be consistently wrong (stable) or inconsistently right/wrong (unstable).

5. Significance and Implications

• Operational Reliability: The findings warn against relying solely on LLM-generated scores for critical business logic (routing, gating, triage). Identical inputs triggering different scores can lead to unfair customer experiences and operational errors.
• Legal and Ethical Risks: Inconsistent scoring raises concerns regarding fairness, auditability, and potential legal liability (e.g., in legal claims or safety-critical maintenance), as the "reason" for a decision may change randomly.
• Recommendations for Practitioners:
  • Monitor Variance: Organizations should evaluate models not just by mean accuracy but by standard deviation (consistency).
  • Robust Parsing: Relying on regex for score extraction is fragile due to formatting inconsistencies; secondary parsers or extraction agents are recommended.
  • Hybrid Approaches: Critical decisions should involve human-in-the-loop verification or ensemble methods to mitigate non-determinism.
  • Model Selection: Temperature=0 is not a universal fix; model architecture plays a larger role in stability than previously thought.

Conclusion

The paper concludes that non-determinism is a practical source of instability in LLM-as-a-Judge systems, not just a theoretical curiosity. Until models can provide stable, reproducible numerical scores, enterprises must treat automated scoring as a probabilistic signal requiring monitoring, robust engineering, and human oversight rather than a deterministic truth.