💻 computer science

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems

The paper introduces JudgeSense, a benchmark designed to quantify the stability of LLM-as-a-judge systems by measuring how much their verdicts change when prompts are paraphrased, revealing significant inconsistencies in task performance and inherent biases across different models.

Original authors: Rohith Reddy Bellibatlu

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rohith Reddy Bellibatlu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Moody Judge" Problem: Why Your AI Evaluator Might Be Unreliable

Imagine you are entering a cooking competition. You present a delicious lasagna to a judge.

The first time you ask, "How would you rate this lasagna on a scale of 1 to 10?", the judge gives you a 9.

The next day, you ask the exact same judge, but you phrase it slightly differently: "On a scale of 1 to 10, how tasty is this lasagna?" Suddenly, the judge gives you a 6.

The lasagna hasn't changed. The judge hasn't changed. Even the scale is the same. But because you changed a few words, the verdict flipped. In the world of Artificial Intelligence, this is a massive problem, and a new research paper called JudgeSense has just put a spotlight on it.

The Core Concept: The "Judge Sensitivity Score" (JSS)

Nowadays, instead of humans grading AI models, we use "AI Judges" (like GPT-4 or Claude) to grade other AIs. We assume these AI judges are objective, like a professional referee.

However, the researchers behind JudgeSense discovered that many AI judges are actually quite "moody." They are hyper-sensitive to how a question is phrased. If you change "Is this correct?" to "Is this accurate?", the AI might change its mind.

To measure this "moodiness," they created the Judge Sensitivity Score (JSS):

A score of 1.0 is like a rock-solid judge: No matter how you ask the question, the answer is always the same.
A low score is like a judge who is easily swayed by the wind: A tiny change in your wording causes them to flip-flop on their decision.

The Four Big Discoveries

The researchers tested nine different AI models across four different "tasks" (like checking facts or judging how well a summary flows). Here is what they found:

1. The "Coherence" Rollercoaster (The Moodiest Task)

When asking AIs to judge how well a story "flows" (coherence), the results were all over the place. Some models, like Claude, were incredibly stable (almost a perfect 1.0). But others, like Gemini, were wildly inconsistent. It turns out that judging "quality" is much harder for an AI than just checking a fact, making it very easy for the wording to trip them up.

2. The "Polarity Trap" (The Confused Student)

The researchers found a weird glitch in how AIs handle facts. If you ask, "Is this true? Answer YES or NO," the AI might say YES. But if you ask, "Does this contain errors? Answer YES or NO," the AI might get confused and say YES again—even though "YES" now means the opposite! This isn't because the AI is "dumb," but because the way the question is flipped (the "polarity") confuses its logic.

3. Size Doesn't Equal Stability (The "Big is Not Always Better" Rule)

In the AI world, bigger models (with more "brain cells" or parameters) are usually assumed to be smarter. But JudgeSense proved that bigger isn't always more consistent. A smaller, more focused model can sometimes be a much more reliable judge than a massive, expensive "frontier" model that gets distracted by fancy wording.

4. The "Always-A" Glitch (The Lazy Referee)

In some tests, the AI judges became "lazy." When asked to choose between two options (Option A or Option B), most of them simply picked Option A every single time, regardless of what the content actually said. It’s like a referee who always awards the penalty to the home team just because they are standing on the left side of the field. This is called "position bias," and it makes the judge useless for making real comparisons.

The Takeaway: How to Use AI Judges Safely

If you are a developer using AI to grade your work, the paper offers three pieces of "survival advice":

Don't trust a single question: Because AIs are sensitive to wording, don't rely on one prompt. If the answer changes when you rephrase the question, your judge is unreliable.
Watch your "Yes/No" logic: Be very careful when asking questions that flip the meaning (like "Is this wrong?" vs "Is this right?").
Pick for stability, not just "smartness": When choosing an AI judge, don't just pick the most famous or biggest one. Pick the one that proves it can give the same answer twice.

In short: JudgeSense reminds us that even in the age of super-intelligent AI, the way you ask a question is just as important as the answer you get.

Technical Summary: JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems

1. Problem Statement

As Large Language Models (LLMs) are increasingly used as automated evaluators ("LLM-as-a-Judge"), a critical methodological vulnerability has emerged: prompt sensitivity. While researchers often treat prompt templates as mere implementation details, the paper argues that slight semantic paraphrases can cause a judge to flip its verdict. This instability transforms the judge from a reliable measurement tool into a source of noise, potentially leading different researchers to draw conflicting conclusions from the same underlying data. Existing literature focuses on prompt sensitivity in task-solvers (how a model answers a question), but there is a lack of standardized metrics for judges (how a model assigns a label to someone else's answer).

2. Methodology

The authors introduce JudgeSense, a framework designed to quantify decision stability across semantically equivalent prompts.

The Metric: Judge Sensitivity Score (JSS): The core contribution is the JSS, defined as the fraction of paraphrase pairs on which a judge returns an identical decision.
$JSS(j, t) = \frac{1}{|P|} \sum_{i=1}^{|P|} \delta(j(p_i), j(p'_i))$
where $j$ is the judge, $t$ is the task, and $P$ is a set of paraphrase pairs. A JSS of 1.0 indicates perfect consistency, while values below 0.8 indicate practical unreliability.
Benchmark Dataset: The authors constructed a dataset of 494 validated paraphrase pairs across four evaluation dimensions:
1. Factuality (from TruthfulQA)
2. Coherence (from SummEval)
3. Relevance (from BEIR)
4. Preference (from MT-Bench)
  Paraphrase equivalence was validated using GPT-4o-mini as an independent classifier.
Experimental Setup: Nine models (5 commercial, 4 open-source) were evaluated using greedy decoding (temperature $\approx$ 0). The study used minimalist instruction templates to isolate the effect of natural language rephrasing from other factors like Chain-of-Thought or role-priming.

3. Key Contributions

A Formal Metric (JSS): A simple, reproducible, and reportable single-number metric for judge stability.
A Standardized Benchmark: A public dataset of validated paraphrase pairs spanning multiple critical evaluation dimensions.
Empirical Analysis: A large-scale study of nine major LLM families, revealing how different tasks and model architectures respond to prompt variation.
Practitioner Guidance: Concrete recommendations on which models to use for specific tasks and how to design prompts to avoid "flip" artifacts.

4. Key Results

The findings reveal that prompt sensitivity is highly task-dependent:

Coherence (The Discriminator): This was the only task where judges showed significant variance. JSS ranged from 0.389 (Gemini-2.5-flash) to 0.992 (Claude-Sonnet-4.5). Notably, model scale did not predict consistency; a frontier model (Gemini) performed significantly worse than mid-sized or specialized models.
Factuality (The Prompt Artifact): All nine judges exhibited a uniform $\sim$ 37% flip rate. The authors discovered this was not a model defect but a prompt design artifact caused by "polarity inversion" (e.g., one prompt asking "Is this correct?" while another asks "Does this contain errors?"). Once corrected for polarity, JSS rose to $\approx$ 0.9+.
Preference & Relevance (The Degeneracy): Eight of nine judges exhibited "always-A" behavior (position bias), returning the same label regardless of the prompt. This resulted in a JSS of 1.0, but it was uninformative because the models were not actually "judging" but rather anchoring to a position.
Negative Kappa ( $\kappa$ ): Two models (Gemini-2.5-flash and Mistral-7b) produced negative Cohen’s $\kappa$ on coherence, meaning they were systematically anti-correlated with themselves—different phrasings pushed them in opposite directions on the Likert scale.

5. Significance and Implications

The paper has profound implications for the reliability of LLM-based benchmarking:

Reliability vs. Accuracy: The authors distinguish between a judge being correct (high human agreement) and being stable (high JSS). A judge can be consistently wrong, or accurately right but highly sensitive to wording. Both must be reported.
Model Selection: Practitioners should not default to the largest/newest model for evaluation. For coherence-based tasks, models like Claude-Sonnet-4.5 and Qwen-2.5-72b are superior due to their high instruction-following discipline.
Prompt Engineering for Judges: To minimize noise, researchers must avoid polarity-inverting templates and implement option-order randomization (A/B testing) to mitigate the position bias observed in pairwise tasks.
Standardization: The paper calls for the community to adopt JSS as a standard reporting metric to ensure that LLM-as-a-judge evaluations are reproducible and scientifically sound.