💬 NLP

Judge Circuits

This paper employs Position-aware Edge Attribution Patching to reveal that LLMs possess a shared, sparse "Latent Evaluator" sub-graph for judgments that is structurally decoupled from fragile, format-specific output branches, explaining why benchmark reliability often measures formatter geometry rather than true evaluation quality.

Original authors: Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia, Qianli Wang, Van Bach Nguyen, Aaron Louis Eidt, Christopher Ebert, Wojciech Samek, Jing Yang, Vera Schmitt, Sebastian Möller, Simon Ostermann

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia, Qianli Wang, Van Bach Nguyen, Aaron Louis Eidt, Christopher Ebert, Wojciech Samek, Jing Yang, Vera Schmitt, Sebastian Möller, Simon Ostermann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Fickle Judge"

Imagine you hire a very smart AI to act as a judge. You ask it to rate a story on a scale of 1 to 5 stars. It gives the story a 5.
Then, you ask the exact same AI the exact same question, but you change the format slightly: instead of asking for a number, you ask it to say "Yes" or "No" to whether the story is good. Suddenly, it says "No."

This is the problem the paper investigates. Large Language Models (LLMs) are supposed to be consistent judges, but they often give different answers just because the format of the question changed (e.g., numbers vs. words). This makes them unreliable for grading things automatically.

The Investigation: Looking Inside the Brain

The researchers didn't just look at the questions and answers; they looked inside the AI's "brain" (its internal computer code) to see where the confusion happens. They used a special tool called PEAP (think of it as a high-tech X-ray) to trace the path of information as the AI processes a judgment.

They discovered that the AI's brain is actually built like a two-part factory:

1. The "Core Judge" (The Latent Evaluator)

Deep inside the middle layers of the AI, there is a small, specialized team of neurons. Let's call them the Core Judges.

What they do: They read the story, think about it, and form a solid, internal opinion. They decide, "This is a good story."
Key Finding: This team is the same whether you ask for a star rating, a Yes/No answer, or a True/False verdict. They are the "truth-tellers" inside the machine. If you turn them off, the AI forgets how to judge entirely, but it still remembers facts (like who the president is).

2. The "Translators" (The Task Formatters)

Once the Core Judges decide the story is good, they pass that opinion to a different team at the very end of the factory line. Let's call them the Translators.

What they do: Their only job is to take the internal opinion ("Good story") and turn it into the specific format you asked for.
- If you asked for stars, they write "5".
- If you asked for Yes/No, they write "Yes".
The Problem: The researchers found that these Translators are fragile and inconsistent. Sometimes, the "Yes/No" Translator gets confused by the format and accidentally writes "No," even though the Core Judge said it was a good story.

The Experiment: Proving the Split

To prove this, the researchers did a clever experiment called Format Transfer Injection.

Imagine the Core Judge has finished its work and is holding a glowing "Good Story" signal. The researchers took that exact signal and forced it into the "Translators" of a different task (one that usually says "Bad Story").

Result: In most cases, the "Bad Story" Translator suddenly changed its mind and said "Good Story" (or "Yes").
Meaning: This proved that the Core Judge was doing the right work all along. The mistake wasn't in the thinking; it was in the translation at the very end. The "Translators" are the weak link that causes the AI to be inconsistent.

Why Some AI Models Are Better Than Others

The paper also found that this "two-part factory" design depends on the specific model architecture, not just how big the model is.

Modular Models (like Qwen): These models have a very clean separation. The Core Judges and Translators are in different rooms. If you break the Translators, the Core Judges still work perfectly.
Entangled Models (like Gemma-3-12B): In these models, the Core Judges and Translators are tangled together in a messy knot. If you try to fix the Translators, you accidentally break the Core Judges too. This makes them harder to fix.

The Takeaway

The paper concludes that when an AI judge gives inconsistent answers, it's usually not because the AI is bad at thinking or evaluating. It's because the output formatting (the way the answer is written down) is glitching.

The Analogy Summary:
Think of the AI as a brilliant chef (the Core Judge) who knows exactly how delicious a meal is.

If you ask the chef to write a review, they write "5 stars."
If you ask them to nod their head, they nod "Yes."
But sometimes, the waiter (the Translator) who takes the chef's order to the customer gets confused. The waiter might hear "5 stars" but accidentally tell the customer "The food is terrible" because the waiter is bad at switching between languages.

The problem isn't the chef's taste; it's the waiter's translation. To fix AI judges, we don't need to retrain the chef; we just need to fix the waiter.

Technical Summary: Judge Circuits

Problem Statement

The "LLM-as-a-Judge" (LaaJ) paradigm has become the standard for scaling model evaluation, yet it suffers from a critical reliability issue: models assign systematically different scores to the same input depending on the output format (e.g., a 1–5 rating versus a True/False classification). Prior work has documented this inconsistency but has stopped at the input-output level, failing to explain the internal computational mechanisms. The central question addressed by this paper is not whether models judge consistently, but where in the computational pathway from input to output token format-induced inconsistency originates.

Methodology

The authors employ Position-aware Edge Attribution Patching (PEAP) to causally investigate the internal mechanisms of judge circuits in Gemma-3, Qwen2.5, and Llama-3 models. Their approach involves:

Circuit Discovery: Using PEAP to trace causal edges across token positions (essential for tasks comparing separated spans like premise vs. hypothesis) from differential input tokens (clean vs. corrupted prompts) to the output.
Contrastive Decomposition: Tracing two circuits for the same data instances: one for a rating task ( $C_{rate}$ ) and one for a classification control task ( $C_{class}$ ). The intersection of these circuits ( $C_{LE} = C_{rate} \cap C_{class}$ ) defines the Latent Evaluator (LE), while the differences define Task Formatters (TF).
Causal Probes: Validating the discovered circuits through three independent methods:
- Zero-Ablation: Clamping LE components to zero to test if world knowledge is preserved while judgment collapses.
- Subspace Steering: Identifying a 1D judgment direction in the LE activation space and steering it to observe output changes.
- Format Transfer Injection (FTI): Capturing LE activations from a rating prompt and injecting them into a classification prompt's graph to see if the output token flips.

Key Contributions and Results

1. Discovery of a Shared Latent Evaluator

The study identifies a sparse, generalized Latent Evaluator (LE) sub-graph located in the mid-to-late Multi-Layer Perceptrons (MLPs) and select late attention heads.

Cross-Task Overlap: Distinct judgment tasks (structured linguistic correctness, preference, and sentiment) share a dense computational trunk. For example, on Qwen2.5-7B, Edge IoU between tasks ranges from 34.9% to 47.0%.
Faithfulness: These sparse circuits (top- $k \le 200$ edges) are causally sufficient, recovering $\ge 87\%$ of the model's evaluation behavior in 21 out of 25 model-task configurations.

2. Architecture-Dependent Modularity

The functional isolation of the LE is not a universal consequence of scale but depends on the model architecture:

Modular Models: In Qwen2.5-7B, Qwen2.5-14B, Llama-3.1-8B, and Gemma-3-27B, zero-ablating the LE collapses judgment performance while leaving world knowledge (MMLU, StrategyQA) intact.
Entangled Models: In Gemma-3-12B, the LE is tightly entangled with world-knowledge pathways; ablating it degrades both judgment and factual recall. This indicates that modularity is an architectural property, not merely a function of parameter count.

3. Mechanism of Format-Induced Inconsistency

The paper provides a mechanistic account of why format changes cause inconsistency:

The Split: Judgment is computed by a stable, continuous signal in the shared LE trunk. This signal is then mapped to specific output tokens by fragile, format-specific terminal branches (Task Formatters).
The Bottleneck: Inconsistency arises not in the evaluation itself, but in the routing. The LE computes a stable 1D judgment direction, but the Task Formatter's "attractor geometry" (how it maps this scalar to discrete tokens) varies by format and model.
- Binary/Ordinal Formats: In formats with clear attractors (e.g., binary Yes/No or 1–5 scales), the LE signal is successfully decoded.
- Multi-Attractor Formats: In tasks like MNLI (Entailment/Neutral/Contradiction), the formatter spreads probability mass across three competing tokens. Injecting a 1D judgment signal often fails to flip the output because the signal fragments across the three attractors, preventing a single label from reaching the argmax.
FTI Evidence: Blanket activation injection from a rating prompt into a classification prompt flips the output token in modular models (e.g., Qwen2.5-7B flips $\ge 99\%$ of cases) but fails in models where the formatter is geometrically insulated (e.g., Gemma-3-27B on MNLI).

4. Practical Implications for Evaluation

The authors demonstrate that the 1D causal direction of the LE can serve as a zero-shot judgment scalar.

On small- $N$ preference data (where prompted outputs are poorly calibrated and supervised probes overfit), this latent signal matches or exceeds supervised residual probes.
However, on tasks where the prompted vocabulary is scale-aligned with human labels (e.g., Yelp 1–5 stars), probability-weighted expected values remain a strong baseline that the latent direction does not surpass.

Significance and Claims

The paper reframes the debate on LaaJ reliability. It argues that behavioral inconsistency under format perturbations is the expected signature of a stable internal judgment routed through a fragile terminal mapping, rather than a failure of the underlying evaluation capability.

Diagnostic Shift: The diagnostic question shifts from "Does the model judge consistently?" to "Does the formatter for this output specification preserve the underlying judgment?"
Benchmark Interpretation: Benchmark-level reliability comparisons across formats are partially measuring formatter geometry rather than evaluation quality.
Architectural Insight: The findings challenge the assumption that clean internal abstractions emerge generically with scale; instead, architectural choices (training data, attention sparsity, normalization) play a load-bearing role in determining whether a model exhibits functional modularity.

The authors conclude that fixes for LaaJ inconsistency should target the formatter (the routing mechanism) without disturbing the model's core judgment competence, and that latent subspace extraction is a viable alternative to prompted outputs, particularly in regimes where discrete outputs are poorly calibrated.