Technical Summary: Evidence-Locked, Non-Compensatory Selection Bounds LLM-Judge Failure in Reasoning Pipelines

1. Problem Statement

The paper addresses a critical flaw in the deployment of Large Language Model (LLM) judges within reasoning pipelines. While judges are increasingly used to gate candidate answers, filter preference data, and supply rewards for reinforcement learning (RL), their output is often treated as a direct decision rather than a measurement. The authors argue that the cost of a judge's decision depends less on the judge's intrinsic accuracy and more on the decision rule governing its authority.

A scalar judge, which outputs a single "quality" score, suffers from conflation: it collapses distinct dimensions of reasoning (e.g., logical validity vs. factual grounding) into one number. This allows a fluent but factually incorrect chain of reasoning (e.g., correctly identifying a host country but incorrectly stating its capital) to receive a high score because the logical structure is sound, while the factual error remains silent. The paper posits that without constraints, a judge can unilaterally overwrite a correct consensus with an incorrect but high-scoring alternative, causing significant accuracy degradation.

2. Methodology

The study evaluates decision rules on frozen candidate pools generated by four GRPO policies. By keeping the candidates, prompts, and compute budget fixed, the authors isolate the impact of the selection rule itself.

2.1 Diagnostic Decomposition (RA-MSD)

The authors first implemented a Retrieval-Augmented Multi-Signal Decision (RA-MSD) reward model. This decomposes the judge's evaluation into seven explicit channels computed at the step level:

1. Logical Validity ($R_{logic}$): Checks inferential connection to preceding steps.
2. Factual Grounding ($R_{fact}$): Checks support against retrieved evidence.
3. Epistemic Calibration ($R_{epist}$): Measures confidence vs. evidence strength.
4. Information Gain ($R_{info}$): Penalizes verbosity/stalling.
5. Source Triangulation ($R_{triang}$): Requires multiple corroborating sources.
6. Adversarial Robustness ($R_{adv}$): Tests sensitivity to lexical vs. semantic changes.
7. Minimal Sufficiency ($R_{min}$): Penalizes padding with unverifiable claims.

Step-Level Gating (Failed Variant): Initially, these channels were combined into a weighted sum with logic and factuality acting as feasibility constraints (threshold $\tau=0.5$). This was used as a training reward for GRPO. However, this approach failed to improve held-out accuracy over plain GRPO, suggesting that summing the channels back into a scalar for the optimizer reintroduces the conflation problem.

2.2 Evidence-Locked Derive–Gate–Repair (EL-DGR)

The successful approach applies the decomposition not as a training signal, but as a post-training selection rule that subordinates the judge to evidence certificates. EL-DGR partitions candidates into feasibility strata:

1. Certificate Definition:

   • Retrieval QA: A strict extractive certificate $c(a, E)$ is set to 1 only if the answer $a$ occurs verbatim in the retrieved evidence $E$.
   • Arithmetic: A derived numeric answer must be independently checked for consistency.

2. Decision Logic (Non-Compensatory):
   The selector chooses the final answer $a_{EL-DGR}$ based on a strict hierarchy:

   • Default: The majority vote answer ($m$) is returned unless displaced.
   • Override: The judge's preferred answer ($s$) is accepted only if it is certified ($c(s, E)=1$) AND the majority is not certified.
   • Repair: A verified repair ($r$) is accepted only if neither $m$ nor $s$ is certified, but $r$ is.

Crucially, the judge's scalar preference can never promote an uncertified candidate across strata. It can only rank candidates within a stratum. This bounds the judge's "blast radius," preventing it from converting a correct consensus into an incorrect answer unless the evidence itself contains the error.

3. Key Contributions

1. Measurement of Judge Authority: The paper demonstrates that on frozen pools, an unconstrained scalar judge offers negligible improvement over majority vote (+1.0 pp on GSM8K, +0.34 EM on HotpotQA) and performs significantly worse (−10.0 pp) on a frozen-rule confirmation split compared to majority vote.
2. EL-DGR Rule: The introduction of a task-adaptive, non-compensatory decision rule that subordinates the judge to evidence certificates. This rule improves performance over all matched selectors (first candidate, majority vote, scalar judge) without generating new candidates or increasing the budget significantly.
3. Decision Audit & Negative Results:
   • Audit: On a 30-question pilot, EL-DGR overturned consensus on only 8 questions and never converted a correct consensus into an incorrect answer.
   • Negative Result: The same seven-channel decomposition used as a training reward (summed into a scalar) yielded null results, confirming that the benefit comes from the partitioning logic, not the scoring granularity itself.

4. Results

Experiments were conducted on GSM8K (500 questions) and HotpotQA (300 questions) using a frozen pool of candidates from a Qwen2.5-3B policy.

• GSM8K (Arithmetic):

  • First Candidate: 55.4%
  • Majority Vote: 55.8%
  • Scalar Judge: 56.8%
  • EL-DGR: 58.2% (+2.8 pp over first candidate, $p=0.0026$).
  • Note: The gain here is driven by the executable "derive-repair" branch, as GSM8K lacks external facts for extraction.

• HotpotQA (Retrieval):

  • First Candidate: 15.33 EM / 22.97 F1
  • Majority Vote: 15.33 EM / 23.19 F1
  • Scalar Judge: 15.67 EM / 23.49 F1
  • EL-DGR: 17.33 EM / 25.46 F1 (+2.00 EM over first candidate, $p=0.0703$; +1.67 EM over majority, $p=0.0625$).
  • Note: The improvement is driven by the evidence-locked constraint preventing the judge from overriding supported consensus.

• Frozen-Rule Confirmation (HotpotQA, n=30):

  • The unconstrained scalar judge scored 10.0 EM, which was 10 points below majority vote (20.0) and 3.3 points below the first candidate (13.3).
  • Under EL-DGR, the same judge achieved 26.7 EM, outperforming all other selectors on this split.

5. Significance and Claims

The paper's primary claim is that bound the judge's authority rather than trying to make the judge accurate.

• Admissibility over Accuracy: The study suggests that in operational loops, the decision rule (admissibility constraints) is more critical to system reliability than the judge's pairwise agreement rate. A judge with a 60% agreement rate is harmless if it can only break ties among admissible answers, but dangerous if it can unilaterally overwrite a correct consensus.
• The "Null" Decomposition: The paper explicitly states that decomposing reasoning into multiple signals does not help if those signals are recombined into a single scalar for training. The decomposition is only effective when used as a structural constraint (a partition) that prevents compensation between dimensions.
• Practitioner Guidance: The authors advise practitioners to define the "default" a judge must beat, the "certificate" required to override that default, and to measure the override rate. They argue that constraining what a judge may overrule is a cheaper and more effective safety mechanism than fine-tuning the judge itself.

Limitations: The authors acknowledge that their results are based on a single training seed, a 3B policy, and a 7B judge. Absolute scores are not comparable to state-of-the-art models due to the constrained budget and legacy artifacts in the candidate pool. The statistical power for some comparisons (e.g., HotpotQA first-candidate contrast) is borderline.