Rethinking Atomic Decomposition for LLM Judges: A Prompt-Controlled Study of Reference-Grounded QA Evaluation

1. Problem Statement

The paper addresses a critical ambiguity in the design of Large Language Model (LLM) judges for reference-grounded evaluation.

• The Context: LLMs are increasingly used as "judges" to evaluate model outputs against a reference (e.g., determining if an answer is supported, partially supported, or unsupported).
• The Conflict: The dominant design pattern is Atomic Decomposition, where the judge breaks a candidate answer into individual factual claims and verifies each against the reference. The alternative is a Holistic Rubric, where the judge evaluates the answer as a whole based on a detailed scoring guide.
• The Conundrum: Atomic prompts are typically longer, richer, and encourage more token generation. It is unclear whether the perceived superiority of atomic judges stems from the decomposition logic itself or simply from prompt richness and increased computational budget.
• The Specific Task: The study focuses on benchmark-style completeness-sensitive reference-support classification. The goal is to classify a candidate answer as fully supported, partially supported (incomplete but factually correct), or unsupported relative to a provided reference. Crucially, "partially supported" penalizes omissions of reference information, a stricter standard than simple factual correctness.

2. Methodology

The authors conducted a controlled empirical comparison to isolate the effect of the design pattern from confounding variables.

• Judge Designs:
  1. Self-Decomposing Atomic Judge: A single-prompt design instructing the model to decompose the answer into claims, verify them, and return a structured verdict.
  2. Prompt-Controlled Holistic Judge: A single-prompt design using the same inputs but asking for a holistic score based on a similarly detailed rubric (covering correctness, completeness, unsupported details, etc.) without explicit decomposition.
  3. Schema-Matched Ablation: A variant that uses atomic-style JSON output fields but removes the decomposition instruction to test if formatting alone drives performance.
• Experimental Controls:
  • Prompt Variants: Three independently authored, pre-frozen prompt variants per design family (6 total) to rule out prompt-wording artifacts.
  • Token Accounting: Realized token usage was measured directly. The study acknowledges that if an atomic judge wins only by spending significantly more tokens, it is a less efficient design.
  • Datasets: Three QA-style benchmarks with supplied references:
    • TruthfulQA (Factuality stress test).
    • ASQA (Ambiguous QA, requiring multi-fact completeness).
    • QAMPARI (Many-answer QA).
  • Models: Four model families (Opus-4.6, GPT-4.1, Gemini-Flash, and DeepSeek-V3.2 open-weight).
  • Sample Size: 200 source examples per dataset (400 rows total per dataset due to paired testing).
• Statistical Analysis:
  • Source-level exact paired sign tests.
  • Cluster bootstrap confidence intervals (10,000 iterations) to ensure stability.
  • Per-class accuracy breakdown (Supported vs. Partially Supported vs. Unsupported).

3. Key Contributions

1. Counterexample to the "Decomposition Heuristic": The paper provides a controlled empirical counterexample to the widely held belief that atomic decomposition is automatically superior for reference-grounded evaluation.
2. Matched Prompting Protocol: Unlike prior work that often compares disparate prompts, this study rigorously controls for input fields, output structure, and prompt richness, isolating the decomposition variable.
3. Identification of the "Incompleteness Detection" Bottleneck: The study identifies that the primary advantage of holistic judges lies specifically in detecting partial support (omissions), rather than in general factual accuracy.
4. Diagnostic Ablations: The authors systematically rule out alternative explanations, such as output format constraints, token budget truncation, and prompt wording artifacts.

4. Key Results

• Overall Performance:
  • ASQA & QAMPARI: The Holistic Judge significantly outperformed the Atomic Judge across all four model families. The gains were statistically reliable (p < 0.05) in three of four families.
  • TruthfulQA: The Atomic Judge held a slight edge, though the gap was small and not always statistically significant.
• Token Efficiency:
  • On ASQA and QAMPARI, Atomic judges produced 1.4x to 2.3x more output tokens than Holistic judges due to intermediate claim enumeration.
  • The Holistic judge achieved higher accuracy and lower cost on completeness-heavy tasks.
• Per-Class Analysis (The Core Finding):
  • The performance gap is concentrated entirely in the partially_supported class.
  • Both judges achieved near-perfect accuracy on fully supported and unsupported cases.
  • Holistic judges were significantly better at detecting incompleteness (omissions of reference information). For example, on ASQA with GPT-4.1, the Holistic judge improved partially_supported detection by +30.5 percentage points over the Atomic judge.
• Robustness & Sensitivity:
  • Reference Quality: Both judge families suffered the largest accuracy drops when the reference was degraded or removed. Neither design is inherently robust to incorrect grounding.
  • Prompt Stability: The results held across all 9 cross-variant pairings (3 atomic prompts × 3 holistic prompts), confirming the result is a property of the design family, not a specific prompt.
  • Label Validity: A human annotation study confirmed that while ASQA's "partial" label is stricter than human intuition (penalizing omissions), the Holistic judge outperformed the Atomic judge even when re-evaluated under a standard "factual correctness" criterion.

5. Significance and Implications

• Practical Guidance: For benchmark-style reference-support classification (specifically where completeness is key), practitioners should consider matched holistic rubrics as a strong, and often superior, starting point compared to self-decomposing single-prompt atomic patterns. Holistic approaches offer better accuracy on incompleteness detection with lower token costs.
• Theoretical Insight: The study suggests that forcing a model to decompose an answer into claims before judging completeness may fragment the reasoning process, making it harder to detect global omissions. A holistic view may better preserve the "big picture" of what is missing.
• Scope Limitations: The authors explicitly state these findings apply to self-decomposing single-prompt patterns. They do not claim that multi-stage atomic pipelines (with externally supplied decompositions) are inferior, nor do they claim atomic evaluation is inherently bad for all tasks.
• Future Directions: The paper highlights the need to test multi-stage extract-then-verify architectures and non-QA tasks, as the current results are specific to the single-prompt, reference-grounded QA domain.

In summary, the paper challenges the default assumption that "decomposition equals better judgment," demonstrating that for tasks requiring the detection of missing information, a well-structured holistic approach is often more effective and efficient than a self-decomposing atomic one.