Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping

Technical Summary: Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping

Problem Statement
Large Reasoning Models (LRMs) generate long, coherent reasoning traces before producing a final answer. While these traces offer rich signals, they often obscure answer-level hallucination detection. Directly probing hidden states along the full trace or using the trace text for detection is brittle; detectors tend to overfit to superficial stylistic patterns in the reasoning process rather than capturing the validity of the final answer. Furthermore, existing methods often rely on expensive test-time sampling (e.g., generating multiple responses to measure consistency) or require human annotations for training classifiers. The core challenge is to distill a stable, answer-centric signal from the reasoning trajectory that is robust to surface-form variations and indicative of hallucination risk without incurring high inference costs.

Methodology: Answer-agreement Representation Shaping (ARS)
The authors propose ARS, a framework that learns detection-friendly, trace-conditioned answer representations by explicitly encoding answer stability through latent interventions. The method operates in two main stages:

1. Latent Intervention for Counterfactual Answers:
   Instead of perturbing the text of the reasoning trace (which can disrupt coherence or introduce spurious cues), ARS intervenes directly in the model's latent state. Specifically, it targets the trace-boundary embedding ($h$), defined as the last-token hidden state of the reasoning trajectory at the penultimate layer of the LRM.

   • Small perturbation vectors ($\delta$) are sampled from a distribution (e.g., isotropic Gaussian) and added to $h$ to create a modified state $\tilde{h} = h + \delta$.
   • The model continues decoding from $\tilde{h}$ to generate a counterfactual answer ($\tilde{a}$).
   • This process is repeated $M$ times to generate a set of counterfactual answers.

2. Answer-Agreement Representation Shaping:
   The framework constructs training pairs based on the agreement between the original answer ($a$) and the counterfactual answers ($\tilde{a}$).

   • Positive Pairs: Counterfactual answers that agree with the original answer ($Agr(a, \tilde{a}) = 1$).
   • Negative Pairs: Counterfactual answers that disagree with the original answer ($Agr(a, \tilde{a}) = 0$).
   • A lightweight mapping function $g_\phi$ is trained to transform the vanilla answer embeddings ($u$) into shaped embeddings ($z$). The objective is a contrastive loss that pulls agreeing embeddings closer to the anchor and pushes disagreeing embeddings apart.
   • Crucially, the base LRM parameters ($\theta$) remain frozen; only the lightweight head $g_\phi$ is trained. This process requires no ground-truth hallucination labels, relying solely on the internal consistency of the model's own generations.

Key Contributions

• First Framework for Trace-Shaping: ARS is presented as the first framework to principledly use reasoning trajectories to shape answer representations for hallucination detection in LRMs, utilizing latent perturbations at the trace boundary to induce counterfactual answers.
• Agreement-Driven Objective: The paper introduces an objective that organizes answer embeddings based on whether small internal changes preserve the answer, avoiding reliance on brittle text perturbations or human annotations.
• Theoretical and Empirical Analysis: The authors provide a theoretical proposition (Proposition 4.2) linking the agreement-separation probability of the shaped embeddings to the upper bound of hallucination detection error. Empirically, they demonstrate that answer stability (measured by counterfactual consistency) is predictive of truthfulness.

Results
Experiments were conducted on four datasets (TruthfulQA, TriviaQA, GSM8K, MATH-500) using multiple LRM families (Qwen3-8B/14B, DeepSeek-R1-Distill variants).

• Performance Gains: ARS consistently outperforms strong baselines, including uncertainty-based methods (Perplexity, Semantic Entropy), consistency-based methods (Lexical Similarity, SelfCKGPT), and LRM-specific detectors (RHD, RACE, G-Detector).
  • On TruthfulQA with Qwen3-8B, ARS achieved an AUROC of 86.64% (using CCS as a downstream detector), a substantial improvement over the vanilla LRM embedding baseline (66.85%) and other state-of-the-art methods (e.g., TSV at 77.08%).
  • Similar gains were observed across all datasets and model sizes, with improvements ranging from 10% to 20% over vanilla embeddings.
• Generalizability: The shaped representations transfer effectively across datasets (e.g., training on GSM8K and testing on TriviaQA) and scale to larger models (14B parameters).
• Efficiency: Unlike sampling-based baselines that require multiple forward passes at inference, ARS requires only a single generation at test time. The training overhead is minimal (e.g., ~649 seconds for TruthfulQA training set).

Significance and Claims
The paper claims that ARS addresses a critical reliability failure in LRMs where fluent reasoning traces can mask factual errors. By shaping representations to reflect answer stability under latent perturbations, ARS exposes the "fragile" internal states associated with hallucinations. The authors emphasize that ARS is a detection component rather than a guarantee of truthfulness, intended to be paired with other safeguards. The work provides insights into the geometry of hallucination in reasoning models, suggesting that hallucinated answers often reside in regions of the latent space with low stability margins, which can be effectively exposed through answer-agreement shaping without human supervision.