Technical Summary: Reasoning Denoiser (REDE)

Problem Statement

Large Reasoning Models (LRMs), such as OpenAI's o-series and DeepSeek-R1, generate extensive reasoning traces before producing final answers. While these traces are intended to enhance reasoning capabilities, they often introduce a critical reliability issue: hallucination. Existing studies have attempted to leverage these traces for hallucination detection, but this approach is hindered by the presence of noisy reasoning steps.

The paper identifies two prevalent forms of noise within LRM trajectories:

1. Irrelevant steps: Steps that do not contribute meaningful, problem-specific information.
2. Repetitive steps: Steps whose information is semantically redundant because it is already covered by later steps in the trajectory.

The authors demonstrate that naively incorporating the full reasoning trace into hallucination detectors degrades performance. Standard filtering methods, such as confidence-based scoring or naive embedding-based filtering (e.g., $k$-NN on raw embeddings), fail to reliably distinguish between informative and noisy steps. Confidence scores overlap heavily between step types, and repetitive steps remain semantically close to informative ones in raw representation spaces.

Methodology: REDE Framework

The authors propose REDE (Reasoning Denoiser), a learning framework designed to denoise reasoning traces by shaping the step-level representation space. The framework operates in three stages:

1. Supervision Signal: Final-Answer Attention

REDE utilizes the LRM's internal attention mechanism as an automatic, annotation-free supervision signal. The core observation is that the attention score from the final answer token to each reasoning step correlates with the step's utility:

• Informative steps receive high attention as they directly support the final answer.
• Irrelevant steps receive low attention due to weak connection to the answer.
• Repetitive steps receive low attention because their content is subsumed by later, more decisive steps.

The step score $s_i$ is calculated as the total attention mass assigned by the final answer token to the tokens within step $c_i$.

2. Learning a Denoised Representation Space

Instead of using attention scores directly as a filter (which incurs inference overhead and lacks robustness), REDE learns a lightweight projection network $f_\phi$ to reshape the step embeddings.

• Embedding Extraction: Step embeddings are constructed using perplexity-weighted aggregation of token hidden states to emphasize content-bearing tokens.
• Proxy Selection: During training, steps are ranked by attention scores. The top-$\rho\%$ form a proxy set of informative steps ($\mathcal{T}$), and the bottom-$\rho\%$ form a proxy set of noisy steps ($\mathcal{B}$).
• Objective Function: The projection network is trained to induce a specific geometric structure in the embedding space using a composite loss function $\mathcal{L}$:
  • $\mathcal{L}_{compact}$: Pulls informative steps together to form a compact cluster.
  • $\mathcal{L}_{disperse}$: Prevents noisy steps from clustering together, keeping them as identifiable outliers.
  • $\mathcal{L}_{separate}$: Pushes the informative and noisy clusters apart to maximize global separability.

3. Inference-Time Filtering

During inference, the projection network maps step embeddings into the learned space. A distance-based filter (using $k$-NN cosine distance) identifies noisy steps as those with large distances from the dense informative cluster. The top-$\zeta\%$ of steps with the highest distance scores are removed, and the filtered trace is passed to a downstream hallucination detector.

Key Contributions

1. Identification of Noise: The paper establishes that irrelevant and repetitive steps are a significant, overlooked obstacle to hallucination detection in LRMs, showing that removing them yields substantial performance gains.
2. REDE Framework: A novel, lightweight learning framework that leverages final-answer attention to shape the representation space, enabling reliable filtering of noisy steps without human annotation.
3. Generalizability and Performance: Extensive experiments demonstrate that REDE is detector-agnostic (compatible with probing, uncertainty-based, and verbalized methods) and consistently improves detection performance across diverse datasets and model families.

Experimental Results

The authors evaluated REDE on four reasoning benchmarks: TruthfulQA, MATH (combining MATH500, AIME 2024/2025), CodeElo, and MULTIHOPQA, using models like Qwen3-8B/32B and DeepSeek-R1-Distill-Llama-8B.

• Performance Gains: REDE consistently outperforms competitive baselines. On the TruthfulQA benchmark with Qwen3-8B, REDE (using CCS as a detector) achieved an AUROC of 87.32%, a significant improvement over the unfiltered trace (68.63%) and state-of-the-art baselines like ARS (81.92%).
• Robustness: The method generalizes well across different LRM families and datasets. Cross-dataset transferability tests showed that a projection learned on one dataset (e.g., MULTIHOPQA) could effectively denoise traces from another (e.g., CodeElo).
• Scalability: REDE remains effective on larger models (Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B), outperforming baselines by margins of up to 13.11% on TruthfulQA.
• Ablation Studies:
  • Direct attention-based filtering underperforms REDE, confirming the necessity of learning a shaped representation space.
  • Removing any component of the loss function ($\mathcal{L}_{compact}$, $\mathcal{L}_{disperse}$, or $\mathcal{L}_{separate}$) degrades performance, validating the geometric shaping strategy.
  • The method is robust to the choice of distance metrics and embedding extraction layers.

Significance and Claims

The paper claims that reasoning-step denoising is an effective and general strategy for improving hallucination detection in LRMs. By filtering out steps that obscure truthfulness cues, REDE allows downstream detectors to operate on a cleaner, more informative signal.

The authors position REDE as a lightweight, plug-and-play module that requires no human annotation and can be integrated into diverse detection pipelines. They emphasize that while REDE enhances reliability, it is a detection aid rather than a guarantee of truthfulness, and it does not intervene in the generation process itself. The work suggests that future research should explore integrating such denoising signals directly into model training to suppress noise at the source.