OSCAR: Orchestrated Self-verification and Cross-path Refinement

1. Problem Statement

Hallucinations in Diffusion Language Models (DLMs):
While autoregressive (AR) models generate text sequentially, Diffusion Language Models (DLMs) generate text through an iterative denoising process, starting from a fully masked sequence and progressively revealing tokens.

• The Core Issue: In DLMs, an early incorrect token commitment acts as a "contextual attractor." Due to bidirectional attention, this error biases subsequent denoising steps, causing the model to converge on a self-consistent but factually incorrect output (a phenomenon termed "hallucination crystallization").
• Limitations of Current Solutions: Existing hallucination detection methods for DLMs (e.g., TraceDet, DynHD) rely on trained classifiers to analyze denoising trajectories after generation or during the process. These methods are passive; they detect errors but cannot correct them. Furthermore, they require labeled data and may not generalize well across domains.
• The Gap: There is a lack of training-free, inference-time frameworks that can actively intervene in the DLM generation process to correct hallucinations using the model's own internal signals.

2. Methodology: OSCAR Framework

The authors propose OSCAR (Orchestrated Selective Correction via Adaptive Remasking), a training-free, zero-shot framework that leverages the unique properties of DLMs to detect and correct hallucinations.

A. Core Insight: Commitment Uncertainty

The fundamental observation is that DLM hallucinations are order-dependent. If a token is factually uncertain, the final output of that token will vary depending on the order in which tokens are revealed (the "reveal order").

• Hypothesis: If multiple denoising chains, each following a different randomized reveal order, disagree on a specific token position, that position is factually uncertain.

B. The Three-Stage Pipeline

1. Diversified Parallel Decoding:

   • OSCAR runs $N$ parallel denoising chains (default $N=8$) simultaneously.
   • Each chain uses a randomized reveal order ($\pi$). Standard inference uses a confidence-based order, which suppresses observable uncertainty; randomization breaks this coupling to expose latent uncertainty.
   • All chains share model weights and are batched for efficiency.

2. Commitment Uncertainty Localization:

   • Metric: The system computes Cross-Chain Shannon Entropy ($H_{\times, i}$) for each token position $i$ across the $N$ chains.
     $$H_{\times, i} = -\sum_{v \in V} \hat{p}_i(v) \log \hat{p}_i(v)$$
     Where $\hat{p}_i(v)$ is the empirical distribution of tokens at position $i$ across the $N$ chains.
   • Localization: Positions with high entropy (where chains disagree) are flagged as "uncertain." A threshold (e.g., top 20% of entropy values) is applied to identify specific spans requiring correction.
   • Metric for Evaluation: The paper introduces CDH (Cross-chain Divergence-at-Hallucination), which measures the fraction of truly hallucinated positions captured by the top-$k\%$ most uncertain positions.

3. Targeted Remasking and Correction:

   • Once uncertain spans are identified, OSCAR remasks these specific tokens in the base response.
   • It performs targeted re-denoising on these spans only, conditioned on:
     • The surrounding context (frozen tokens).
     • Retrieved evidence (optional, using a retriever like Contriever).
   • Re-denoising Schedule: The first step of re-denoising uses a learned (confidence-based) order to anchor the correction, followed by random orders to allow the span to settle without reproducing the original error.

3. Key Contributions

1. Native Uncertainty Signal: The paper identifies and demonstrates that cross-chain entropy is a model-native signal inherent to DLMs. Unlike AR models where uncertainty is hidden after token commitment, DLMs maintain full probability distributions, allowing uncertainty to be measured without external classifiers or labeled data.
2. Commitment Uncertainty Localization: The authors formalize the task of identifying unreliable token positions and introduce the CDH metric to evaluate localization quality. They show that unsupervised localization via cross-chain entropy outperforms trained trajectory classifiers.
3. OSCAR Framework: The first training-free, inference-time hallucination mitigation framework for DLMs. It transforms the denoising trajectory from a passive diagnostic signal into an active control mechanism for targeted correction.
4. Crystallization Analysis: The paper provides a mechanistic analysis of when hallucinations form during denoising. It finds that factual uncertainty crystallizes early in QA tasks but later in summarization, suggesting adaptive early-exit strategies.

4. Experimental Results

The framework was evaluated on LLaDA-8B and Dream-7B across benchmarks including TriviaQA, HotpotQA, CommonsenseQA, and RAGTruth.

• Detection Performance (AUROC):
  • OSCAR achieved 86.5% AUROC (LLaDA-8B) and 85.7% (Dream-7B) under LLM-as-Judge evaluation.
  • This surpasses the strongest trained detectors (e.g., DynHD) by +2.3 and +1.4 points respectively, despite OSCAR being training-free.
• Generation Quality (F1 Score):
  • OSCAR improved the macro-average F1 score by +6.1 points on LLaDA-8B and +8.3 points across both models.
  • On TriviaQA, the improvement was +10.7 points.
  • In CommonsenseQA (where chains naturally agree on single-token answers), OSCAR correctly identified no uncertainty and made no changes, demonstrating high selectivity.
• Hallucination Reduction:
  • On RAGTruth, OSCAR reduced hallucinated spans by 38–44% across subsets.
  • Factual precision (FS) improved significantly (e.g., +0.092 on Data2Text).
• Efficiency:
  • The total computational overhead is approximately 1.3× wall-clock time (due to batching $N=8$ chains and a lightweight correction phase).
  • This establishes a new Pareto-optimal point, balancing high detection/correction performance with manageable latency.

5. Significance and Implications

• Paradigm Shift: OSCAR challenges the reliance on external, trained classifiers for hallucination detection. It proves that DLMs possess an inherent structural capacity to identify factual uncertainty through their denoising dynamics, a feature absent in autoregressive models.
• Active Correction: Unlike previous methods that only flag errors, OSCAR actively corrects them by leveraging the bidirectional nature of DLMs to re-mask and re-denoise specific spans.
• Generalizability: The approach is model-agnostic within the DLM family and requires no fine-tuning, making it immediately applicable to existing and future diffusion language models.
• Future Directions: The work suggests that early-exit detection (stopping the process once uncertainty is resolved) could further reduce overhead, and that integrating this localization with retrieval planning could enhance general inference-time control.

In summary, OSCAR represents a significant advancement in making Diffusion Language Models more reliable by turning their unique generative mechanics into a self-correcting mechanism for factual accuracy.