Probabilistic distances-based hallucination detection in LLMs with RAG

1. Problem Statement

Large Language Models (LLMs) frequently suffer from hallucinations, generating content that is factually incorrect or inconsistent with the provided context. This issue is particularly critical in Retrieval-Augmented Generation (RAG) systems, where models are expected to ground their answers in retrieved documents.

• Limitations of Current Methods: Existing detection approaches often fail in RAG settings because they:
  • Analyze generations in isolation without considering the relationship to the context.
  • Rely on consistency (requiring multiple generations, which is computationally expensive) or token probabilities (which struggle with complex dependencies).
  • Overlook the rich semantic information encoded in transformer hidden states, focusing instead on attention maps or surface-level features.
  • Often require labeled hallucination data, limiting their applicability in new domains.

2. Methodology

The authors propose a model-intrinsic, unsupervised approach that detects hallucinations by measuring the statistical divergence between the hidden state distributions of the prompt (context) and the response.

Core Hypothesis

Hallucinated responses exhibit a systematic deviation in the model's latent space compared to grounded responses. Specifically, the hidden state embeddings of a hallucinated response will be statistically further from the prompt's embeddings than those of a factually grounded response.

Technical Framework

1. Hidden State Extraction:

   • Instead of using standard layer-wise embeddings, the method extracts hidden states from specific attention heads.
   • The authors hypothesize that "faithfulness signals" are more distinct at the head level and become "blurred" when aggregated at the layer level.

2. Probabilistic Distance (MMD):

   • The core metric is the Maximum Mean Discrepancy (MMD), a non-parametric statistic used to measure the distance between two probability distributions in a Reproducing Kernel Hilbert Space (RKHS).
   • The squared MMD is calculated between the set of prompt token embeddings ($P$) and response token embeddings ($R$).
   • A Laplacian kernel with an $L_1$-norm is used.

3. Wild Bootstrap for Significance Testing:

   • To ensure the score is comparable across sequences of different lengths and valid in non-asymptotic settings, the raw MMD value is converted into a $p$-value.
   • The authors employ Wild Bootstrap (specifically with first-order autoregressive weights) to approximate the null distribution of the MMD statistic.
   • Why Wild Bootstrap? Unlike label shuffling, wild bootstrap preserves the sequential dependence and local correlations inherent in text token embeddings (syntax and semantics).
   • The final hallucination score is defined as $1 - p\text{-value}$. A higher score indicates a higher probability of hallucination.

4. Head Selection:

   • An algorithm selects the optimal subset of attention heads ($N_{opt} \le 6$) based on a probe dataset. Heads are ranked by their "separation capability" (the difference in scores between hallucinated and grounded samples).

5. Unsupervised Transfer (NLI Connection):

   • The method leverages the similarity between Hallucination Detection and Natural Language Inference (NLI). In NLI, a contradiction exists if the hypothesis contains information absent from or opposite to the premise.
   • The authors demonstrate that heads selected on NLI datasets (e.g., SNLI) can be transferred to hallucination detection tasks without labeled hallucination data, enabling a fully unsupervised detector.

3. Key Contributions

1. Novel Metric: Introduction of probabilistic distances (MMD) between prompt and response hidden state distributions as a hallucination indicator.
2. Model-Intrinsic Approach: A method that requires no external knowledge bases, auxiliary models, or multiple generations.
3. Head-Level Granularity: Demonstration that selecting specific attention heads yields superior performance compared to layer-level aggregation.
4. Unsupervised Transferability: Successful transfer of the detection mechanism from NLI tasks to RAG hallucination detection, allowing operation without labeled hallucination examples.
5. Robust Statistical Framework: Utilization of Wild Bootstrap to generate normalized, length-invariant scores ($p$-values) rather than relying on raw statistics.

4. Experimental Results

The method was evaluated on four datasets (RAGTruth, CoQA, SQuAD, XSum) using three LLMs (Mistral-7B, LLaMA-2-7B, LLaMA-2-13B).

• Performance: The proposed MMD method achieved State-of-the-Art (SOTA) or competitive results across most benchmarks.
  • It excelled particularly on LLaMA-2-13B, achieving ROC-AUC scores of 0.94 on SQuAD and 0.93 on XSum.
  • It outperformed consistency-based methods (like EigenScore, Semantic Entropy) and uncertainty-based methods (Perplexity, Max Entropy).
• Ablation Studies:
  • Bootstrap Necessity: The bootstrapped version consistently outperformed the "raw" MMD statistic, providing more stable and normalized scores.
  • Head vs. Layer: Head-level embeddings significantly outperformed layer-level embeddings (e.g., +0.2 ROC-AUC on SQuAD for LLaMA-2-7B), confirming that layer aggregation dilutes factuality signals.
  • Bootstrap Type: Wild bootstrap with AR(1) weights (accounting for autocorrelation) performed better than label shuffling or i.i.d. weights.
  • Transferability: The method maintained high performance when heads were selected on the SNLI dataset and applied to hallucination detection, proving its utility in low-resource/unsupervised settings.

5. Significance and Conclusion

This paper presents a simple yet powerful geometric approach to hallucination detection. By treating hallucination as a statistical divergence in the latent space, the authors bypass the need for complex external verification or expensive multi-generation sampling.

Key Implications:

• Efficiency: The method is computationally efficient as it requires only a single forward pass and does not need retraining or external models.
• Scalability: It performs exceptionally well on larger models (13B+), suggesting that as models grow, their latent representations of "factuality" become more distinct and detectable via MMD.
• Generalizability: The strong correlation with NLI tasks suggests a fundamental link between logical contradiction and hallucination, allowing the method to be deployed in domains where labeled hallucination data is scarce.

The work establishes that analyzing the statistical distance between prompt and response distributions is a highly effective, model-intrinsic signal for detecting hallucinations in RAG systems.