Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders

Technical Summary: Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders (RAGLens)

1. Problem Statement

Retrieval-Augmented Generation (RAG) aims to improve the factuality of Large Language Models (LLMs) by grounding responses in external evidence. However, "faithfulness failures"—where the model contradicts the retrieved context, introduces unsupported details, or extrapolates beyond the evidence—remain a critical challenge.

Existing hallucination detection methods face a trade-off:

• Specialized Detectors: Require massive amounts of high-quality annotated data for fine-tuning.
• LLM-as-a-Judge: High inference costs and potential for "plausible but unfaithful" reasoning.
• Internal Representation Probing: While promising, traditional methods (using hidden states or attention scores) struggle with the polysemanticity of neurons (where one neuron represents multiple unrelated concepts) and the opacity of high-dimensional hidden states.

2. Methodology: RAGLens

The authors propose RAGLens, a lightweight, interpretable detector that leverages Sparse Autoencoders (SAEs) to disentangle the internal activations of LLMs into semantically meaningful, "monosemantic" features.

The RAGLens Pipeline:

1. Sparse Feature Extraction: The model extracts hidden states from a specific layer of the LLM. An SAE encoder transforms these states into a sparse vector of features ($z_t$), where only a few features are active at any given time.
2. Instance-level Feature Summary: To move from token-level to instance-level detection, the method uses channel-wise max pooling. This captures the peak activation of each feature across the entire generated sequence.
3. Information-based Feature Selection: Instead of using all SAE features (which are numerous), the authors use Mutual Information (MI) to rank features based on how much information they provide about the hallucination label ($\ell$). Only the top $K'$ most informative features are selected.
4. Transparent Prediction via GAMs: The selected features are fed into a Generalized Additive Model (GAM). Unlike deep neural networks, GAMs model the output as a sum of univariate shape functions: $g(E[\ell|\tilde{z}]) = \beta_0 + \sum f_j(\tilde{z}_j)$. This structure allows the model to be both highly accurate and fully interpretable.

3. Key Contributions

• SAE-based Detection: Proves that SAE features can capture nuanced, interpretable signals specifically triggered by RAG hallucinations (e.g., unsupported numbers or dates).
• RAGLens Framework: Introduces a lightweight, high-performance pipeline for detection, explanation, and mitigation.
• Interpretability & Mitigation: Provides both local explanations (identifying specific tokens that caused the hallucination) and global explanations (understanding what concepts a feature represents). This enables "post-hoc mitigation," where the model is prompted to revise its output based on specific feedback.
• Theoretical Justification: Provides a mathematical proof (Theorem 1) justifying why max pooling is an effective strategy in the sparse-activation regime for maximizing mutual information.

4. Experimental Results

• Superior Performance: On benchmarks like RAGTruth and Dolly, RAGLens consistently outperformed existing baselines, including prompt-based methods, uncertainty-based methods (Perplexity, Entropy), and internal representation methods (ReDeEP). It achieved AUC scores $>80\%$ on several tasks.
• Cross-Model Generalization: The authors demonstrated that an SAE-based detector trained on one model can effectively detect hallucinations in outputs produced by other models, suggesting that LLMs possess latent internal knowledge of faithfulness that exceeds their explicit reasoning capabilities.
• Effective Mitigation: Using RAGLens feedback (especially token-level feedback) significantly reduced hallucination rates in Llama2-7B outputs, as verified by multiple LLM judges and human annotators.
• Design Insights:
  • Layer Selection: Performance typically peaks in the middle layers of the LLM.
  • Predictor Choice: GAMs outperformed more complex models like MLPs and XGBoost, proving that the relationship between SAE features and hallucinations is effectively captured through additive modeling.

5. Significance

This paper marks a significant step in mechanistic interpretability applied to RAG. By moving away from "black-box" hidden states toward "transparent" sparse features, RAGLens provides a blueprint for building trustworthy AI systems. It demonstrates that we can not only detect when a model is hallucinating but also why (by identifying the specific semantic concept being misapplied) and how to fix it, making it highly applicable to high-stakes domains like medicine or law.