Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction

Technical Summary: Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction

Problem Statement
Retrieval-Augmented Generation (RAG) has become a standard for equipping Large Language Models (LLMs) with domain-specific knowledge. However, a critical vulnerability remains: there is no inherent guarantee that the context retrieved by similarity search actually supports the generated facts, nor that the LLM does not hallucinate information despite the presence of relevant context. Current safety assurance methods often rely on heuristic improvements or specific component modifications, lacking a general, architecture-agnostic protocol to provide certifiable guarantees for the RAG pipeline. The core challenge is to associate meaningful confidence measures with both the retrieval process (ensuring source fidelity) and the generation process (ensuring factuality relative to the retrieved context).

Methodology
The authors propose a two-staged, architecture-agnostic certification protocol that operates without modifying the underlying retriever or LLM components.

1. Retrieval Confidence Estimation (Block 1):

   • Approach: The authors apply Conformal Prediction (CP) to the retrieval process to select only those document chunks with a statistically guaranteed chance of originating from the correct source.
   • Mechanism: Using a calibration set of question-reference-answer triplets, the system calculates a non-conformity threshold based on retrieval scores and a similarity metric (RougeL). A threshold $s_{thres}$ is established to distinguish correct chunks from incorrect ones.
   • Guarantee: Under the assumption of sample exchangeability, this method provides a statistical guarantee (controlled by error rate $\alpha$) that the set of "trusted" chunks includes the true source chunk.
   • Diagnostics: The authors introduce two metrics, $m_1$ (binary existence of a trusted chunk) and $m_2$ (proportion of trusted chunks), to diagnose whether a specific retriever-dataset combination satisfies the exchangeability assumption required for CP to hold.

2. Response Confidence Estimation (Block 2):

   • Approach: To detect hallucinations in the final answer, the authors utilize an attention-based factuality classifier.
   • Mechanism: The system extracts intermediate attention values from the LLM during generation. Specifically, it calculates chunk-wise lookback ratios (CW LR), which measure the average attention the model pays to specific retrieved context chunks relative to the total attention paid to the context and generated output.
   • Classifier: A simple logistic regression unit is trained on these lookback ratios to predict a factuality score. Crucially, the authors exclude attention from system prompts and the user question from the feature vector to prevent the classifier from learning domain-specific biases (e.g., "finance questions are harder") rather than factual grounding.

Key Contributions

• Certifiable RAG Pipeline: A novel workflow that integrates conformal prediction for retrieval and attention-based detection for generation, providing separate confidence metrics for each stage.
• Diagnostic Metrics: The introduction of $m_1$ and $m_2$ metrics to assess the suitability of a RAG setup. The paper demonstrates that if these metrics indicate poor retriever performance (violating exchangeability), the statistical guarantees of CP fail, and chunk filtering may degrade answer quality.
• Refined Hallucination Detection: A demonstration that while full-context attention models (including prompts) may achieve higher AUROC in cross-validation, chunk-wise lookback ratios provide more reliable factual grounding by excluding non-retrieval attention sources that could mislead the classifier.

Results
The methodology was evaluated on the Natural Questions (NQ) dataset and the Ragbench (RB) collection (12 diverse datasets) using Llama-3-8B-Instruct and BM25 retrieval.

• Retrieval Confidence:

  • On datasets where the retriever performs well (e.g., NQ, RB-cuad), the $m_1$ metric aligns with the theoretical expectation of $1-\alpha$, validating the exchangeability assumption.
  • On datasets with difficult retrieval tasks (e.g., RB-finqa, RB-tatqa), $m_1$ drops significantly, indicating that correct chunks are not appearing in the top-K. In these cases, applying CP-based filtering actually degraded answer consistency by up to 10%, confirming that the statistical guarantees do not hold when the retriever fails to find the source.
  • For strong retrievers, filtering untrusted chunks improved answer consistency by up to 6%.

• Response Confidence:

  • The chunk-wise lookback ratio classifier achieved a maximum test AUROC of 77% in same-group cross-validation.
  • Cross-validation across different dataset groups showed performance drops of up to 16%, highlighting the challenge of generalization.
  • Comparisons showed that Full-Context (FC) lookback ratios (including prompts) yielded higher AUROCs (up to 82%) but were deemed less reliable for factual grounding due to potential bias from prompt patterns. The proposed Chunk-Wise (CW) approach is preferred for dependable systems despite slightly lower AUROC.

Significance and Claims
The paper claims to establish a foundation for certified RAG systems suitable for safety-critical industry applications. Its primary significance lies in shifting from heuristic robustness to statistical certification. The authors emphasize that their approach is not a "silver bullet" for all RAG setups; rather, it provides a diagnostic framework. The paper explicitly states that the validity of the retrieval confidence guarantees depends on the exchangeability of samples, which is contingent on the retriever's ability to surface correct chunks. Therefore, the proposed metrics ($m_1, m_2$) are essential prerequisites for determining if a specific RAG configuration is suitable for deployment with these confidence guarantees. The work aims to guide the development of dependable RAG applications by identifying where the pipeline is weak (retrieval vs. generation) and offering quantifiable measures of trust.