Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation

1. Problem Statement

Retrieval-Augmented Generation (RAG) systems combine Large Language Models (LLMs) with external knowledge retrieval to mitigate hallucinations. However, existing approaches to detecting hallucinations often conflate faithfulness (whether the output is supported by the retrieved context) with factuality (whether the output is objectively true).

• The Flaw in Current Methods: Traditional methods often label any statement not explicitly supported by retrieved documents as a hallucination. This is problematic because an LLM might generate a factually correct statement based on its internal knowledge that happens to be absent from the retrieved context. Conversely, a statement might be faithful to the retrieved context but factually incorrect if the retrieved documents themselves contain errors.
• The Core Challenge: There is a need for a method that distinguishes between these two failure modes:
  1. Faithfulness Failure: The model ignores or contradicts the retrieved evidence.
  2. Factuality Failure: The model generates incorrect information, either by misinterpreting the evidence or by relying on erroneous internal knowledge.

2. Methodology: FRANQ

The authors propose FRANQ (Faithfulness-aware Retrieval Augmented UNcertainty Quantification), a novel framework that decomposes factuality estimation into three probabilistic components based on the relationship between the generated claim and the retrieved context.

The Core Decomposition

FRANQ estimates the probability that a claim $c$ is true using the following law of total probability:
$$P(c \text{ is true}) = P(c \text{ is faithful}) \cdot P(c \text{ is true} | \text{faithful}) + P(c \text{ is unfaithful}) \cdot P(c \text{ is true} | \text{unfaithful})$$

This decomposition allows the system to apply different Uncertainty Quantification (UQ) techniques depending on the nature of the claim:

1. Faithfulness Estimation ($P(c \text{ is faithful})$):

   • Method: Uses AlignScore, a RoBERTa-based metric fine-tuned for factual alignment.
   • Function: Measures the degree to which a claim is semantically entailed by the retrieved documents ($r$). It provides a continuous probability score rather than a binary decision, capturing partial grounding.

2. Factuality under Unfaithful Condition ($P(c \text{ is true} | \text{unfaithful})$):

   • Context: When a claim is not supported by the retrieval, it originates from the LLM's internal (parametric) knowledge.
   • Method:
     • Long-form QA: Uses Parametric Knowledge, calculating the likelihood of the claim based solely on the LLM's internal parameters (inputting the query $x$ without the retrieved context $r$).
     • Short-form QA: Uses the Sum of Eigenvalues method (based on sample diversity) to estimate uncertainty.

3. Factuality under Faithful Condition ($P(c \text{ is true} | \text{faithful})$):

   • Context: When a claim is supported by the retrieval, the error might stem from the model failing to apply the evidence correctly (e.g., selecting the wrong entity from the text).
   • Method:
     • Long-form QA: Uses Max Claim Probability (the likelihood of the claim given the query and context).
     • Short-form QA: Uses Semantic Entropy, which captures uncertainty by measuring the diversity of semantic meanings across multiple sampled generations.

Calibration

To ensure the different UQ components (which may have different probability distributions) are comparable, FRANQ employs Isotonic Regression.

• Condition-Calibrated Strategy: The authors propose a specific calibration where the faithful and unfaithful components are calibrated separately using subsets of training data corresponding to their respective conditions. This prevents miscalibration caused by mixing distributions.

3. Key Contributions

1. FRANQ Framework: A new UQ method that explicitly separates faithfulness and factuality, tailoring uncertainty estimation to the specific failure mode (retrieval grounding vs. internal knowledge).
2. New Dataset (Long-form QA): The authors constructed a new dataset for long-form question answering containing 76 challenging questions.
   • Annotation: It includes fine-grained labels for both faithfulness and factuality at the atomic claim level.
   • Process: Uses a hybrid approach combining automated annotation (GPT-4o-search) with targeted manual validation for difficult cases (False/Unverifiable claims), achieving high inter-annotator agreement (0.87).
3. Comprehensive Evaluation: Extensive experiments across multiple LLMs (Llama 3/8B, Falcon, Gemma) and datasets (both long-form and short-form QA benchmarks like TriviaQA, Natural Questions, etc.).

4. Experimental Results

• Performance: FRANQ (specifically the Condition-Calibrated variant) consistently outperforms existing baselines, including general UQ methods (e.g., Perplexity, Semantic Entropy), RAG-specific baselines (e.g., AlignScore alone), and supervised XGBoost classifiers.
  • Metrics: It achieves the highest PR-AUC (Precision-Recall Area Under Curve) and PRR (Prediction Rejection Ratio) across most models and datasets.
  • Example: On the long-form QA benchmark with Llama 3B, Condition-Calibrated FRANQ achieved a PR-AUC of 0.140, significantly outperforming the best general baseline (0.117).
• Robustness: FRANQ demonstrates strong robustness against retrieval corruption.
  • Irrelevant Retrievals: When 50% of retrieved passages were replaced with unrelated text, FRANQ maintained high performance.
  • Factually Corrupted Retrievals: When retrieved passages contained plausible but false information, FRANQ effectively detected the resulting factual errors, outperforming methods that rely solely on faithfulness.
• Ablation Studies:
  • Component Importance: For long-form QA, the choice of the unfaithful estimator (Parametric Knowledge) was critical. For short-form QA, the method was robust to the choice of estimators.
  • Calibration: Condition-specific calibration was shown to be superior to global calibration or no calibration.
  • Continuous vs. Binary: Using continuous AlignScore probabilities yielded better results than binary thresholding, highlighting the value of probabilistic faithfulness weighting.
• Efficiency: The method is computationally lightweight. The calibration step takes less than one second and results in models of only a few hundred bytes, making it practical for deployment.

5. Significance and Impact

• Redefining Hallucination: The paper argues that "hallucination" in RAG should be defined by factual inaccuracy rather than strict adherence to retrieved context. This prevents penalizing models for generating correct information that the retrieval system failed to find.
• Practical Deployment: By providing a lightweight, self-contained framework that does not require external search agents or complex training, FRANQ offers a practical solution for improving the reliability of RAG systems in knowledge-intensive domains like healthcare, law, and education.
• Dataset Resource: The release of the new long-form QA dataset with dual annotations (faithfulness + factuality) fills a critical gap in the literature, enabling more rigorous evaluation of RAG systems beyond simple context-matching metrics.

In conclusion, FRANQ represents a significant step forward in RAG evaluation by decoupling the source of information (retrieved vs. internal) from the truth value of the output, leading to more accurate and nuanced hallucination detection.