Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models

1. Problem Statement

Large Language Models (LLMs) frequently generate fluent but factually incorrect outputs (hallucinations), posing significant risks in high-stakes domains like medicine. While Conformal Prediction (CP) has been applied to LLMs to provide statistical guarantees on factuality (ensuring that the true answer lies within a prediction set with probability $1-\alpha$), existing methods suffer from a critical limitation: lack of prompt adaptivity.

• The Issue: Standard CP methods apply a single, global calibration threshold (quantile) uniformly across all test prompts, regardless of the prompt's difficulty, ambiguity, or domain.
• The Consequence: This leads to marginal coverage (correct on average) but poor conditional coverage.
  • For "easy" prompts, the method may be overly conservative (over-coverage), filtering out too few items and retaining unnecessary noise.
  • For "hard" prompts, the method may be insufficiently strict (under-coverage), failing to filter out hallucinations.
• Goal: The authors aim to develop a method that adapts the confidence threshold based on the specific input prompt, improving conditional coverage while maintaining rigorous marginal guarantees.

2. Methodology

The paper proposes an Adaptive Conformal Prediction framework that extends conformal score transformation methods to LLMs. The core idea is to normalize nonconformity scores based on the input prompt's characteristics before calculating the global threshold.

A. Core Framework

1. Nonconformity Score ($V$):
   • Long-form QA: The model generates a set of atomic claims. $V(x, y)$ is defined as the maximum uncertainty threshold such that all retained claims are factually correct (verified against a ground truth $y$).
   • Multiple-Choice QA: Uses the Least Ambiguous Classifier (LAC) score, where $V(x, y) = 1 - p(x)_y$ (1 minus the predicted probability of the true class).
2. Score Transformation (Adaptivity):
   • Instead of using raw scores $V(x, y)$, the method transforms them using an input-dependent function: $\tilde{V}(x, y) = V(x, y) / \hat{\tau}(x)$.
   • Here, $\hat{\tau}(x)$ is an estimate of the conditional $(1-\alpha)$-quantile of the original score given the prompt $x$.
   • This transformation effectively rescales the scores so that their conditional distributions are approximately invariant across different inputs.
3. Conditional Quantile Estimation:
   • The authors use embedding-conditioned quantile regression to estimate $\hat{\tau}(x)$.
   • Prompt embeddings (derived from a multilingual sentence encoder) serve as the input features for a Multi-Layer Perceptron (MLP) trained to predict the quantile of the nonconformity score.
4. Two-Stage Calibration:
   • $D_{cal1}$: Used to train the conditional quantile estimator $\hat{\tau}(x)$.
   • $D_{cal2}$: Used to compute the transformed scores $\tilde{V}$ and determine the final global threshold $Q_{1-\alpha}$.
   • $D_{test}$: At inference, claims are filtered if their normalized uncertainty score $s(c)/\hat{\tau}(x)$ is below the calibrated threshold.

B. Theoretical Guarantee

The method preserves the finite-sample marginal coverage guarantee of split conformal prediction:
$$P(\text{True answer} \in \text{Filtered Set}) \geq 1 - \alpha$$
Simultaneously, it significantly improves conditional coverage, ensuring the guarantee holds more consistently across heterogeneous prompt categories.

3. Key Contributions

1. Novel Adaptive Framework: Proposes a prompt-adaptive conformal prediction approach for LLM hallucination detection using embedding-conditioned quantile regression.
2. Theoretical Validity: Demonstrates that the method maintains the rigorous marginal coverage guarantees of standard conformal prediction while achieving superior conditional coverage.
3. Empirical Superiority: Shows significant performance gains over existing baselines (like Conformal Factuality) in both long-form generation and multiple-choice QA across diverse models (Mistral, Llama, Gemma) and domains.
4. Selective Prediction: The approach naturally supports selective prediction, allowing unreliable claims or answer choices to be filtered out in downstream applications without sacrificing statistical validity.

4. Experimental Results

The authors evaluated their method on 8 categories of long-form QA (e.g., Inventions, Biographies) and 16 categories of Multiple-Choice QA (MMLU dataset) using Mistral-7B, Llama-3.1-8B, and Gemma-3-12B.

• Claim Scoring: Among various uncertainty quantification methods (Entropy, Perplexity, etc.), Claim Conditioned Probability (CCP) was found to be the most effective for filtering claims.
• Conditional Coverage:
  • Long-form QA: In experiments comparing "Inventions" (complex) vs. "Biographies" (simple), standard CP showed over-coverage for Inventions and under-coverage for Biographies. The adaptive method achieved coverage close to the target (0.80) for both categories.
  • Multiple-Choice QA: Similar improvements were observed across 16 MMLU categories. The adaptive method reduced the variance in coverage errors, with the Dolan-Moré performance profiles showing it consistently outperformed the global baseline across all $\alpha$ levels.
• Efficiency: The adaptive method often reduced the fraction of removed claims (false negatives) while maintaining the target coverage, indicating a more efficient use of the model's generation capabilities.

5. Significance

This work addresses a fundamental weakness in applying conformal prediction to the heterogeneous nature of LLM inputs. By moving from global to prompt-adaptive thresholds, the paper provides a practical solution for:

• Reliability: Ensuring that factuality guarantees hold not just on average, but for specific, difficult, or rare queries.
• Safety: Enabling safer deployment of LLMs in critical fields by dynamically adjusting confidence thresholds based on input complexity.
• Scalability: Demonstrating that learned representations (embeddings) can effectively replace hand-crafted grouping or feature engineering to achieve adaptivity.

The proposed framework bridges the gap between theoretical statistical guarantees and the practical, variable difficulty of real-world LLM prompting, offering a robust tool for hallucination detection and selective generation.