Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal

Technical Summary: UniCR – A Unified Framework for Confidence Calibration and Risk-Controlled Refusal

Problem Statement
Deployed large language models (LLMs) face a critical challenge: they must not only determine what to answer but also when to abstain. Current systems often lack reliable mechanisms to distinguish between high-confidence correct answers and high-confidence hallucinations, particularly under distribution shifts or when generating long-form content. Existing approaches, such as simple entropy or logit thresholds, post-hoc calibrators, and end-to-end selective baselines, often fail to provide robust, distribution-free guarantees or to effectively fuse heterogeneous sources of uncertainty evidence.

Methodology: The UniCR Framework
The paper introduces UniCR, a unified framework designed to transform diverse uncertainty signals into a calibrated probability of correctness and enforce user-specified error budgets through principled refusal. The methodology operates through the following stages:

1. Heterogeneous Evidence Fusion: UniCR aggregates multiple sources of uncertainty evidence, including:
   • Sequence likelihoods.
   • Self-consistency dispersion.
   • Retrieval compatibility.
   • Feedback from tools or verifiers.
2. Calibration Head: A lightweight calibration head is trained using temperature scaling and proper scoring rules. This component maps the fused evidence into a calibrated probability of correctness. Crucially, this design supports API-only (black-box) models by relying on features extractable without internal model access.
3. Risk-Controlled Refusal: The framework employs conformal risk control to provide distribution-free guarantees. This allows the system to enforce a strict, user-defined error budget, ensuring that the model refuses to answer when the predicted risk exceeds the threshold.
4. Long-Form Generation Alignment: For long-form tasks, UniCR aligns confidence scores with semantic fidelity. It supervises the calibration process using atomic factuality scores derived from retrieved evidence. This specific alignment aims to reduce "confident hallucinations" while maintaining high coverage of valid information.

Key Contributions

• Unified Architecture: A single framework that integrates heterogeneous uncertainty signals (likelihood, consistency, retrieval, tool feedback) into a coherent decision-making process.
• Black-Box Compatibility: The ability to calibrate and control risk for models accessible only via API, without requiring fine-tuning of the base model.
• Distribution-Free Guarantees: The use of conformal risk control ensures validity even under distribution shifts, a common failure point for standard calibration methods.
• Factuality-Aware Calibration: A novel approach for long-form generation that ties confidence directly to atomic factuality derived from external evidence, specifically targeting the reduction of hallucinations.

Experimental Results
The paper evaluates UniCR across three distinct domains:

1. Short-form Question Answering (QA)
2. Code Generation (evaluated via execution tests)
3. Retrieval-Augmented Long-form QA

The results demonstrate consistent improvements over baseline methods (including entropy/logit thresholds, post-hoc calibrators, and end-to-end selective baselines). Specifically, UniCR achieves:

• Improved calibration metrics.
• Lower Area Under the Risk-Coverage Curve (AURC).
• Higher coverage at fixed risk levels.

Analysis of Refusal Drivers
An analysis of the refusal mechanism reveals that evidence contradiction, semantic dispersion, and tool inconsistency are the primary drivers of abstention. The framework successfully translates these internal signals into informative, user-facing refusal messages, enhancing transparency.

Significance and Claims
The paper positions UniCR as a "portable recipe" that moves from evidence fusion to calibrated probability to risk-controlled decision-making. Its primary significance lies in improving the trustworthiness of LLMs without the need to fine-tune the underlying base model. The authors claim that the framework remains valid under distribution shift, offering a practical solution for deploying safer, more reliable language models in real-world scenarios where error budgets must be strictly controlled.