LLMs Should Express Uncertainty Explicitly

1. Problem Statement

Large Language Models (LLMs) are probabilistic systems that inherently carry uncertainty. However, current downstream applications (e.g., Adaptive RAG, tool use, verification) often rely on uncertainty signals that are latent or post-hoc.

• The Gap: Existing methods typically treat uncertainty as a hidden state to be estimated after generation (e.g., via entropy, log-probabilities, or external probes). This requires downstream controllers to "read between the lines" of the model's behavior, which is indirect and often unreliable.
• The Core Question: How should uncertainty be exposed in LLMs to make it a legible, actionable interface for decision-making? The authors argue that uncertainty should be treated as a communication interface rather than just a diagnostic metric.

2. Methodology

The paper proposes and compares two complementary interfaces for expressing uncertainty, both trained within a unified post-training framework using GRPO (Group Relative Policy Optimization):

A. Global Interface: Verbalized Confidence

• Mechanism: The model generates a scalar confidence score (e.g., "Confidence: 0.85") after completing its reasoning trajectory and final answer.
• Training Objective: A confidence-aware reward function is applied:
  • $r = p$ if the answer is correct.
  • $r = -p$ if the answer is wrong.
  • This rewards justified confidence and penalizes overconfident errors.
• Goal: To improve the calibration of the final answer, allowing downstream systems to decide whether to trust the output or abstain.

B. Local Interface: Reasoning-Time Signaling

• Mechanism: The model emits a special token <uncertain> during the reasoning process when it encounters a high-risk state (missing fact, ambiguity).
• Training Objective: A hierarchical reward structure:
  • $r(\text{correct}, \text{no emit}) > r(\text{correct}, \text{emit}) > r(\text{wrong}, \text{emit}) > r(\text{wrong}, \text{no emit})$.
  • Crucially, silent failure (wrong answer without emitting <uncertain>) is penalized most heavily.
• Goal: To surface "silent failures" as explicit intervention points, allowing retrieval systems to trigger before the model commits to a wrong trajectory.

3. Key Contributions

1. Interface Design Framework: The authors frame uncertainty as a multi-scale design problem, distinguishing between global signals (for final answer trust) and local signals (for mid-reasoning intervention).
2. Verbalized Confidence as a Global Controller: Demonstrated that training models to verbalize confidence significantly improves calibration (reducing Expected Calibration Error from 0.383 to 0.049) and suppresses overconfident hallucinations without degrading reasoning accuracy.
3. Reasoning-Time Signaling as a Local Controller: Showed that training models to emit <uncertain> converts previously silent epistemic failures into visible events. This increases the coverage of wrong answers detected by downstream retrieval controllers from ~15% to ~88%.
4. Mechanistic Insights: Provided deep analysis into how these interfaces work internally:
   • Verbal Calibration: Acts as a geometry-preserving readout. It sharpens an existing latent confidence manifold in the late layers without drastically rewriting the reasoning trajectory.
   • Local Signaling: Induces a broader late-layer reorganization. The model must actively construct a new internal state to emit the <uncertain> token, changing the representation geometry significantly.

4. Key Results

Calibration Performance

• Verbal Interface:
  • ECE (Expected Calibration Error): Reduced from 0.383 (Base) to 0.049 (Calibrated).
  • Overconfidence Gap: Reduced from +0.523 to +0.045.
  • Error Shift: The model shifted from being "confidently wrong" (Epistemic errors) to "uncertain when wrong" (Aleatoric errors). Epistemic errors dropped from 92.4% to 34.9% of total errors.
  • Accuracy: Slight improvement (0.345 $\to$ 0.358).

Retrieval Control (Adaptive RAG)

• Global (Verbal) Controller: Achieved the best overall Adaptive RAG performance (41.6% EM, 50.5% F1) with a moderate trigger rate (48.1%). It outperformed strong baselines like ADARAGUE and DRAGIN by retrieving more selectively.
• Local (Token) Controller: Achieved high recall for intervention (88.2% coverage of wrong answers) but triggered retrieval more aggressively (61.4% rate). It excelled in datasets requiring multi-hop reasoning where early intervention is critical.
• Comparison: Both learned interfaces significantly outperformed post-hoc baselines (e.g., temperature scaling, entropy-based triggers) and non-adaptive baselines.

Mechanistic Analysis

• Logit Lens & PCA: Calibration sharpens the mapping from hidden states to confidence outputs. The base model has a diffuse confidence manifold; the calibrated model organizes this into a clean, monotonic low-to-high confidence axis.
• CKA (Centered Kernel Alignment):
  • Verbal: CKA remains near 1.0 across layers, indicating the internal reasoning geometry is preserved; only the final readout is refined.
  • Local: CKA diverges in late layers, indicating the model rewrites its internal computation to support the explicit emission of the uncertainty token.

5. Significance and Conclusion

The paper argues that effective uncertainty in LLMs is not a single output format but a task-matched communication strategy:

• Global Confidence is best for deciding whether to trust a final answer (abstention, verification).
• Local Signaling is best for deciding when to intervene during reasoning (retrieval, tool use).

Practical Implication: By training models to explicitly express uncertainty in these specific forms, developers can build more robust, efficient, and controllable AI systems. The model becomes a partner that knows when it is unsure, rather than a black box that must be probed for hidden signals. This approach moves uncertainty from a latent diagnostic property to an explicit, trainable, and actionable interface feature.