💬 NLP

Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution

This paper proposes a method for improving large language model confidence estimation by prompting the model to verbalize a full probability distribution rather than a single guess, demonstrating that this approach enhances reasoning efficacy, achieves superior performance across tasks, and significantly reduces computational costs during inference-time scaling compared to existing baselines.

Original authors: Ante Wang, Weizhi Ma, Yang Liu

Published 2026-04-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ante Wang, Weizhi Ma, Yang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a very smart, well-read friend (a Large Language Model, or LLM) a difficult question. You don't just want the answer; you want to know how sure they are about it.

If your friend says, "The capital of France is Paris," you trust them. But if they say, "The capital of France is... well, I'm 99% sure it's Paris, but maybe it's Lyon?" you need to know if that 99% is a genuine feeling or just them guessing confidently.

This paper is about teaching AI to be a better judge of its own certainty. Here is the breakdown using simple analogies.

The Problem: The Overconfident Friend

Currently, AI models are great at generating text, but they often suffer from overconfidence. They might give a wrong answer with 100% certainty because they are trained to sound helpful and decisive.

The Old Way: The AI just picks an answer and says, "I'm 90% sure." It's like a student taking a test, guessing "A," and immediately writing "90%" next to it without thinking about why it might be wrong.
The Flaw: This often leads to "hallucinations" where the AI is confidently wrong. In high-stakes fields like medicine or law, this is dangerous.

The Solution: Let the Model "Distribute Its Doubt"

The authors propose a new method called Verbalized Probability Distribution. Instead of asking the AI to pick one answer and guess a confidence score, they ask it to list all possible answers and assign a probability to each one.

Think of it like this:

Old Method (Verbalized Confidence): The AI is a gambler placing a single bet. "I bet on Red!" (It doesn't think about Black or Green).
New Method (Verbalized Distribution): The AI is a weather forecaster. Instead of just saying "It will rain," it says: "There is a 60% chance of rain, a 30% chance of clouds, and a 10% chance of sun."

How It Works: The "All Options" Rule

The researchers found that when you force the AI to consider every possible option (even the silly ones) and make sure all the percentages add up to 100%, something magical happens:

It thinks harder: The AI has to weigh the pros and cons of every option, not just the one it likes.
It becomes honest: If the AI is confused, it can't just dump 100% confidence on one wrong answer. It has to spread its "doubt" around. Maybe it gives 40% to Answer A, 30% to Answer B, and 30% to "I don't know."
The "None of the Above" Safety Net: For open-ended questions where there are infinite answers, the AI is told to include an option called "None of the above." This acts as a bucket for all the uncertainty that doesn't fit the specific guesses.

The "Coach" (Reinforcement Learning)

The paper also tested training the AI using a "coach" (Reinforcement Learning).

Imagine a coach watching the AI play a game. Every time the AI gives a confident answer that turns out to be wrong, the coach gives a penalty. Every time the AI distributes its probabilities correctly (matching its confidence to how often it's actually right), it gets a reward.
Result: The AI learns to be a better judge of its own skills. It learns that being "calibrated" (accurate in its confidence) is more important than just being loud and confident.

Why This Matters: The "Efficiency" Win

One of the coolest findings is about speed and cost.

Usually, to get a better answer, you have to ask the AI to think longer or run the calculation many times (like asking a friend to think about a problem for an hour).
This new method gets better results without needing that extra time. It's like the AI suddenly learned to "think before it speaks" just by changing the rules of the game. The paper notes it saved nearly 6x the computation to reach the same level of accuracy as other complex methods.

The Catch: It's Not Perfect for Everything

The method works great for questions with multiple choices (like medical exams or trivia) or open-ended questions where there are many possibilities.
However, for math problems, it's a bit trickier. In math, there is usually only one right answer, and the steps to get there are absolute. If you ask a math AI to guess probabilities for "2+2=5" and "2+2=4," it might get confused because the answer isn't a matter of opinion; it's a fact. The paper admits this method needs some tweaking for pure math reasoning.

The Big Picture

This paper teaches us that uncertainty is a feature, not a bug. By forcing AI models to explicitly show their "doubt" and spread their confidence across all possibilities, we get:

Safer AI: We know when to trust the AI and when to double-check it.
Smarter AI: The act of listing all options forces the model to reason better.
Cheaper AI: We get better results without needing supercomputers to run the model longer.

In short: Don't just ask the AI "What is the answer?" Ask it, "What are all the possibilities, and how likely is each one?" That simple shift makes the AI much more reliable.

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed in high-stakes domains (e.g., healthcare, law, finance), yet they inevitably make errors. A critical gap in their deployment is the lack of well-calibrated confidence estimates.

The Issue: Traditional methods relying on token probabilities often fail because modern LLMs (especially those fine-tuned with RLHF) may sacrifice calibration to better follow user instructions, leading to overconfidence.
Limitations of Current Solutions:
- Verbalized Confidence: Asking the model to state a single confidence score often results in overconfidence because the model focuses on a single guess without considering alternatives.
- Verbalized Top- $k$ : Asking for $k$ guesses improves calibration but often fails to enforce a valid probability distribution (e.g., the sum of probabilities may exceed 100%), leading to infeasible confidence quantification.
Research Gap: It remains unclear what specific reasoning strategies effectively elicit accurate confidence estimates. Specifically, how forcing a model to reason about the entire space of possibilities affects its uncertainty estimation is under-explored.

2. Methodology: Verbalized Probability Distribution

The authors propose Verbalized Probability Distribution, a prompting strategy that forces the LLM to articulate its belief state over the entire answer space rather than focusing on a single prediction.

Core Mechanism:

Prompting: The model is instructed to:
- Identify all possible answers (or a representative set of candidates).
- Assign a confidence score to each candidate.
- Ensure the sum of all confidence scores equals 1.0 (a valid probability distribution).
Handling Open-Set Tasks: For tasks where enumerating all answers is impossible (open-set), the method includes a "None of the above" option. This aggregates the probability mass of low-likelihood candidates, ensuring the distribution remains valid.
Optimization via RL: The authors explore optimizing this process using Reinforcement Learning with Calibration Rewards (RLCR).
- Reward Function: Based on the Brier score ( $r = y - (c-y)^2$ ), where $y$ is correctness and $c$ is confidence.
- Constraint: A strict penalty ( $r = -1$ ) is applied if the output does not sum to 1, enforcing the probability distribution constraint during training.

Comparison with Baselines:

Verbalized Confidence: Single guess + single score (Prone to overconfidence).
Verbalized Top- $k$ : $k$ guesses + scores (Mitigates overconfidence but lacks distributional rigor).
Verbalized Distribution: Full belief state + normalized scores (Forces consideration of alternatives and logical consistency).

3. Key Contributions

Novel Prompting Strategy: Introduced a method requiring LLMs to generate a normalized probability distribution over possible answers, effectively forcing the model to "distribute its doubt" rather than fixating on a single hypothesis.
Systematic Evaluation: Conducted extensive experiments across multiple LLMs (Qwen3 series, GPT-4.1, DeepSeek-V3, Llama-3) and diverse tasks (Closed-set: MedQA, MMLU-Pro; Open-set: HotpotQA, MedCaseReasoning).
RL Optimization: Demonstrated that Reinforcement Learning can further refine this method, leading to faster convergence and better out-of-domain generalization compared to other verbalization methods.
Efficiency Analysis: Showed that the proposed method achieves superior confidence estimation with significantly lower computational costs (fewer tokens) during inference-time scaling compared to baselines.

4. Experimental Results

The paper evaluates performance using AUROC (discrimination), ECE (calibration error), and Brier Score (overall accuracy of probability estimates).

Training-Free Prompting:
- Verbalized Distribution consistently outperformed Verbalized Confidence and Verbalized Top- $k$ across all models and tasks.
- It achieved the lowest Brier scores and ECE, indicating superior calibration.
- Small Models Benefit Most: Smaller LLMs (e.g., Qwen3-4B) showed the most significant gains, suggesting the method helps smaller models reason more effectively about uncertainty.
Reinforcement Learning (RL):
- RL training improved all methods, but Verbalized Distribution maintained the lead.
- It demonstrated faster convergence during training (reaching optimal performance in fewer steps).
- It showed robust performance on Out-of-Distribution (OOD) tasks (e.g., training on MedQA, testing on MMLU-Pro).
Inference-Time Scaling:
- When combining methods with answer consistency (sampling multiple trajectories), Verbalized Distribution was the most sample-efficient.
- It reached the best Brier score of the strongest baseline on MMLU-Pro with nearly 6 $\times$ less computation.
Ablation Studies:
- Removing the "None of the above" option caused significant performance drops, especially in open-set tasks.
- Removing the normalization requirement (sum=1) degraded calibration, proving that the constraint is crucial for forcing the model to reason about relative likelihoods.

5. Significance and Limitations

Significance:

Trustworthy AI: Provides a practical, training-free (or lightly trained) mechanism to make LLMs more reliable in critical applications by accurately signaling when they are unsure.
Reasoning Mechanism: Reveals that the act of distributing probability forces the model to engage in deeper reasoning, considering alternatives and avoiding the "sunk cost" of a single initial guess.
Efficiency: Offers a computationally efficient path to high-quality confidence estimation, crucial for scaling LLM applications.

Limitations:

Task Dependency: The method struggles with tasks where the answer space is inherently singular or deterministic (e.g., strict mathematical reasoning), where "distributing doubt" is less natural.
Token Overhead: Generating a full distribution consumes more tokens in a single pass compared to a single answer, though it is more efficient than repeated sampling.
Prompt Sensitivity: Effectiveness varies by task; automatic selection of prompting strategies based on task features is suggested as future work.

Conclusion

The paper establishes that Verbalized Probability Distribution is a superior approach for confidence estimation. By forcing LLMs to explicitly model their uncertainty across the entire answer space and adhere to probability axioms, the method significantly improves calibration and reasoning efficacy, particularly for smaller models and complex reasoning tasks.