💬 NLP

How Uncertainty Estimation Scales with Sampling in Reasoning Models

This paper demonstrates that while both self-consistency and verbalized confidence improve with sampling in reasoning models, a hybrid estimator combining these signals achieves superior uncertainty estimation with minimal samples, particularly excelling in mathematics compared to STEM and humanities domains.

Original authors: Maksym Del, Markus Kängsepp, Marharyta Domnich, Ardi Tampuu, Lisa Yankovskaya, Meelis Kull, Mark Fishel

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Maksym Del, Markus Kängsepp, Marharyta Domnich, Ardi Tampuu, Lisa Yankovskaya, Meelis Kull, Mark Fishel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but sometimes overconfident student named Reasoning-Robo. When you ask Robo a hard question (like a complex math problem or a tricky history question), it doesn't just blurt out an answer. Instead, it sits down, thinks out loud, writes a long essay of its thought process, and then gives you the final answer.

This paper is about figuring out how to know when Robo is actually sure of its answer, and when it's just guessing.

Here is the breakdown of the study using a simple analogy: The "Group Study" vs. The "Self-Reflection" test.

The Problem: How do we measure doubt?

When you hire a smart AI, you don't want it to confidently give you the wrong answer. You want it to say, "I'm not sure about this," so you can double-check. But how do you measure that "uncertainty" without peeking inside the robot's brain (which is impossible)?

The researchers tested two main ways to ask Robo how sure it is:

The "Self-Reflection" Method (Verbalized Confidence): You ask Robo, "On a scale of 1 to 100, how confident are you that your answer is right?"
The "Group Study" Method (Self-Consistency): You ask Robo the same question 5 or 10 times. If it gives you the same answer every time, it's probably confident. If it gives you 10 different answers, it's confused.

The Big Discovery: The "Two-Person Team" Wins

The researchers tried these methods alone and together, using different numbers of "tries" (samples). Here is what they found, translated into everyday terms:

1. The "Self-Reflection" is a Good Start, but gets tired fast.

Asking Robo once, "How sure are you?" works pretty well. It's like asking a student, "Do you know this?" and them saying, "Yeah, I'm pretty sure."

The Catch: If you ask Robo to do this 8 times to get a better average, the improvement is small. It's like asking the same student the same question 8 times; they just repeat the same feeling.

2. The "Group Study" is slow to start.

Asking Robo to answer the same question 8 times and seeing if they agree (Self-Consistency) is a bit like asking 8 different students to solve a problem.

The Catch: In the beginning (with just 2 or 3 tries), this method is actually worse than just asking Robo once how sure it is. It takes a lot of "tries" before the group starts agreeing enough to be useful.

3. The Magic Combo: The "Two-Person Team" (The Hybrid)

This is the most important finding.
Instead of asking Robo to do 8 deep-thought sessions alone, the researchers found that asking it just twice and combining the two methods is a game-changer.

The Strategy:
1. Ask Robo the question once and ask, "How sure are you?" (Self-Reflection).
2. Ask Robo the question a second time and see if the answer matches the first one (Group Study).
3. Combine the two signals.

The Analogy: Imagine you are trying to guess the winner of a horse race.

Method A: You ask one expert, "How sure are you?" (Good, but maybe biased).
Method B: You ask 8 different experts to pick a horse, and see if they agree (Slow, takes a lot of effort).
Method C (The Winner): You ask one expert how sure they are, and then ask a second expert to pick a horse. If the second expert picks the same horse the first one is confident about, you are extremely confident. If they pick a different horse, you know to be careful.

The paper found that this "Two-Person Team" approach was significantly better than asking one person to think really hard 8 times. It gave the most accurate "doubt meter" with the least amount of work.

The "Math vs. History" Difference

The study also looked at what kind of questions Robo was answering.

Math Problems: Robo is like a math whiz who has been trained specifically for this. In math, the "Two-Person Team" worked incredibly well. The signals (confidence + agreement) complemented each other perfectly.
History/Humanities: Robo is a bit more like a generalist here. The "Two-Person Team" still worked, but the improvement wasn't as dramatic as in math. It's like asking a math whiz to guess the plot of a movie; they might be good, but they aren't as specialized.

The Takeaway for Real Life

If you are building or using AI systems that need to be safe and reliable (like medical diagnosis or legal advice):

Don't just rely on one deep thought. Asking the AI to think really hard once isn't enough to know if it's right.
Don't just ask it to think 100 times. That's too expensive and slow.
Do the "Two-Step Dance": Ask the AI to solve the problem twice.
- Check if the answers match.
- Check how confident the AI said it was.
- Combine those two pieces of info.

This simple trick gives you the best "lie detector" for AI uncertainty, saving you time and money while making the AI much more trustworthy. It turns a "maybe" into a "probably" or a "definitely not" with very little extra effort.

1. Problem Statement

Reasoning Language Models (RLMs) extend standard Large Language Models (LLMs) by employing extended Chain-of-Thought (CoT) deliberation. While this improves answer accuracy, it introduces a critical challenge for uncertainty estimation:

High Cost of Sampling: In RLMs, generating a single sample involves a long, computationally expensive reasoning trace. Therefore, traditional uncertainty estimation methods relying on massive parallel sampling (e.g., $K=64$ or $100$) are often impractical due to cost.
Unknown Scaling Dynamics: It is unclear how uncertainty signals behave as the number of samples ( $K$ ) increases in RLMs. Specifically, does the extended internal deliberation within a single sample reduce the marginal value of additional samples?
Signal Interaction: It is unknown how introspective signals (verbalized confidence) and agreement-based signals (self-consistency) interact and complement each other in the context of extended reasoning, compared to standard non-reasoning models.

2. Methodology

The authors conducted a large-scale empirical study using a fully black-box approach across three open-source reasoning models and 17 tasks.

Models:
- gpt-oss-20b (Mixture-of-Experts, RLVR-trained).
- Qwen3-30B-A3B (Mixture-of-Experts, RLVR-trained).
- DeepSeek-R1-8B (Dense, fine-tuned on reasoning traces).
Tasks: 17 tasks spanning three domains:
- Mathematics: GSM8K, AIME 2024/2025, MMLU-Pro Math.
- STEM: Health, Biology, Chemistry, Economics, Physics, CS, Engineering, GPQA Diamond.
- Humanities: Psychology, Law, Business, History, Philosophy, etc.
Uncertainty Signals Evaluated:
1. Verbalized Confidence (VC): The model explicitly outputs a confidence score (0–100) alongside its answer. The study tested various elicitation prompts (Vanilla, Verification, Epistemic) and "Judge" methods (where a second pass evaluates the trace).
2. Self-Consistency (SC): Confidence derived from the agreement rate of answers across $K$ independent samples (majority vote).
3. Hybrid Estimator (SCVC): A linear combination of VC and SC ( $SCVC = \lambda \cdot SC + (1-\lambda) \cdot VC$ ).
Evaluation Metric: AUROC (Area Under the Receiver Operating Characteristic Curve). The authors chose AUROC over calibration metrics (like ECE) because VC is often self-reported and mis-scaled, and SC is a discrete statistic. AUROC measures the signal's ability to discriminate between correct and incorrect answers regardless of scale.
Protocol:
- Generated a pool of $R=10$ independent samples per question.
- Used bootstrap sampling to estimate performance at varying budgets ( $K \in \{1, 2, 5, 8\}$ ).
- Analyzed scaling behavior and signal complementarity.

3. Key Contributions

First Systematic Benchmark: The first large-scale study characterizing how black-box uncertainty estimation scales with parallel sampling specifically in Reasoning Language Models.
Signal Complementarity Discovery: Demonstrated that combining introspection (VC) and agreement (SC) yields significantly higher gains than increasing the sampling budget for either signal alone.
Domain-Dependent Scaling: Identified that uncertainty signals scale fastest and combine most effectively in Mathematics, aligning with the RLVR (Reinforcement Learning with Verifiable Rewards) training focus of these models.
Practical Recipe: Established a cost-effective strategy for uncertainty estimation in RLMs: Use only 2 samples and combine VC and SC, rather than deep sampling of a single signal.

4. Key Results & Findings

Finding I: Self-Consistency Lags Behind Verbalized Confidence

VC is Strong Baseline: Even with a single sample ( $K=1$ ), VC provides strong discrimination (e.g., ~78.8 AUROC on MMLU-Pro Math).
SC Starts Low: Self-consistency starts substantially lower than VC at comparable budgets (e.g., 70.5 vs. 73.5 AUROC at $K=2$ in Math).
Scaling: VC scales well in Math (gaining +10.1 AUROC from $K=1$ to $K=8$ ) but saturates earlier in STEM and Humanities. SC improves with more samples but fails to overtake VC within the tested range ( $K \le 8$ ).

Finding II: Signal Complementarity Dominates

Hybrid Superiority: Combining VC and SC with just two samples ( $K=2$ $K = 2$ ) yields massive gains.
- Math: SCVC at $K=2$ achieves 84.2 AUROC, outperforming single-sample VC (71.3) by +12.9 points. It even exceeds the performance of VC or SC scaled to $K=8$ .
- STEM/Humanities: SCVC at $K=2$ improves over single-sample VC by +6.4 points and outperforms single signals scaled to $K=8$ .
Diminishing Returns: Further increasing $K$ beyond 2 yields diminishing returns, especially in non-math domains. The hybrid estimator reaches near-optimal performance quickly.

Finding III: Domain Dependence (Math vs. Others)

Mathematics: Shows the strongest scaling and complementarity. The hybrid signal continues to improve significantly from $K=2$ to $K=5$ and $K=8$ .
STEM/Humanities: Saturate earlier (around $K=2$ or $K=5$ ) with smaller gains.
Correlation Analysis: VC and SC are less correlated in Mathematics (especially at low $K$ ), indicating they capture distinct uncertainty information. In non-math domains, they correlate more strongly, suggesting redundancy.

Finding IV: Elicitation Variants

Simplicity Wins: In Mathematics, "Vanilla" elicitation (asking for a confidence number directly) performed best. Complex "Epistemic" or "Judge" prompts did not significantly improve performance and sometimes underperformed.
Cost-Benefit: Using a "Judge" (a second model pass) adds computational cost without the significant gains achieved by simply adding a second sample for the hybrid estimator.

5. Significance and Implications

Efficiency in RLMs: Since each sample in an RLM is expensive (long CoT), the finding that $K=2$ with a hybrid estimator outperforms deep sampling ( $K=8$ ) is crucial for deployment. It suggests that users should prioritize signal diversity (combining VC and SC) over sampling depth.
Deployment Strategy: For reliable uncertainty estimation in reasoning models, the optimal strategy is:
1. Generate 2 samples.
2. Extract Verbalized Confidence from both.
3. Calculate Self-Consistency (agreement) between the two.
4. Combine them (e.g., average or weighted sum).
Domain Specificity: The results highlight that RLMs are currently most reliable and "self-aware" in Mathematics due to RLVR training. Uncertainty estimation in Humanities and general STEM remains more challenging and saturates faster.
Theoretical Insight: The study challenges the assumption that extended reasoning makes cross-sample agreement redundant. Instead, it shows that introspection and agreement remain highly complementary even after deep deliberation.

Conclusion

The paper provides a definitive guide for uncertainty estimation in reasoning models. It argues against the naive application of standard LLM sampling strategies (deep sampling) and instead advocates for a hybrid, low-budget approach. By leveraging the complementarity of verbalized confidence and self-consistency with minimal sampling ( $K=2$ ), practitioners can achieve state-of-the-art uncertainty discrimination at a fraction of the computational cost.