💬 NLP

Tracing Uncertainty in Language Model "Reasoning"

This paper introduces a method to quantify and analyze the uncertainty profiles of language model reasoning traces, demonstrating that distinct patterns in these profiles can predict answer correctness with high accuracy and enable early error detection.

Original authors: Nils Grünefeld, Bertram Højer, Philipp Mondorf, Barbara Plank, Anna Rogers, Christian Hardmeier, Stefan Heinrich, Jes Frellsen

Published 2026-05-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nils Grünefeld, Bertram Højer, Philipp Mondorf, Barbara Plank, Anna Rogers, Christian Hardmeier, Stefan Heinrich, Jes Frellsen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a large language model (LM) as a student taking a very long, complex math or logic test. Instead of just writing down the final answer, the student is asked to "show their work" by writing out a long, step-by-step thought process first. This is what researchers call "Chain-of-Thought" or "reasoning."

The big question this paper asks is: Can we tell if the student is going to get the answer right while they are still writing, just by looking at how "confident" they seem to be at each step?

Here is the breakdown of their discovery, using simple analogies:

1. The "Confidence Meter"

Usually, when a model generates text, it picks the next word based on probability. The authors decided to treat this process like a confidence meter that fluctuates as the student writes.

The Trace: The long string of words the model writes before the final answer is the "trace."
The Uncertainty: At every single word, the model has a level of "uncertainty" (how unsure it is about what comes next).
The Profile: Instead of just looking at the final result, the authors mapped out the shape of this uncertainty over time. They called this an "uncertainty trace profile."

2. Two Types of "Confusion"

The paper distinguishes between two different kinds of confusion the model might feel, using a coin flip analogy:

Aleatoric Uncertainty (The "Randomness" Confusion): This is like flipping a fair coin. Even if you know everything about the coin, you can't predict the next flip. It's just inherently random. In the model, this happens when the model is genuinely torn between two or many options.
Epistemic Uncertainty (The "Knowledge" Confusion): This is like flipping a coin you've never seen before. You don't know if it's fair or weighted. You are confused because you lack information or training data about this specific situation.

3. The "Shape" of Success vs. Failure

The researchers looked at thousands of examples of correct and incorrect answers. They found that the shape of the confidence meter tells a very clear story:

The "Good" Student (Correct Answer): As the student writes their reasoning, their confidence grows steadily. The "uncertainty meter" drops sharply and smoothly, like a skier going down a steep, straight hill. They are getting more sure of themselves as they get closer to the finish line.
The "Struggling" Student (Incorrect Answer): Their confidence meter is messy. It doesn't drop as fast, and the path is wobbly and irregular. They seem to be "eliminating" wrong options one by one rather than naturally "concentrating" on the right one.

The Key Finding: By looking at just the first 300 words of a student's "thinking" (before they even finish the problem), the authors could predict with 80% accuracy whether the final answer would be right or wrong. This is much better than previous methods that tried to guess based on the final answer alone or by counting how many times the model repeated itself.

4. The "Trap" of False Confidence

One of the most surprising discoveries was a "trap" in how the models fail.

When a model gets the answer wrong, the steps it took to get there often look very confused (high randomness/aleatoric uncertainty).
However, the final wrong answer it lands on often looks surprisingly confident and familiar (low knowledge-based/epistemic uncertainty).
The Analogy: Imagine a hiker who is wandering aimlessly in the woods (high confusion during the walk) but eventually stumbles into a familiar parking lot (low confusion at the end). The hiker thinks, "Ah, I'm in a safe place!" because the parking lot looks like one they've seen before, even though they took a terrible path to get there. The model is confident in the wrong answer because it looks like something it has seen in its training data, even though the logic used to get there was shaky.

5. Why This Matters (According to the Paper)

The paper claims that by treating the "reasoning" process as a time-series of confidence levels, we can:

Spot errors early: We don't have to wait for the model to finish writing to know it's likely wrong.
Understand the "Why": We can see how the model fails. It's not just that the answer is wrong; it's that the path to the answer was "wobbly" and the model was unsure of its steps, even if it felt sure about the destination.

In short, the authors built a "lie detector" for AI reasoning. It doesn't read the words to check if they make sense; it watches the AI's "heartbeat" (uncertainty) to see if the journey was smooth and confident (likely correct) or shaky and confused (likely incorrect).

Technical Summary: Tracing Uncertainty in Language Model "Reasoning"

Problem Statement

Language models (LMs) increasingly generate long sequences of intermediate tokens, termed "reasoning" traces (e.g., Chain-of-Thought or test-time scaling), which improve performance on verifiable tasks like mathematical and logical reasoning. However, the dynamics underlying this generative process remain poorly understood. While the correlation between trace length and performance is established, the mechanisms distinguishing correct from incorrect reasoning remain opaque. Existing approaches to verifying or improving these traces often aggregate information (discarding temporal structure) or rely on semantic content, which has been shown to be unreliable. Furthermore, the definition of "reasoning" in AI remains contested, with evidence suggesting that models may generate semantically invalid traces that still yield correct answers, or valid-looking traces that fail.

This paper addresses the gap in understanding the generative process by treating the "reasoning" trace not as a static semantic object, but as an evolving model state characterized by uncertainty. The authors posit that the dynamics of uncertainty over the course of generation contain predictive signals regarding the correctness of the final answer, independent of the semantic content of the trace.

Methodology

Uncertainty Quantification Framework

The authors apply Uncertainty Quantification (UQ) to model the generative process as a time series. They decompose uncertainty into two primary categories, further subdivided for granular analysis:

Epistemic Uncertainty ( $U_E$ ): Reflects the model's lack of knowledge regarding a prediction based on its training data. It is estimated using the squared $L_2$ norm of the gradient of the predicted probability with respect to model parameters, derived via a first-order delta-method expansion.
Aleatoric Uncertainty: Reflects the inherent randomness of the task. The authors distinguish two sub-types:
- Committal Aleatoric Uncertainty ( $U_C$ ): The Bernoulli variance of the predicted probability of the selected token. It measures how "committed" the model is to a specific token (concentration) versus selecting a token by elimination among many unlikely alternatives.
- Distributional Aleatoric Uncertainty ( $U_D$ ): The expected conditional entropy of the predictive distribution over the full vocabulary. It captures the shape of the entire distribution, not just the mode.

Trace and Answer Channels

For every token position $t$ in a generated trace, the authors compute these three uncertainty types across two channels:

Trace Uncertainty ( $U_{Tr}$ ): The target $y$ is the next token $v_t$ . This characterizes the model's state regarding the immediate generation step.
Answer Uncertainty ( $U_A$ ): The target $y$ is the eventual final answer $\hat{y}$ . This characterizes the model's state regarding the final output, requiring a forward/backward pass conditioned on the trace prefix but applied to the final answer span.

This results in six time series per generation ( $U_{Tr}^D, U_{Tr}^C, U_{Tr}^E, U_{A}^D, U_{A}^C, U_{A}^E$ ).

Feature Engineering and Prediction

To summarize these time series, the authors extract a set of features for each uncertainty trace, termed the uncertainty trace profile:

Static Features: Mean uncertainty in the early ( $\mu_{early}$ ), middle ( $\mu_{mid}$ ), and late ( $\mu_{late}$ ) segments of the trace.
Dynamic Features: The linear slope ( $m$ ) of the uncertainty trajectory and the goodness-of-fit ( $r^2$ ) of this linear trend.

These features are used to train binary classifiers (Logistic Regression and Gradient Boosting) to predict whether a trace yields a correct final answer. The study evaluates five models (Llama-3.1, Llama-3.2, Qwen-2.5, DeepSeek R1, Qwen-3) ranging from standard LMs to "reasoning" models (RMs) trained with reinforcement learning, across two datasets: GSM8K (math) and ProntoQA (logic).

Key Results

Predictive Performance

The uncertainty trace profile is a strong predictor of correctness, significantly outperforming existing baselines:

AUROC Scores: The method achieves AUROC scores up to 0.807 (Qwen-2.5 on GSM8K), improving upon the state-of-the-art Circuit-Based Reasoning Verification (CRV) by 11.5% and Self-Certainty by up to 59%.
Early Detection: Correctness can be predicted with high accuracy using only the first few hundred tokens. On GSM8K, the model reaches an AUROC of 0.801 at the 300th token, suggesting errors can be detected early in the generation process.
Robustness: The predictive power holds across different model architectures and training regimes (standard LMs vs. RMs), whereas previous methods like Self-Certainty showed variable performance depending on the model type.

Qualitative Differences in Uncertainty Dynamics

A detailed analysis of correct versus incorrect traces reveals distinct patterns:

Aleatoric Uncertainty: Incorrect traces generally exhibit higher levels of committal and distributional uncertainty throughout the generation compared to correct traces.
Epistemic Uncertainty: The behavior is counter-intuitive. Incorrect traces often show lower trace-level epistemic uncertainty ( $U_{Tr}^E$ ) in reasoning models, indicating that the steps in an incorrect trace are surprisingly well-supported by training data, even though they lead to a wrong conclusion. Conversely, the final wrong answer often has lower answer-level epistemic uncertainty than the correct answer.
Temporal Dynamics: Correct traces tend to show a steeper and more linear decline in uncertainty (higher $r^2$ ) as generation progresses. Incorrect traces often display flatter, less linear trajectories, or in the case of RMs, a steeper decline that converges on a wrong, well-supported answer.

Significance and Claims

The paper claims to provide a principled lens for studying the generative process of LM "reasoning" grounded in decision-making under uncertainty. Its primary contributions are:

Temporal Characterization: Unlike prior work that aggregates uncertainty or relies on hidden states, this approach treats the trace as an evolving time series, capturing how and where reasoning quality changes.
Diagnostic Signals: The study demonstrates that aleatoric and epistemic uncertainty carry diagnostic signals in opposite directions. A failure mode is characterized by elevated aleatoric uncertainty (indicating difficulty or lack of commitment) coupled with depressed epistemic uncertainty (indicating the wrong path is familiar to the model). Neither measure alone reveals this failure mode.
Practical Utility: The high predictive power of early trace dynamics suggests a practical application for selective generation and test-time scaling, where traces could be resampled or terminated early based on uncertainty profiles before the full generation is complete.
Complementarity to Semantic Analysis: The authors argue that uncertainty-based analysis can identify failures that semantic inspection misses, as evidenced by case studies where the model commits to a wrong interpretation (e.g., ratio vs. difference) with a distinct uncertainty spike that semantic analysis might overlook.

The authors maintain a modest stance, acknowledging that their method relies on approximations of uncertainty (specifically the isotropic approximation for epistemic uncertainty) and that the results may vary between datasets (e.g., GSM8K vs. ProntoQA) due to distribution shifts. They position this work as a proof-of-concept for the informativeness of uncertainty measures across reasoning traces, inviting future work to expand to other tasks and architectures.