💬 NLP

How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures

This paper identifies and characterizes two distinct token-level reasoning failure modes in language models—committed failure, marked by early path-locking, and persistent uncertainty, characterized by accumulating doubt—demonstrating that these signatures are reproducible across diverse configurations and can guide the optimization of failure detection strategies like self-consistency.

Original authors: Tanvi Thoria, Kiana Jafari, Marc R. Schlichting, Mykel J. Kochenderfer

Published 2026-06-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tanvi Thoria, Kiana Jafari, Marc R. Schlichting, Mykel J. Kochenderfer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a detective solve a mystery. Sometimes, the detective makes a mistake early on, locks onto the wrong suspect, and then spends the rest of the movie confidently arguing why that suspect is guilty, even though they are innocent. Other times, the detective is genuinely confused the whole time, flipping through clues, unsure of the answer until the very last second.

This paper is about figuring out which kind of mistake a Large Language Model (LLM) is making just by watching how it "thinks" (its chain of thought) as it writes its answer. The researchers found that models don't just fail in one way; they fail in two distinct patterns, and recognizing the pattern changes how we should try to catch the error.

Here is a breakdown of their findings using simple analogies:

1. The Two Types of Mistakes

The researchers discovered that when a model gets an answer wrong, it usually happens in one of two ways:

Type A: The "Early Commitment" (The Stubborn Detective)

What happens: The model picks a wrong path very early in its reasoning. Once it picks that path, it gets "locked in." It stops exploring other possibilities and just keeps writing more sentences to support that wrong idea.
The Signature: If you look at the model's "uncertainty" (how unsure it feels) as it writes, you see a spike early on, and then it drops. The model becomes too confident too quickly.
The Lesson: You don't need to wait for the whole story to finish to know it's wrong. In fact, reading the end of the story might actually confuse you because the model is so confident in its wrong answer that it looks like a good one. The best time to spot this error is right after the model makes its first big mistake.

Type B: "Persistent Uncertainty" (The Confused Detective)

What happens: The model never really picks a side. It wanders through the reasoning, unsure of the answer the entire time. It keeps changing its mind or hesitating until the very last word.
The Signature: The model's uncertainty stays high or slowly builds up from start to finish. It never locks onto a single path.
The Lesson: You must read the whole story to know if it failed. If you stop halfway through, you might think it's just being careful. The error only becomes clear when you see the whole messy trail of thought.

2. How They Found This (The "Uncertainty Meter")

The researchers didn't need to look inside the model's brain (which is impossible for many popular AI models). Instead, they looked at the log probabilities—a technical way of saying "how sure the model was about every single word it typed."

They treated the model's reasoning like a movie. They watched the "uncertainty meter" tick up and down as the model typed word by word.

For Type A, the meter showed a clear peak early on, then flattened out.
For Type B, the meter kept climbing or staying high until the end.

They tested this on 23 different combinations of AI models and tasks (like math, coding, and science). In 20 out of 23 cases, their theory held up perfectly. They could tell the difference between a "stubborn" mistake and a "confused" mistake just by looking at these patterns.

3. Why This Matters: The "Second Opinion" Strategy

The paper also looked at a common trick called Self-Consistency. This is when you ask the AI the same question 10 times and take the answer that appears most often.

For the "Stubborn" Model (Type A): Asking the same question 10 times is useless. If the model is stubborn, it will give you the same wrong answer 10 times in a row. The "majority vote" will just confirm the wrong answer. In this case, looking at the uncertainty of a single answer is actually better than asking it multiple times.
For the "Confused" Model (Type B): Asking the question 10 times is very helpful. Since the model is genuinely unsure, it will give you different answers each time. The fact that it can't agree with itself is a huge red flag.

The Bottom Line

The paper argues that we shouldn't use a "one-size-fits-all" approach to catching AI errors.

If the AI locks onto a wrong idea early, we should catch it quickly by looking at its early confidence levels, and we shouldn't bother asking it for a second opinion (because it will just repeat the mistake).
If the AI is genuinely confused, we should let it finish its thought process and then ask it for multiple opinions to see if it can agree with itself.

By understanding how the AI is failing, we can choose the right tool to catch the mistake, saving time and improving reliability.

Technical Summary: How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures

Problem Statement

Detecting when Large Language Models (LLMs) will fail on complex reasoning tasks remains a critical challenge for deployment reliability. Existing failure detection methods, such as self-consistency (sampling multiple chains of thought) and uncertainty quantification, typically treat failure as a binary prediction task. These approaches assume a singular process of failure emergence, which limits the ability to adapt detection strategies. Specifically, self-consistency fails when a model "commits" to an incorrect reasoning path early, reproducing the same wrong answer consistently across samples. Conversely, for models exhibiting genuine uncertainty throughout a trace, single-sample uncertainty signals may be insufficient. The paper argues that reasoning failures develop through distinct processes that require different detection strategies, yet current methods apply a uniform approach regardless of the underlying failure mode.

Methodology

The authors propose a framework that characterizes reasoning failures using token-level uncertainty signals derived from chain-of-thought (CoT) traces. The framework requires only log probabilities from a single model completion, making it applicable to closed-API frontier models (e.g., GPT-4o, Gemini) where internal weights are inaccessible.

Core Concepts

Failure Modes: The framework identifies two qualitatively distinct failure modes:
- Committed Failure: The model locks onto an incorrect reasoning path early in the trace. Uncertainty signals are most predictive over an early prefix of the trace; adding subsequent tokens dilutes the signal.
- Persistent Uncertainty: The model never commits to a single path. Uncertainty accumulates monotonically throughout the trace, and the full trace is required to distinguish failing from successful completions.
Commitment Point ( $T^*$ ): Defined as the token position in a reasoning trace where uncertainty signals are maximally predictive of failure. Beyond this point, subsequent tokens represent downstream noise rather than signal regarding the eventual outcome.
Uncertainty Features: The authors compute five token-level signals over prefix windows ( $W_T$ ) of the reasoning trace:
- Entropy: Spread of the top- $K$ distribution.
- Margin: Difference between the top two probabilities.
- NLL (Negative Log-Likelihood): Confidence in the top token.
- Nucleus: Tokens needed to capture a 0.9 probability threshold.
- Near-Tie: Fraction of top- $K$ tokens within 90% of the top probability.
- Aggregation: Both mean and maximum features are computed for each signal across the window.
Diagnostic Metric (PR-AUC): The framework uses Precision-Recall Area Under the Curve (PR-AUC) to quantify failure detection at varying prefix window sizes.
- Committed Signature: An inverted-U shape in the PR-AUC curve. Predictive power peaks at $T^*$ and declines as the window expands.
- Persistent Signature: A monotonically rising PR-AUC curve where the full trace always outperforms any early window.
Pre-final Analysis: To control for the confound of trace length (where failing models often write longer traces), the framework strips all tokens occurring after the final answer marker before computing uncertainty signals.

Experimental Setup

The framework was evaluated across 23 model-dataset configurations involving:

Models: Five families (Qwen, Llama, GPT-OSS, Gemma, GPT-4o/Gemini) ranging from 2B to 122B parameters, including open-source and frontier models.
Datasets: Four reasoning domains (GSM8K, MATH-500, GPQA Diamond, LiveCodeBench).
Constraints: Experiments were limited to configurations with failure rates between 15% and 60% and sufficient failure counts for statistical reliability. Frontier models were constrained to the top-20 log probabilities.

Key Results

Reproducibility of Failure Modes: The framework's falsifiable predictions held in 20 of 23 cases, significantly above chance.
- Committed Failure: Observed in 14 configurations (e.g., Gemma4-31B on LiveCodeBench, GPT-OSS-20B on MATH-500). These exhibited the predicted inverted-U PR-AUC curve with a distinct commitment point.
- Persistent Uncertainty: Observed in 9 configurations (e.g., Llama3.1-8B on MATH-500, Qwen3.5-122B on GPQA). These showed monotonically increasing PR-AUC curves where the full trace was strictly more informative than any prefix.
Statistical Validation: Meta-analytic pooling confirmed the bidirectional predictions:
- Sign test on committed configurations: 14/14 consistent ( $p = 6.1 \times 10^{-5}$ ).
- Joint sign test across all configurations: 20/23 consistent ( $p = 2.4 \times 10^{-4}$ ).
- Pooled effect size ( $\hat{\Delta}$ ) for committed cases was $+0.013$ (95% CI $[+0.005, +0.020]$ ).
Implications for Self-Consistency:
- Committed Regime: Self-consistency is ineffective because the model reproduces the same wrong answer. Here, single-completion uncertainty signals (specifically early-window features) provide superior detection. The authors demonstrate that for committed models, self-consistency can be skipped for the top 30% of confident inputs without sacrificing failure recall.
- Persistent Regime: Self-consistency remains effective as failure manifests as cross-completion disagreement.
- Complementarity: Combining single-completion uncertainty with self-consistency agreement rates consistently improved PR-AUC across all tested regimes, capturing within-completion reasoning quality that agreement rates alone miss.

Significance and Claims

The paper claims to offer a foundational framework for understanding when and how LLM reasoning failures become detectable. By characterizing failures as either "committed" or "persistent," the authors argue that detection strategies should be adapted to the specific failure mode rather than applied uniformly.

Key contributions include:

Token-Level Signatures: Demonstrating that token-level uncertainty signals can distinguish between two distinct failure processes using only external log probabilities.
Identification of the Commitment Point: Defining and empirically validating a specific point in a reasoning trace where the model locks onto a path, marking the transition from signal to noise.
Practical Deployment Guidance: Providing a mechanism to triage when self-consistency is necessary versus when it can be selectively skipped, and showing how uncertainty features complement existing verification methods.

The authors maintain modesty regarding their claims, noting that the framework operates on visible CoT traces (without asserting these traces perfectly reflect internal computation) and is constrained by API limitations on log probability access. They emphasize that the framework requires moderate failure rates to function reliably and that the commitment point is identified at the granularity of window sizes rather than exact token positions.