💬 NLP

Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models

This paper reveals a significant "Amplification-Lift Gap" in reasoning models where training amplifies deliberative behaviors like self-correction and uncertainty acknowledgment that do not necessarily predict correctness, while failing to sufficiently boost high-value behaviors such as confidence calibration and knowledge alignment.

Original authors: Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig

Published 2026-08-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a magician perform a trick. You see them waving their hands dramatically, muttering to themselves, and checking their cards three times before revealing the final card. To an observer, this looks like deep, careful thinking. But what if the magician was actually just panicking? What if all that "thinking" was just nervous fidgeting, while the real secret to the trick was a simple, quiet sleight of hand that happened instantly?

This is the puzzle scientists are trying to solve with a new generation of artificial intelligence called "thinking models." These are AI systems trained to write out long, step-by-step explanations before giving an answer, much like a student showing their work on a math test. The big question is: Does writing more words and showing more hesitation actually mean the AI is smarter? Or is it just putting on a show? To understand this, we need to know that AI researchers have been trying to teach computers to "reason" by making them generate these long chains of thought. The hope was that if the AI talks through a problem, it would make fewer mistakes. But until now, we didn't have a good way to tell if the AI was actually thinking correctly or just acting like it was.

A team of researchers decided to investigate this by acting like detectives, analyzing over 15,000 reasoning traces from 15 different AI models. They created a special checklist, or "taxonomy," to spot specific behaviors. They looked for things like "self-correction" (admitting a mistake and fixing it), "hypothesis testing" (trying out different ideas), and "uncertainty acknowledgment" (saying "I'm not sure"). They also looked for more subtle signs like "confidence calibration" (knowing when to be bold and when to be cautious) and "knowledge alignment" (using the right facts for the job).

The researchers introduced a clever new way to measure these behaviors called "Behavioral Lift." Think of it like a weather vane. If a specific behavior, like saying "I'm not sure," is a good sign of a correct answer, the weather vane points to "sunny." If that behavior usually appears when the AI gets the answer wrong, the vane points to "stormy." They compared how often these behaviors appeared in the new "thinking" models versus older "instruction" models, and then checked if those behaviors actually predicted a correct answer.

Here is the twist they found: The "thinking" models were great at putting on a show. They were 3 to 7 times more likely to say "I'm not sure" or to try out different ideas and then change their minds. They were full of dramatic hesitation and self-correction. However, the researchers discovered that these loud, dramatic behaviors were not the ones that actually predicted a correct answer. In fact, saying "I'm not sure" was often a sign that the AI was confused and likely to get the answer wrong.

The real heroes of the story were the quiet, steady behaviors that the "thinking" models didn't really get better at. The behaviors that were most strongly linked to getting the right answer were confidence calibration (knowing exactly how sure to be based on the evidence) and knowledge alignment (using the right tools for the job). These behaviors were just as common in the older, simpler models as in the new, fancy "thinking" models. The new models were just louder and more hesitant, but not necessarily more accurate.

The study suggests that these "thinking" models are actually very good at one specific thing: recovery. When a task is hard and requires long, step-by-step work (like solving a complex math problem), the thinking models are better at spotting when they've made a mistake and fixing it. They can recover from errors about 2 to 3 times better than the older models. But on tasks that just require spotting a pattern quickly, the "thinking" models sometimes do worse because they get too distracted by their own long, winding thoughts.

So, the paper concludes that just because an AI sounds like it's thinking hard doesn't mean it is. The "Amplification-Lift Gap" is the name for this disconnect: the behaviors that training makes the AI do more of (like hesitating and self-correcting) are not the same behaviors that make it right. To make AI smarter, we shouldn't just tell it to "think longer" or "be more hesitant." Instead, we need to teach it to be more calibrated—knowing when it is right and when it is wrong—and to use the right facts, rather than just filling up the page with dramatic doubt.

Technical Summary: Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models

Problem Statement

Recent advances in reasoning-oriented post-training have produced "thinking" models (e.g., OpenAI's o1, DeepSeek-R1, Qwen3) that generate extended reasoning traces before answering. While these models often outperform instruction-tuned counterparts, a critical ambiguity remains: does the increased deliberation in these traces reflect genuinely improved reasoning, or merely a change in surface form? Accuracy metrics indicate whether a model is correct but fail to reveal which behaviors led to success or failure. The core problem addressed is the potential disconnect between the behaviors that thinking-oriented training amplifies (makes more frequent) and the behaviors that are actually predictive of correctness.

Methodology

The authors introduce a framework to decouple behavioral prevalence (how often a behavior appears) from behavioral predictive power (how strongly a behavior correlates with correctness).

1. Behavioral Taxonomy

The study defines a cross-modal taxonomy applicable to both Large Language Models (LLMs) and Vision-Language Models (VLMs). It categorizes reasoning traces into:

9 Higher-Order Behaviors: Grouped into control/regulation (planning, goal tracking, hypothesis testing, self-correction), monitoring/judgment (uncertainty acknowledgment, confidence calibration, self-awareness), and epistemic grounding (evidence citation, knowledge alignment).
Failure Modes: 7 modes per modality (e.g., logical failure, post-hoc rationalization, visual hallucination for VLMs).
Grounding & Quality: Labels for visual grounding (for VLMs) and reasoning quality (sound vs. lucky guess).

2. Metrics

Two primary metrics are formalized:

Behavioral Lift: Measures the change in correctness probability when a behavior is present versus absent.
$\text{Lift}(b) = P(\text{correct} \mid b=\text{true}) - P(\text{correct} \mid b=\text{false})$
A positive lift indicates the behavior is associated with higher accuracy; a negative lift suggests the opposite.
Recovery Rate: Measures the probability of reaching a correct answer despite the presence of at least one detected reasoning failure.
$\text{Recovery}(m) = P(\text{correct} \mid \exists f \in F_{all} : f=\text{true})$

3. Experimental Setup

Models: 15 models from 7 families (3B–9B parameters), including matched pairs of thinking and instruction-tuned variants for LLMs and VLMs.
Benchmarks: 6 benchmarks spanning pure reasoning (VisualPuzzles, LogiQA2), mixed (MathVista, MATH-500), and knowledge-heavy (MMMU, MMLU-Pro) tasks.
Data: 15,282 annotated traces.
Annotation: Performed using LLM-as-judge (GPT-4o) with strict definitions, validated against independent judges and manual checks.

Key Contributions

Behavioral Lift Metric: A quantitative measure separating the frequency of a reasoning behavior from its diagnostic value for correctness.
Cross-Modal Taxonomy: A unified framework for analyzing reasoning behaviors across text-only and vision-language modalities, including specific failure modes for each.
Empirical Evidence of the Amplification-Lift Gap: The finding that training objectives which increase the frequency of certain behaviors do not necessarily increase the frequency of the behaviors most predictive of success.
Recovery Mechanism Analysis: Identification of "recovery" (correcting errors after they occur) as a primary driver of performance gains in thinking models, particularly on tasks requiring extended computation.

Results

The Amplification-Lift Gap

The study identifies a significant mismatch between what thinking training amplifies and what predicts success:

Amplified Behaviors: Thinking models show a 3–7× increase in self-correction, hypothesis testing, and uncertainty acknowledgment compared to instruction-tuned models.
High-Lift Behaviors: The behaviors most strongly associated with correctness are confidence calibration (+72% to +80% lift), knowledge alignment, and self-awareness.
The Disconnect:
- Confidence Calibration: One of the strongest positive signals of correctness, yet it is not amplified by thinking training (prevalence remains similar between thinking and instruct models).
- Uncertainty Acknowledgment: Strongly amplified (3–7× increase) but exhibits negative lift (−13.9% to −16.1%), meaning its presence is often associated with incorrect answers.
- Self-Correction: Shows modest positive lift (+12% to +20%), but this is significantly lower than confidence calibration.

Task-Dependent Gains and Recovery

Thinking models do not uniformly outperform instruction-tuned models; their advantage is task-dependent:

Extended Reasoning Tasks: On benchmarks rewarding step-by-step computation (VisualPuzzles, MATH-500, MMLU-Pro), thinking models recover from detected failures at 2.3× to 2.7× the rate of instruction-tuned models.
Pattern Recognition Tasks: On tasks where fast recognition of logical form suffices (LogiQA2), instruction-tuned models outperform thinking models, often by taking efficient shortcuts that thinking models over-complicate.
Failure Modes: Thinking models exhibit fewer failures on knowledge-heavy tasks but may exhibit more failures on pure reasoning tasks, relying on higher recovery rates to maintain accuracy.

Scaling and Robustness

Scale: The Amplification-Lift gap persists across model scales (2B to 32B). While gaps in amplified behaviors (self-correction, uncertainty) remain large, gaps in high-lift behaviors (confidence calibration) narrow or reverse at larger scales, with instruction-tuned models sometimes matching or exceeding thinking models in calibration.
Stability: The ranking of behaviors by Lift is stable across different benchmarks, model families, and response complexity bins.
Probing: Linear probes on hidden states confirm that high-lift behaviors (like confidence calibration) are decodable from internal representations, suggesting they are not merely surface artifacts.

Significance and Claims

The paper claims that amplified deliberation does not track correctness. The authors argue that current reasoning-oriented training often rewards final-answer correctness, which inadvertently encourages surface markers of deliberation (hedging, backtracking, expressing doubt) without necessarily fostering the underlying cognitive mechanisms that ensure reliability.

The central significance lies in the distinction between process and surface form:

Process Supervision: The authors suggest that future training objectives should explicitly reward behaviors associated with correctness (calibrated confidence, grounded evidence, appropriate domain framing) rather than simply encouraging longer traces or visible hesitation.
Evaluation: Evaluations of reasoning models should move beyond accuracy to report how models succeed—specifically, whether gains come from preventing failures, recovering from them, or improving calibration.
Limitations: The authors acknowledge that their analysis is limited to visible traces, which may not fully reflect internal computation, and that Behavioral Lift is a descriptive metric, not a causal proof.

In summary, the paper demonstrates that making models "think longer" does not automatically make them "think better." True reasoning improvement requires aligning training objectives with the specific behaviors (like confidence calibration and knowledge alignment) that are empirically proven to predict correctness, rather than the behaviors that merely look like deliberation.