💬 NLP

Intention Collapse: Intention-Level Metrics for Reasoning in Language Models

This paper introduces "intention collapse" as a framework for analyzing language model reasoning by measuring pre-generation internal states via entropy, effective dimensionality, and recoverability metrics, revealing that chain-of-thought prompting induces heterogeneous internal uncertainty regimes and that latent knowledge can remain recoverable even when final behavioral accuracy degrades, particularly in multiple-choice formats.

Original authors: Patricio Vera

Published 2026-01-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Patricio Vera

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are about to give a speech. Before you open your mouth, your brain is buzzing with a chaotic storm of memories, half-formed arguments, emotions, and potential word choices. It’s a rich, messy, high-dimensional cloud of "intent."

But when you finally speak, you don’t release the whole cloud. You pick one specific sentence. You collapse that vast internal storm into a single, linear string of words.

This paper argues that Large Language Models (LLMs) do the exact same thing. The authors call this process "Intention Collapse."

Here is a breakdown of what they found, using simple analogies.

1. The Core Idea: The "Pre-Speech" Moment

Most people look at an AI’s answer and judge it by the final words. But this paper says we should look at what’s happening inside the AI’s "brain" right before it picks the very first word.

Think of it like a photographer taking a picture.

The Scene: The complex, 3D world (the AI’s internal state).
The Photo: The flat, 2D image (the text output).
The Collapse: The moment the shutter clicks. Information is lost. The 3D depth is flattened into 2D pixels.

The authors wanted to measure the "Scene" (the internal state) to see if we could predict whether the "Photo" (the answer) would be good or bad, and how different prompting techniques change that scene.

2. The Three "Health Checks" for the AI’s Mind

To measure this internal state, the authors created three simple metrics. Think of these as vital signs for the AI’s intention:

Intention Entropy (The "Confusion" Meter):
- Analogy: Imagine you’re at a crossroads. If you know exactly which way to go, your "entropy" is low. If you have no idea and are equally likely to go left, right, or straight, your entropy is high.
- In the AI: This measures how spread out the AI’s probability is for the next word. Low entropy means it’s very sure; high entropy means it’s unsure or considering many options.
Effective Dimensionality (The "Richness" Meter):
- Analogy: Imagine a painting. Is it a simple stick figure (low dimensionality), or is it a complex, detailed landscape with many layers of meaning (high dimensionality)?
- In the AI: This measures how much "space" the AI’s internal thoughts are using. Are its thoughts simple and narrow, or complex and multi-faceted?
Recoverability (The "Hidden Knowledge" Meter):
- Analogy: Imagine a student who knows the answer to a math problem but writes down the wrong number because they got nervous. If you could look inside their head, you’d see they knew it.
- In the AI: The researchers trained a simple "probe" (a mini-detector) to look at the AI’s internal state and guess if the final answer would be correct. If the probe can guess correctly, it means the AI had the right information internally, even if it failed to write it down.

3. The Experiment: Testing "Chain-of-Thought"

"Chain-of-Thought" (CoT) is a popular trick where you tell an AI, "Think step-by-step before answering." Everyone assumes this makes the AI smarter. The researchers tested this on three different AI models (Mistral, LLaMA, Qwen) across three types of tasks (Math, Abstract Reasoning, and Math Word Problems).

They compared three conditions:

Baseline: Just give the answer.
CoT: Think step-by-step.
Babble: Write a long, rambling stream of consciousness (to see if just writing more is what helps, or if the thinking helps).

4. The Surprising Findings

Finding 1: "Thinking Step-by-Step" isn’t always better.

Math (Free Response): CoT worked great. Accuracy went up significantly.
Abstract Reasoning (Multiple Choice): CoT actually made things worse. The AI performed worse when forced to "think out loud" than when it just gave a direct answer.
The Takeaway: Forcing an AI to verbalize its reasoning can sometimes confuse it, especially when it has to pick from a list of options (A, B, C, D).

Finding 2: Different AIs "Think" Differently.
When using CoT, the internal "Confusion" (Entropy) changed differently for different models:

Mistral: Became more certain (lower entropy). It narrowed its focus.
LLaMA: Became less certain (higher entropy). It opened up its possibilities.
The Takeaway: There is no single way "reasoning" looks inside an AI. One model’s "thinking" looks like another model’s "confusion."

Finding 3: The AI Might Know the Answer, But Fail to Say It.
This is the most interesting part. In some cases (like Qwen on the Abstract Reasoning task), the AI’s internal state showed high Recoverability. The "probe" could easily see that the AI knew the right answer. However, the AI’s final written answer was wrong.

Analogy: It’s like a driver who knows the road perfectly (high internal knowledge) but accidentally turns the wrong way at the last second (failed collapse).
The Takeaway: The problem isn’t always that the AI lacks knowledge. Sometimes, the problem is the "collapse"—the final step of converting that knowledge into a specific format (like a multiple-choice letter) is where the error happens.

Finding 4: Babbling isn’t Thinking.
The "Babble" control (just writing a lot of nonsense) did not produce the same internal changes as CoT. This proves that CoT isn’t just about making the output longer; it actually changes the internal structure of the AI’s thoughts.

Summary

The paper suggests we stop looking at AI answers as just "right" or "wrong." Instead, we should look at the Intention Collapse—the moment the AI decides to speak.

Sometimes, asking an AI to "think out loud" helps it focus (lower entropy).
Sometimes, it makes it overthink and get confused (higher entropy).
Sometimes, the AI knows the truth internally but fails to express it correctly in the required format.

By measuring the "pre-speech" state, we can better understand why an AI fails, rather than just knowing that it failed.

Technical Summary: Intention Collapse: Intention-Level Metrics for Reasoning in Language Models

Problem Statement
The paper addresses the structural disconnect between the rich, high-dimensional internal states of Large Language Models (LLMs) and the discrete, linear token sequences they produce. The authors formalize this many-to-one mapping as intention collapse: the irreversible projection from an internal intention space $I$ (the pre-generation latent state) to an external language space $L$ . Existing reasoning-enhancing techniques (e.g., Chain-of-Thought, self-refinement) are often evaluated solely on final accuracy, obscuring how these methods reshape the internal state prior to generation. The core problem is the lack of a unified framework to measure the informational and geometric properties of the pre-collapse state $I$ and to determine when reshaping $I$ helps or harms performance, particularly across different response formats.

Methodology
The authors operationalize intention collapse by defining the intention state $I$ as the collection of hidden representations at the final prompt position, immediately before the first output token is selected. They propose three model-agnostic, computationally inexpensive metrics computed on $I$ :

Intention Entropy ( $H_{int}(I)$ ): The Shannon entropy of the next-token distribution at the pre-collapse moment, measuring the concentration or diffusion of the model’s immediate continuation.
Effective Dimensionality ( $d_{eff}(I)$ ): The participation ratio of PCA eigenvalues over pre-collapse hidden states, capturing the geometric richness of the intention representation.
Latent Knowledge Recoverability ($Recov(I; Z)$): The AUROC of linear probes trained on $I$ to predict task outcomes (e.g., answer correctness), quantifying information present pre-collapse that may fail to be expressed in the final output.

Experimental Design
The framework is evaluated in a $3 \times 3$ empirical study:

Models: Mistral-7B-Instruct, LLaMA-3.1-8B-Instruct, and Qwen-2.5-7B-Instruct.
Benchmarks: GSM8K (free-response math), ARC-Challenge (multiple-choice abstract reasoning), and AQUA-RAT (multiple-choice math).
Inference Regimes: Baseline (direct answer), Chain-of-Thought (CoT), and Babble (a verbosity control generating unrelated stream-of-consciousness text).
Protocol: $n=200$ items per cell; greedy decoding ( $T=0$ ) to isolate prompt-conditioning effects; extraction of $I$ strictly before the first token generation.

Key Results

Format-Dependent Efficacy of CoT: CoT is not universally beneficial. It yields large accuracy gains on GSM8K (free-response) across all models but consistently degrades performance on ARC-Challenge (multiple-choice) and produces mixed results on AQUA-RAT. This suggests that changing the pre-collapse state does not guarantee a reliable final discrete commitment under all response formats.
Heterogeneous Entropy Regimes: CoT induces distinct entropy shifts relative to baseline ( $\Delta H = H_{int}(CoT) - H_{int}(Base)$ ). Mistral exhibits lower pre-collapse entropy under CoT ( $\Delta H < 0$ ), whereas LLaMA exhibits higher pre-collapse entropy ( $\Delta H > 0$ ). This highlights significant heterogeneity in how different model families manage internal uncertainty during reasoning.
Dissociation of Signal and Accuracy: Recoverability ($Recov$) can dissociate from behavioral accuracy. In some settings (e.g., Qwen on ARC-Challenge), probe AUROC remains significantly above chance despite degraded CoT accuracy. This indicates that informative internal signals may exist in $I$ but fail to be reliably converted into correct discrete decisions, particularly in multiple-choice formats.
Verbosity Control: The Babble control confirms that CoT effects are not merely due to output length; Babble does not replicate the metric-performance relationships observed under CoT.

Significance and Contributions
The paper claims that intention collapse provides a unifying lens for reasoning about LLM behavior, reframing diverse inference-time methods as interventions on a common object ( $I$ ). Key contributions include:

Conceptual Bridge: Connecting literatures on latent knowledge elicitation, pre-generation adequacy, and uncertainty estimation through the specific boundary of the pre-collapse state.
Diagnostic Utility: Demonstrating that internal separability (recoverability) and final commitment reliability are distinct phenomena. The framework explains why models may "know" the answer (high $Recov$) but fail to output it correctly (low accuracy) due to format-induced commitment failures.
Research Agenda: Proposing concrete directions for adaptive inference, such as state-dependent decoding policies (adjusting sampling based on $H_{int}$ ) and early-exit strategies (using $Recov$ to avoid unnecessary computation), thereby turning intention collapse into a practical tool for optimizing test-time compute.

The authors maintain a modest scope, noting that their findings are correlational and limited to text-only benchmarks and specific model families. They emphasize that intention collapse is a structural analogy rather than a claim of conscious intention in LLMs.

1. The Core Idea: The "Pre-Speech" Moment

2. The Three "Health Checks" for the AI’s Mind

3. The Experiment: Testing "Chain-of-Thought"

4. The Surprising Findings

Summary

More like this