🤖 AI

How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

This paper investigates the internal mechanisms of Chain-of-Thought prompting by tracing information flow across decoding, projection, and activation phases, revealing that it functions as a decoding space pruner guided by answer templates and modulates neuron engagement differently depending on whether the task is open- or closed-domain.

Original authors: Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng, Mengda Yu

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng, Mengda Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but sometimes scattered student (the AI model). When you ask them a hard question, they might jump straight to an answer and get it wrong because they didn't think it through. But if you ask them to "show their work" step-by-step (Chain-of-Thought, or CoT), they suddenly become much smarter.

This paper is like a detective story where the authors peek inside the student's brain to see how that "show your work" trick actually changes their thinking process. They didn't just look at the final grade; they traced the information flow from the moment the question was read to the moment the answer was written.

Here is what they found, explained through simple analogies:

1. The "Template" Effect (Decoding)

The Analogy: Imagine the student is trying to write a story. Without a guide, they might wander off-topic. But if you give them a specific outline (a template), they stick to the plot.
The Finding: The authors found that CoT works partly because the model starts mimicking the structure of the prompt. It's not necessarily learning deep logic; it's learning to follow a pattern.

If the prompt says, "First, do this. Then, do that," the model copies those connecting words (like "therefore," "then," "so").
The Key Insight: The more closely the model's answer follows this "template structure," the better its score. It's like the model is using the template as a guardrail to keep its thoughts from wandering off a cliff.

2. The "Flashlight" Effect (Projection)

The Analogy: Imagine the model is standing in a dark room full of thousands of doors (possible words).

Without CoT: The model shines a dim, wide flashlight. It sees many doors as possibilities, so it's unsure which one to pick. This is "high uncertainty."
With CoT: The model turns on a bright, focused laser pointer. The CoT steps act like a guide, narrowing the beam so only a few specific doors look open.
The Finding: When the model uses CoT, it becomes much more confident. The "probability" of the correct next word becomes much higher, and the "noise" (confusion about other words) drops significantly. The model isn't guessing as much; it's committing to a path.

3. The "Light Switch" Effect (Activation)

The Analogy: Think of the model's brain as a massive city with millions of light switches (neurons).

Open-Domain Tasks (The "Exploration" Mode): For open-ended questions (like "Solve this math problem where the answer could be anything"), CoT acts like a Pruner. It turns off many lights, focusing the city's energy on just a few specific streets. It makes the brain more efficient and focused.
Closed-Domain Tasks (The "Search" Mode): For questions with a fixed set of answers (like a multiple-choice quiz), CoT acts like an Amplifier. It actually turns on more lights. It seems to make the model work harder to compare all the possible options (A, B, C, D, E) to find the right one.

The Big Picture

The paper concludes that Chain-of-Thought isn't magic. It works by doing three specific things:

Narrowing the path: It forces the model to follow a structural template, preventing it from getting lost.
Boosting confidence: It reduces the model's confusion, making it sure of its next step.
Adjusting the brain: It changes how many "neurons" (brain cells) light up, depending on whether the task is open-ended (focus less) or multiple-choice (focus more).

In short, CoT is like giving the AI a map and a flashlight. It doesn't necessarily make the AI "smarter" in a human sense, but it organizes its internal machinery so it can navigate the problem much more effectively.

Technical Summary: How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

Problem Statement

While Chain-of-Thought (CoT) prompting significantly enhances the reasoning capabilities of Large Language Models (LLMs) across arithmetic, commonsense, and symbolic tasks, its internal operational mechanisms remain poorly understood. Prior hypotheses suggest CoT reduces task complexity, imitates answer templates, or relies on prompt features unrelated to logic, yet these claims lack direct experimental evidence linking internal model states to observed performance gains. This work addresses the "black box" nature of CoT by investigating how it alters the flow of information within the model's computational architecture.

Methodology

The authors employ a mechanistic interpretability approach, reverse-engineering the model's internal computations by tracing information flow backward from the output generation to the internal activation states. The study utilizes:

Models: Six pretrained models ranging from 3B to 70B parameters (LLaMA3.1, LLaMA3.2, Gemma2).
Datasets: Nine datasets spanning three reasoning types: arithmetic (GSM8K, SVAMP, AQuA), commonsense (Bamboogle, StrategyQA, Date, Sports), and symbolic (Coin Flip, Last Letters Concatenation).
Analysis Framework: The investigation is structured into three phases:
1. Decoding: Analyzing generated text for keyword imitation and adherence to abstract reasoning structures.
2. Projection: Examining probability distributions over the vocabulary to measure prediction uncertainty and confidence.
3. Activation: Quantifying the engagement of Feed-Forward Network (FFN) neurons to understand how CoT modulates internal processing.

Key metrics include "Imitation Count" (adherence to a defined reasoning structure), kernel density estimates of token probabilities, entropy of logits, and the count of activated neurons per layer.

Key Contributions and Results

1. CoT as a Decoding Space Pruner via Structural Adherence

The study proposes that CoT functions by constraining the decoding space through the imitation of specific reasoning structures.

Keyword Imitation: Models selectively imitate structural keywords (e.g., "time," "action") from the prompt while extracting content-specific keywords (e.g., "numbers") from the input question.
Structure-Performance Correlation: A strong positive correlation ( $r = 0.75$ to $0.92$) was found between the model's adherence to a formalized reasoning structure (defined as $E_p \xrightarrow{O} E_g + S_l$ ) and task accuracy. High adherence to this structure explains a significant portion of performance variance ( $R^2$ up to 0.84), suggesting that the structure of the reasoning, rather than the logical validity of individual steps, is the primary driver of CoT's efficacy.

2. Reduction of Predictive Uncertainty in Projection

Analysis of the projection phase reveals that CoT fundamentally alters the model's confidence.

Concentrated Distributions: CoT-generated sequences exhibit higher and more concentrated probability distributions compared to standard prompts.
Lower Entropy: CoT prompts result in significantly lower entropy in token predictions, particularly at the decision point (e.g., "the answer is..."). This indicates that CoT narrows the set of plausible next tokens, reducing ambiguity and increasing model conviction.

3. Task-Dependent Modulation of Neuron Engagement

A novel finding is the non-uniform effect of CoT on FFN neuron activation, which varies by task type:

Overall Reduction: In general, CoT leads to a lower total count of activated neurons compared to standard prompts, suggesting a more focused processing regimen.
Layer-Specific Differences: The most pronounced changes occur in the final third of the model's layers.
Open vs. Closed Domain Divergence:
- Open-Domain Tasks (e.g., GSM8K, Bamboogle): CoT reduces neuron activation in later layers. The authors hypothesize this acts as a "pruner," allowing the model to selectively engage relevant features while navigating a vast solution space.
- Closed-Domain Tasks (e.g., Coin Flip, AQuA, Sports): CoT increases neuron activation in later layers. This is interpreted as an "amplifier" effect, where the model thoroughly evaluates features associated with limited, predefined options.

Significance and Claims

The paper claims to provide a novel mechanistic interpretability framework for understanding CoT. By tracing information flow from decoding to activation, the authors offer critical insights that move beyond surface-level observations:

Mechanism of Action: CoT's success is attributed to its ability to guide models toward generating structurally coherent reasoning steps, which constrains the decoding space and reduces predictive uncertainty.
Dynamic Modulation: CoT does not operate uniformly; it dynamically tailors neural engagement based on task demands (pruning for open domains, amplifying for closed domains).
Practical Implications: These findings offer a basis for designing more efficient and robust prompts by leveraging structural adherence and understanding how different task types interact with internal model states.

The authors maintain a modest stance, acknowledging that while their findings provide strong empirical evidence and plausible mechanistic hypotheses, the fundamental opacity of LLMs means that definitive causal links between specific internal processes and external behaviors remain a frontier for future research. They emphasize that their work contributes to the fragmented but growing field of mechanistic interpretability.