Technical Summary: What Makes Chain-of-Thought Work at Probe Time?

Problem Statement

While Chain-of-Thought (CoT) prompting is widely recognized for improving language model accuracy on reasoning tasks, the specific textual properties of a rationale that drive this improvement remain poorly understood. Prior research has predominantly focused on generation-time behavior, investigating how CoT affects the model's internal computation or how variants like self-consistency influence reasoning quality. However, the probe-time behavior—the process by which a model reads a fixed, injected rationale and uses it to answer a question—has received less attention.

This paper addresses the question: Given a fixed rationale in the context, what specific properties of that text change the model's answer? The authors challenge the prevailing assumption that the benefit of CoT arises primarily from the model reconstructing a global, sentence-level logical derivation. Instead, they hypothesize that the gains may rely on surprisingly local properties of the text.

Methodology

The authors employ a controlled generator–probe framework to isolate the effects of rationale text structure:

1. Generator: A strong model generates a complete CoT rationale for a given question.
2. Injection & Perturbation: The generated rationale is injected into the context of a probe model. Crucially, explicit answer-declaration sentences are automatically removed to prevent trivial answer leakage.
3. N-gram Perturbation Protocol: The core experimental manipulation involves partitioning the rationale token sequence into contiguous windows of size $n$ and randomly shuffling these windows while preserving the internal order of tokens within each window.
   • $n=1$ (Word Shuffle): Destroys all compositional structure, leaving only a global bag of words.
   • $n=2, 3, \dots$: Preserves increasingly larger local co-occurrence structures.
   • Sentence Shuffle (SS): Randomly reorders full sentences while keeping them intact.
4. Gap-Recovery Statistic ($GR(n)$): The authors define a metric to quantify how much of the performance gap between a word-bag baseline ($WS$) and full CoT is recovered at a given window size $n$:
   $$GR(n) = \frac{Acc(n) - Acc(WS)}{Acc(CoT) - Acc(WS)}$$
   A critical window size $n^*$ is defined as the smallest $n$ where $GR(n) > 0.5$.

The study spans multiple datasets (MMLU-Pro, LogiQA, MedQA, GSM8K, MATH500) and various model configurations (Qwen2.5, Qwen3.5, GPT-4o-mini) across different parameter scales (0.8B to 35B).

Key Contributions

1. Probe-Time Perspective: The paper introduces a systematic framework for studying how perturbations to a fixed rationale affect downstream answer prediction, distinct from generation-time analysis.
2. Identification of Two Gain Sources: The authors identify two complementary mechanisms driving CoT performance at probe time:
   • Lexical Activation: The mere presence of domain-relevant tokens provides a significant boost over a no-rationale baseline, even when all structure is destroyed.
   • Local Co-occurrence Activation (LCA): The majority of the remaining gain (the difference between a word bag and full CoT) is recovered by preserving only very short local token windows ($n^* \approx 2\text{--}3$).
3. Ruling Out Alternatives: Through targeted control experiments, the paper demonstrates that the observed gains are not primarily driven by:
   • Copying explicit answer declarations or answer values.
   • The presence of fully grammatical, coherent sentences.
   • Generic topical relevance (tested via Wikipedia retrieval).
   • A general model robustness to shuffling (tested via question-stem shuffling).

Key Results

• Sentence Order is Secondary: Shuffling the order of sentences (Sentence Shuffle) results in negligible accuracy degradation compared to full CoT (e.g., 83.1% vs. 83.9% on MMLU-Pro), suggesting sentence-level logical ordering is not the primary driver of probe-time gains.
• Lexical Activation is Strong: A globally word-shuffled rationale (Word Shuffle) substantially outperforms the input-only baseline (e.g., 66.5% vs. 50.9%), confirming that domain vocabulary alone activates relevant knowledge.
• Short Windows Recover Most Gain: The transition from a word bag ($n=1$) to preserving just 2–3 token windows is steep. On MMLU-Pro, preserving $n=2$ windows recovers approximately 57.5% of the remaining gap between the word bag and full CoT. The critical window size $n^*$ remains consistently small (2–3) across different model families and datasets.
• Generalization: The qualitative pattern holds across:
  • Model Families: Qwen2.5, Qwen3.5, and GPT-4o-mini.
  • Parameter Scales: From 0.8B to 35B parameters.
  • Task Types: Both multiple-choice (MMLU, LogiQA) and open-ended generation (GSM8K, MATH500).
• Specificity to CoT: The low-$n$ recovery effect is specific to CoT-style rationale text. Injecting topic-matched Wikipedia passages or shuffling the question stem requires much larger windows to recover performance, indicating the effect is not a generic property of language processing.

Significance and Claims

The paper claims to support a Local Co-occurrence Activation (LCA) account of probe-time CoT. The authors argue that the observed performance gains arise primarily from lexical activation (the presence of relevant tokens) and short-range token co-occurrence (local statistical associations), rather than from the model performing a faithful, sentence-level logical derivation or reconstructing a global reasoning chain.

The significance of this finding lies in its challenge to the intuition that CoT works because it forces the model to "think step-by-step" in a logically coherent manner. Instead, the results suggest that at probe time, the model may be leveraging local statistical cues within the rationale text to activate the correct answer, even when the text is fragmented and lacks global logical coherence. This reframes the utility of CoT from a mechanism of explicit reasoning execution to one of local contextual activation.