Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought

Technical Summary: Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought

1. Problem Statement

Large Language Models (LLMs) rely heavily on Verbalized Chain-of-Thought (CoT)—generating explicit, natural language rationales—to solve complex reasoning tasks. While effective, this approach suffers from three major drawbacks:

• Inference Cost: Long rationales significantly increase latency and token consumption.
• Training Inefficiency: Long traces bloat the computational cost during Reinforcement Learning (RL).
• Unfaithfulness: Research suggests that verbalized rationales can be unfaithful to the model's actual internal reasoning process.

Existing attempts to compress CoT into continuous latent spaces (e.g., Coconut) or use "filler" tokens (e.g., Pause Tokens) often struggle with a trade-off between efficiency and expressivity, or suffer from "cold-start" problems where new tokens lack meaningful semantic grounding.

2. Methodology: Abstract Chain-of-Thought (Abstract-CoT)

The authors propose Abstract-CoT, a mechanism that replaces natural language rationales with a short sequence of discrete tokens from a reserved, non-verbal abstract vocabulary ($V_{abs}$). This acts as a "latent scratchpad" that the model uses to "think" before generating a response.

The training is divided into two distinct stages:

Stage I: Policy Iteration Warm-up (Addressing the Cold-Start Problem)
Since abstract tokens are randomly initialized, they initially carry no meaning. To bridge this, the authors use a two-step iterative loop:

1. Bottlenecked SFT: The model is trained on sequences containing a prompt ($x$), a gold verbal CoT ($c$), abstract tokens ($\tilde{z}$), and an answer ($y$). A block-structured attention mask is applied so that the answer can only attend to the prompt and the abstract tokens, but not the verbal CoT. This forces the abstract tokens to act as an information bottleneck, compressing the reasoning from the verbal CoT into the abstract representations.
2. Self-Distillation: The model is then trained to generate these abstract tokens directly from the prompt alone (discarding the verbal CoT), effectively learning to internalize the reasoning process.

Stage II: Warm-started Reinforcement Learning
After the embeddings are "warmed up," the model undergoes RL using Group Relative Policy Optimization (GRPO).

• Constrained Decoding: During rollout, the model is restricted to generating only tokens from the abstract vocabulary.
• Generative Reward Model: A large model (e.g., GPT-OSS-20B) scores the final response. The RL objective optimizes both the abstract sequence and the final answer, rewarding trajectories that lead to high-quality, accurate responses.

3. Key Contributions

• New Reasoning Paradigm: Introduced a discrete latent reasoning mechanism that operates entirely within the token space, avoiding the complexities of continuous latent vectors.
• Effective Training Recipe: Developed a policy iteration warm-up (Bottlenecked SFT $\rightarrow$ Self-Distillation) that successfully solves the cold-start problem for new vocabularies.
• Discovery of an "Abstract Language": Observed that the model develops a power-law distribution (similar to Zipf's Law) over the abstract tokens, suggesting that the model learns a structured, compositional "reasoning language" where specific tokens represent recurring concepts.
• Robustness and Scalability: Demonstrated that the method generalizes across different model families (Qwen, Granite) and scales with model size (up to 32B parameters).

4. Results

The experimental results across MATH-500, AlpacaEval, and HotpotQA show:

• Extreme Token Efficiency: Abstract-CoT achieves up to 11.6× fewer reasoning tokens compared to verbal CoT.
• Performance Parity/Superiority: Despite the massive reduction in tokens, Abstract-CoT matches or outperforms verbalized CoT (SFT+RL) in accuracy and win-rates.
• Graceful Degradation: Truncation analysis showed that Abstract-CoT is more robust to short reasoning budgets than verbal CoT, which suffers significantly when its natural language rationales are cut short.
• Compositionality: Permutation tests confirmed that the abstract tokens are not just random fillers; the model's performance drops when the sequence order is scrambled, proving it has learned a meaningful, ordered reasoning logic.

5. Significance

This work represents a significant step toward efficient inference scaling. By moving reasoning from human-readable text to a learned, discrete abstract language, LLMs can perform deep, multi-step reasoning with a fraction of the computational overhead. It provides a middle ground between the high cost of explicit CoT and the lack of control in fully implicit reasoning, opening new avenues for "thinking" models that are both powerful and economically viable for deployment.