💬 NLP

Self-Speculation for Faster Reasoning Models

This paper introduces SSR (Self-Speculation for Reasoning Models), a training-free decoding method that accelerates large language models by leveraging partial chain-of-thought responses as drafts to verify and extend full reasoning traces, achieving up to 24.1% latency reduction on complex, long-form generation tasks.

Original authors: Ravisri Valluri, Tung Nguyen, Aditya Grover

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ravisri Valluri, Tung Nguyen, Aditya Grover

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, large language models have evolved from simple chatbots into sophisticated problem-solvers capable of planning, coding, and making complex decisions. To achieve this level of intelligence, these models often engage in a process called "chain-of-thought" reasoning. Instead of jumping straight to an answer, the model generates a long, step-by-step internal monologue, working through the problem before producing a final response. While this extra thinking time significantly improves the quality of the answer, it also creates a bottleneck: the longer the model thinks, the longer a user must wait to see the result. For interactive applications like voice assistants or coding tools, this delay can be frustrating, breaking the flow of conversation or slowing down a developer's workflow. The challenge for researchers has been to find a way to keep the high-quality reasoning without forcing the user to wait for every single step of the thinking process to finish before the answer begins to appear.

A team of researchers at the University of California, Los Angeles, has developed a new method called Self-Speculation for Reasoning Models, or SSR, which aims to solve this latency problem without sacrificing the quality of the output. Their approach is built on a simple but powerful observation: as a model thinks through a problem, its early guesses about the final answer often contain significant clues about what the final answer will actually be. Even before the model finishes its internal reasoning, it has usually settled on the general structure and key details of the solution. The researchers realized that they could use these early, incomplete thoughts as a "draft" to predict the final answer, while the model continues its full reasoning process in the background.

The method works by running two versions of the same model at the same time. One version, acting as a drafter, stops the reasoning process early and immediately tries to generate the final answer based on the partial thoughts it has so far. Simultaneously, the main version of the model continues its full, deep reasoning process. Once the main model finishes its long chain of thought, it acts as a verifier, checking the draft generated by the early version. If the draft matches the final, fully reasoned answer, the system accepts the draft tokens, effectively skipping the time it would have taken to generate them one by one. Because the drafting happens in parallel with the main reasoning, the time spent creating the draft is hidden; the user does not wait for it. This allows the system to deliver the final response much faster, particularly for tasks that require long, structured outputs like writing code or planning complex sequences.

To make this even more effective, the researchers added a second layer to the process. In many cases, the early draft might get a few details wrong at the very beginning—perhaps using the wrong variable name or a slightly different phrasing—but then align perfectly with the final answer for a long stretch of text. Standard methods would discard the entire draft at the first mistake, but the new system includes a "suffix decoding" feature. This allows the system to look past the initial errors and find matching sections of text later in the draft that the final answer also uses. By recovering these useful chunks of text, the system can accept even more of the draft, further reducing the time needed to generate the response.

The researchers tested this method on several demanding tasks, including generating code for complex software classes and creating long, structured documents. They found that for models working on these types of problems, the new method reduced the total time it took to generate a response by up to 24.1 percent. This speedup was achieved without changing the model's weights or requiring any additional training, meaning it can be applied to existing systems immediately. The results were most pronounced in tasks where the final answer was long and structured, such as coding, where the model tends to commit to the overall structure early in its thinking process. In contrast, for shorter tasks where the thinking time itself is the dominant factor, the speed gains were smaller, as the method primarily accelerates the answer generation phase rather than the reasoning phase.

The study also explored how the method behaves when the model is asked to think for different lengths of time. They found that the best results come from starting the draft generation when the model has completed a substantial portion of its reasoning, ensuring the draft is based on enough context to be accurate. They also developed an iterative version of the method that updates the draft at multiple points during the reasoning process, allowing the system to refine its predictions as the model thinks deeper. This approach ensures that even if an early draft is not perfect, later drafts can build upon it, maintaining a high rate of success.

Ultimately, this work demonstrates that the structure of the reasoning process itself can be used as a resource to speed up artificial intelligence. By treating the model's own intermediate thoughts as a source of prediction, the researchers have created a way to deliver high-quality, reasoned answers with significantly less waiting time. The method is particularly valuable for interactive applications where speed is essential, proving that it is possible to have both deep thinking and fast responses without compromising the quality of the solution.

Technical Summary: Self-Speculation for Faster Reasoning Models (SSR)

Problem Statement
Large Language Models (LLMs), particularly reasoning models, achieve high performance on complex tasks like planning, coding, and decision-making by generating long chains of thought (CoT). However, this reliance on extended reasoning traces significantly increases end-to-end generation latency, creating a bottleneck for latency-sensitive interactive applications such as voice assistants and coding agents. Existing acceleration methods, including speculative decoding, typically focus on token-level generation using separate draft models or architectural modifications. These approaches often fail to leverage the specific structural properties of reasoning workflows, where the reasoning process itself evolves toward a final answer.

Methodology: Self-Speculation for Reasoning Models (SSR)
The authors propose SSR, a training-free self-speculative decoding method that utilizes the chain-of-thought as a source of speculation. The core insight is that partial CoT responses, as reasoning progresses, exhibit increasing semantic and lexical overlap with the final high-budget response. SSR leverages this by using the same language model at two different reasoning budgets:

Drafting: A lower-budget instance ( $b_d$ ) generates draft answer tokens based on a partial CoT prefix ( $r_{\le b_d}$ ).
Verification: A higher-budget instance ( $b_v$ ) continues the reasoning process to the full budget, acting as the verifier.
Concurrency: Draft generation and continued reasoning proceed concurrently. The overhead of drafting is hidden behind the ongoing CoT generation, as the draft is produced while the model is still reasoning toward the final budget.

Key Technical Components

Prefix Verification: SSR employs standard speculative decoding verification to accept the longest matching prefix between the draft and the final answer. If the draft is accurate, multiple tokens are accepted in a single verifier pass, reducing sequential decoding steps.
Suffix Decoding: Recognizing that standard prefix verification discards the entire draft upon the first mismatch, SSR introduces suffix decoding. It builds a suffix cache from the draft tokens to propose spans that may match the final answer after the initial rejection point. This recovers useful text in scenarios where the draft and final answer diverge on minor early details (e.g., variable names) but agree on long subsequent spans, a common occurrence in structured tasks like coding.
Iterative and Multi-Stage Variants: To address the uncertainty of the optimal sampling point (since total CoT length is unknown), SSR includes an iterative variant. This approach samples drafts at multiple points during CoT generation, using earlier accepted prefixes to bootstrap later drafts, thereby reducing the overhead of multi-point sampling.
Implementation: The method is implemented in vLLM as a drop-in addition. It duplicates the request at the draft budget, utilizing prefix caching to avoid redundant computation on the shared reasoning prefix.

Key Contributions

Training-Free Self-Speculation: A method that requires no auxiliary parameters or target-model-specific modifications, deriving both draft and verifier distributions from the same model at different reasoning budgets.
Suffix Decoding Extension: An extension to standard verification that recovers useful draft spans beyond the first mismatch, significantly improving acceptance rates for long-form and structured outputs.
Iterative Sampling: A strategy to sample drafts at multiple points during generation, using earlier drafts to bootstrap later ones and mitigate the risk of selecting a suboptimal draft budget.
Practical Integration: A vLLM implementation that integrates seamlessly into existing serving configurations without altering model weights.

Experimental Results
The authors evaluated SSR on structured and long-form generation tasks, including HumanEval, ClassEval, and LongProc, using open-source 4B-class reasoning models (Qwen3.5-4B and Gemma-4-E4B-it).

Latency Reduction: SSR achieved relative latency reductions of up to 24.1% on ClassEval (Gemma-4-E4B-it) and 18.5% on the same benchmark for Qwen3.5-4B.
Task Variance: Improvements were most pronounced on tasks with long, structured outputs (ClassEval, LongProc 2K), where accepted draft tokens effectively offset drafting overhead. Gains were more modest on shorter tasks like HumanEval, where CoT generation dominates total latency.
Ablation: Experiments confirmed that while prefix verification alone yields modest gains, the combination with suffix decoding is critical for maximizing speedup, particularly in scenarios where drafts share high lexical overlap with the final answer despite early mismatches.
Overhead: The concurrent drafting process introduces minimal effective overhead (under 2% in measured throughput) because the drafting cost is hidden behind the reasoning process, and prefix caching minimizes redundant computation.

Significance and Claims
The paper claims that SSR offers a "drop-in" solution for accelerating reasoning models without the need for training or architectural changes. By treating the chain-of-thought not just as a reasoning step but as a resource for speculation, SSR effectively utilizes the convergence properties of reasoning models. The authors emphasize that the method preserves output quality (under exact verification) while providing consistent latency reductions on tasks where the answer length is comparable to the reasoning trace length. The work suggests that reasoning structure itself can be a valuable resource for efficient generation, encouraging further exploration into leveraging internal model states for acceleration.

Technical Summary: Self-Speculation for Faster Reasoning Models (SSR)

More like this