💬 NLP

$\texttt{SPECS}$ : Faster Test-Time Scaling through Speculative Drafts

The paper introduces $\texttt{SPECS}$ , a latency-aware test-time scaling method that leverages a smaller model for speculative drafting and reward-guided verification to achieve beam search-level accuracy with up to 19.1% reduced latency.

Original authors: Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer, Ion Stoica, Kannan Ramchandran, Ahmad Beirami, Ziteng Sun

Published 2026-02-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer, Ion Stoica, Kannan Ramchandran, Ahmad Beirami, Ziteng Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a very difficult math problem. You have two helpers:

The Professor: A brilliant, slow-thinking expert who gets almost everything right but takes a long time to write down every single step.
The Speedy Intern: A fast, energetic worker who writes quickly but makes mistakes.

The Problem: The "Thinking" Bottleneck

In the world of AI, we want Large Language Models (LLMs) to "think" harder to solve complex problems. This usually means asking the "Professor" to generate multiple possible solutions (like brainstorming 10 different ways to solve an equation) and picking the best one.

But here's the catch: Asking the Professor to do all that thinking takes forever. If you ask them to check 10 paths, you wait 10 times longer. Users hate waiting. They want the answer now.

The Old Solution: Just Use the Intern?

Some people tried just using the Speedy Intern to save time. But the Intern is too error-prone. If you ask them to solve a hard problem, they often get lost in the first few steps, and the whole answer is garbage.

The New Solution: SPECS (The Smart Switch)

The paper introduces SPECS, a clever new way to use both the Professor and the Intern together. Think of it as a Smart Traffic Controller for your problem-solving process.

Here is how SPECS works, step-by-step, using a simple analogy:

1. The "Safety First" Start

When you first ask the AI to solve a problem, the system starts with the Professor. Why? Because the beginning of a problem is the most critical. If you start with the wrong premise, the whole answer is wrong. The Professor sets a solid foundation.

2. The "Confidence Check"

As the Professor writes down the first few steps, a special Scorekeeper (called a Reward Model) watches closely.

If the Professor is struggling (low score): The system keeps the Professor working. It's too risky to let the Intern take over yet.
If the Professor is crushing it (high score): The Scorekeeper says, "Hey, this path looks great! We are on the right track."

3. The Dynamic Switch (The Magic Moment)

Once the Scorekeeper confirms the path is strong, SPECS flips a switch. It stops asking the slow Professor to write the next steps and hands the pen to the Speedy Intern.

Because the path is already proven to be good, the Intern can race ahead and finish the rest of the solution very quickly without messing up.

4. The "Draft & Verify" Loop

Sometimes, the Intern might get a little too excited and suggest a few different endings. The system doesn't just blindly accept them. It quickly checks the Intern's suggestions against the Professor's logic (a process called "soft verification"). If the Intern's idea is good, it's accepted. If not, it's tossed out.

Why is this a big deal?

Speed: By letting the fast Intern do the heavy lifting on the "easy" parts of the reasoning, the total time drops by up to 18%. That's like shaving 30 seconds off a 3-minute wait.
Accuracy: Because the system only switches to the Intern when it's sure the path is correct, the final answer is just as accurate as if the Professor had done the whole thing alone.
Smart Resource Use: It's like hiring a CEO to start a project and a junior employee to finish the paperwork once the plan is solid. You get the best of both worlds: high quality and low cost.

The Bottom Line

SPECS is a method that stops AI from wasting time doing hard thinking when it doesn't need to. It uses a "slow but smart" model to start the journey, checks if the path is safe, and then lets a "fast but less smart" model zoom to the finish line.

It's the difference between driving a luxury car at 20 mph the whole way versus driving that same car to the highway entrance, checking the traffic, and then switching to a sports car to blast the rest of the way home. You get there faster, but you still arrived safely.

1. Problem Statement

Recent advances in Large Language Models (LLMs) have leveraged test-time scaling (increasing compute at inference time) to improve reasoning capabilities. However, traditional scaling methods like Beam Search or Tree Search often prioritize accuracy at the expense of latency.

The Bottleneck: In autoregressive generation, latency is often limited by memory loading (bandwidth) rather than total FLOPs. Generating multiple candidate reasoning traces (beams) using a large, capable target model is computationally expensive and slow.
The Gap: Existing speculative decoding methods (using a small draft model to speed up generation) typically start with the draft model and switch to the target model only when the reward signal is low. This approach often wastes latency on low-quality drafts or sacrifices accuracy.
The Goal: Design a test-time scaling method that optimizes for inference latency while maintaining or improving accuracy, specifically for complex reasoning tasks.

2. Methodology: SPECS

The authors propose SPECS (SPECulative drafting for faster test-time Scaling), a latency-aware algorithm that combines speculative decoding with dynamic model switching based on reward signals.

Core Components

Hybrid Generation Strategy:
- Unlike standard speculative decoding (Draft $\to$ Target), SPECS starts generation with the large Target Model ( $\pi_{target}$ ).
- It dynamically switches to a small, fast Draft Model ( $\pi_{draft}$ ) only when the reasoning trace achieves a high reward score.
- Rationale: Empirical observations show that for "high-reward" traces (indicating a correct reasoning path), the accuracy gap between small and large models is negligible. Thus, the small model can complete these steps faster without hurting accuracy. Conversely, for "low-reward" traces, the system defers to the large model to avoid errors.
Reward-Guided Soft Verification (SUBSAMPLE):
- Instead of hard rejection (accept/reject), SPECS uses a scoring function to select the best candidate block from a set of $n$ candidates.
- Score Function ( $S_i$ ):
  $S_i = \log \left( \frac{\pi_{target}(Y_i | tr)}{\pi_{gen}(Y_i | tr)} \right) + \beta r(Y_i | tr)$
  - $\pi_{target}$ : The large target model.
  - $\pi_{gen}$ : The current generator (either target or draft).
  - $r$ : The Process Reward Model (PRM) score.
  - $\beta$ : Inverse temperature parameter.
- This score integrates the likelihood ratio (ensuring the draft aligns with the target distribution) and the reward signal (ensuring reasoning quality).
Dynamic Switching Mechanism:
- The algorithm monitors the maximum reward ( $\max r(Y_i|tr)$ ) among the current beam candidates.
- If the max reward exceeds a threshold $\tau$ , the generation mode switches to the Draft Model for subsequent steps.
- If the reward is low, it remains with or reverts to the Target Model.

Algorithm Flow

Initialize: Start with the Target Model.
Generate: Produce $n$ candidate blocks (beams).
Score: Evaluate candidates using the Target Model (for log-likelihood) and PRM (for reward).
Select: Use the SUBSAMPLE routine to probabilistically select the best block based on the score $S_i$ .
Switch: If the selected block's reward is high ( $> \tau$ ), switch the generator to the Draft Model for the next iteration.
Repeat: Continue until the sequence is complete.

3. Key Contributions

Novel Algorithm (SPECS): A test-time scaling method that utilizes speculative drafting to achieve a favorable latency-accuracy tradeoff. It uniquely starts with the large model and switches to the small model only for high-reward traces, contrasting with prior work that starts small and switches large.
Theoretical Grounding:
- The authors prove that as the beam width ( $n$ ) increases, the SPECS policy converges to the optimal solution of a KL-regularized reinforcement learning objective.
- They provide theoretical bounds showing that the KL divergence between the SPECS policy and the optimal policy decreases as $O(1/n^{3/2})$ .
- The analysis extends to cases with noisy Process Reward Models (PRMs), showing graceful degradation.
Empirical Performance:
- Extensive evaluation on MATH500, AMC23, OlympiadBench, and GPQA.
- Demonstrates that SPECS matches or exceeds the accuracy of standard Beam Search while reducing latency by up to ~18%.
- Ablation studies confirm that both the dynamic switching and the log-likelihood based scoring are critical for performance.

4. Experimental Results

Datasets: Mathematical reasoning (AMC23, MATH500, OlympiadBench) and General QA (GPQA).
Models:
- Draft/Target Pairs: Qwen2.5-1.5B / Qwen2.5-7B and Llama3.2-1B / Llama3.1-8B.
- Reward Model: Skywork-o1-Open-PRM-Qwen-2.5-7B.
Key Findings:
- Latency-Accuracy Pareto Frontier: SPECS dominates the Pareto frontier. For a fixed accuracy, it is significantly faster than Beam Search with the large model. For a fixed latency, it is more accurate than Beam Search with the small model.
- Adaptivity: On harder datasets (e.g., OlympiadBench), SPECS automatically uses the large model for a higher percentage of steps, whereas on easier tasks, it relies more heavily on the draft model.
- Robustness: The method remains robust even when the PRM is noisy (simulated with Gaussian noise), outperforming baselines without hyperparameter retuning.

5. Significance

Efficiency in Reasoning: SPECS addresses a critical bottleneck in deploying LLMs for reasoning tasks: the high latency of multi-step inference. It proves that "thinking" (test-time compute) does not have to be slow.
Paradigm Shift: It challenges the standard speculative decoding workflow (Small $\to$ Large) by introducing a Large $\to$ Small switching strategy based on trajectory quality. This insight—that small models are sufficient for "easy" (high-reward) reasoning steps—is a novel contribution.
Theoretical Rigor: By linking the algorithm to KL-regularized RL and providing convergence guarantees, the paper moves beyond heuristic engineering to provide a theoretically sound framework for test-time scaling.
Practical Impact: The method is compatible with existing inference infrastructure (e.g., vLLM) and offers immediate latency reductions for applications requiring high-quality reasoning, such as math solvers and scientific QA.