💬 NLP

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

This paper introduces SlowFast Sampling, a dynamic strategy guided by three golden principles that adaptively alternates between exploratory and accelerated decoding to significantly accelerate diffusion-based language models, achieving up to 34.22 $\times$ speedup with minimal accuracy loss while outperforming autoregressive baselines in throughput.

Original authors: Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Puyu Zeng, Yuxuan Wang, Biqing Qi, Dongrui Liu, Linfeng Zhang

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Puyu Zeng, Yuxuan Wang, Biqing Qi, Dongrui Liu, Linfeng Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to write a long, complex story, but you have a magical assistant who can see the whole story at once, rather than writing it word-by-word like a human. This is what Diffusion Large Language Models (dLLMs) do. Instead of writing one word after another (which is slow), they look at a blank page full of "masks" (erasers) and try to fill in the whole story at once, refining it step-by-step until it makes sense.

However, there's a catch. Because the assistant is trying to guess the whole story at once, it often gets confused. It might confidently guess the first word, but then change its mind about the last word five times before it's done. This "changing its mind" process takes a lot of time, making the model slower than we'd like.

Existing methods to speed this up are like a rigid robot: they either guess too many words too early (causing mistakes) or guess too few (wasting time). They don't adapt to how the story is actually unfolding.

The Solution: SlowFast Sampling

The authors of this paper propose a new way to talk to this magical assistant called SlowFast Sampling. Think of it as a smart editor who knows exactly when to slow down and when to speed up.

They discovered Three Golden Principles that govern how the model thinks:

The Certainty Principle (The "I Know This!" Rule):
Sometimes, the model is 100% sure about a word. It's like when you're writing a sentence and you know the subject is "The cat." You don't need to second-guess it. The paper says: If the model is confident, lock that word in immediately and stop wasting time re-checking it.
The Convergence Principle (The "Settling Down" Rule):
At the start, the model's guesses bounce around wildly. But as it gets closer to the end of the process, the guesses stop changing. They "converge" to a stable answer. It's like a spinning top that eventually stops wobbling and stands still. Once the top stops wobbling, you don't need to watch it anymore; you know it's done.
The Positional Principle (The "Neighborhood" Rule):
The model doesn't just get confident about random words scattered everywhere. It tends to get confident about chunks of words together. If it figures out a sentence about "The cat," it likely figures out the next few words about "sat on the mat" at the same time. It works in neighborhoods, not isolated islands.

How SlowFast Sampling Works

Based on these rules, the method uses a two-step dance:

Phase 1: The Slow Walk (Exploration)
The model walks slowly and cautiously. It looks at the text, finds the small "neighborhoods" where the model is starting to feel confident, and checks if those feelings are stable. It's like a scout walking ahead to find a safe path. It doesn't rush; it just wants to find a spot where the model says, "I'm sure about this part."
Phase 2: The Fast Run (Acceleration)
Once the scout finds a stable, confident neighborhood, the model switches to "Turbo Mode." It grabs that whole chunk of text and locks it in all at once. It skips over the parts that are still confusing (leaving them for later) and focuses its energy only on the parts that are ready.

The Magic of "Caching"

The paper also mentions combining this with a "cache" (a memory bank). Imagine that while the model is running fast on the "safe" parts, it also remembers the guesses it made for the "unsafe" parts. If those unsafe parts become safe later, the model doesn't have to re-calculate them from scratch; it just pulls the answer from its memory. This is like remembering your grocery list so you don't have to re-write it every time you go to the store.

The Results: Why It Matters

The authors tested this on some very hard math and science questions (like the GPQA benchmark).

Without this method: The model was slow, generating about 1.6 words per second.
With SlowFast Sampling: It got 15 times faster.
With SlowFast + Memory Cache: It got 34 times faster.

Even more impressively, this new method made the diffusion model faster than the famous LLaMA 3 (a standard, word-by-word AI), while keeping the answers just as accurate.

The Big Picture

Think of traditional AI as a person typing a letter one key at a time. It's reliable but slow.
Think of this new Diffusion AI as a painter who tries to paint the whole canvas at once. It's powerful but messy.
SlowFast Sampling is the artist's new technique: they slowly sketch the outlines (Slow) to make sure the composition is right, and then they rapidly fill in the colors (Fast) once they know the shapes are solid.

This paper proves that by understanding how AI thinks (its confidence, its stability, and its patterns), we can make it work much, much faster without losing its intelligence.

1. Problem Statement

Diffusion-based Large Language Models (dLLMs) offer a promising alternative to traditional autoregressive (AR) models by enabling parallel token generation, which theoretically reduces inference latency. However, existing sampling strategies for dLLMs suffer from significant limitations:

Static Behavior: Current methods (e.g., confidence-based selection like Fast-dLLM or semi-autoregressive decoding) use fixed rules or block sizes throughout the generation process.
Suboptimal Efficiency: These static approaches often fail to adapt to the dynamic nature of the diffusion process, leading to either excessive redundant computation (re-evaluating stable tokens) or significant accuracy drops when attempting aggressive parallel decoding.
Latency Bottleneck: The iterative denoising process remains computationally expensive, preventing dLLMs from matching the throughput of state-of-the-art AR models like LLaMA3.

The core challenge is to design a dynamic sampling strategy that intelligently decides how many tokens to sample and where to sample them at each step, balancing the trade-off between generation speed and quality.

2. Methodology: SlowFast Sampling

The authors propose SlowFast Sampling, a novel two-stage dynamic decoding strategy guided by three empirically derived "Golden Principles." The method alternates between an Exploratory Stage (Slow) and an Accelerated Decoding Stage (Fast).

The Three Golden Principles

The strategy is built upon three observations regarding token behavior during the diffusion process:

Certainty Principle: Tokens predicted with high confidence are more likely to be correct and require fewer subsequent adjustments. High-confidence tokens can be "locked in" early.
Convergence Principle: As the diffusion process iterates, the confidence scores of many tokens stabilize (converge) to a steady value. Once a token's identity and confidence stabilize over a short window, it is unlikely to change further.
Positional Principle: High-confidence and converging tokens do not appear randomly; they tend to emerge in contiguous blocks or localized regions. This allows for targeted decoding and cache reuse in specific spans.

The Two-Stage Pipeline

Exploratory Stage (Slow Phase):
- Goal: Cautiously advance decoding to identify a stable region for rapid processing.
- Mechanism: The model performs conservative decoding, selecting only the top- $k$ high-confidence tokens within a current window.
- Stability Check: The model tracks the "end point of convergence" (the furthest token with confidence above a threshold). It calculates the variance of these end points over a sliding window.
- Transition: Once the variance of the convergence horizon drops below a stable threshold ( $\sigma^2_{stable}$ ), the system identifies a stable span $[s_{cycle}, e_{cycle}]$ and transitions to the Fast Phase.
Accelerated Decoding Stage (Fast Phase):
- Goal: Rapidly denoise the identified stable span in parallel.
- In-Span Parallel Decoding: Within the stable span, all tokens with confidence exceeding a high threshold ( $\tau_{high\_conf}$ ) are unmasked simultaneously in a single step.
- Out-of-Span Caching: For tokens outside the stable span (future tokens), the model computes predictions but caches them. These cached values are reused in subsequent steps if the token remains outside the active decoding span, reducing redundant computation.
- Fallback: If insufficient tokens meet the high-confidence threshold, the system reverts to a conservative top- $k$ update to ensure progress.

Integration with dLLM-Cache

The method is designed to be compatible with dLLM-Cache, a feature caching mechanism. By identifying stable regions, SlowFast Sampling maximizes the utility of cached features, further reducing redundant computations.

3. Key Contributions

Three Golden Principles: The paper formally defines the Certainty, Convergence, and Positional principles, providing a theoretical and empirical foundation for understanding token dynamics in dLLMs.
SlowFast Sampling Algorithm: A novel, adaptive two-stage sampling strategy that dynamically switches between cautious exploration and aggressive parallel decoding based on real-time token stability.
State-of-the-Art Performance: Demonstrates that dLLMs, when optimized with this sampling strategy, can surpass traditional autoregressive models in throughput while maintaining comparable accuracy.

4. Experimental Results

Experiments were conducted on LLaDA 8B and Dream 7B across multiple benchmarks (GSM8K, GPQA, MMLU, BBH, HumanEval, etc.).

Speedup on GPQA:
- SlowFast Sampling alone: Achieved a 15.63× speedup on LLaDA 8B (from 3.31 to 16.36 TPS) with minimal accuracy drop (31.47% $\to$ 31.91%).
- SlowFast + dLLM-Cache: Achieved a 34.22× speedup (54.75 TPS) with an accuracy of 28.79%.
Comparison with Autoregressive Models:
- On the GPQA benchmark, the optimized LLaDA (SlowFast + Cache) achieved 54.75 TPS, significantly outperforming LLaMA3 8B (33.79 TPS) while maintaining comparable accuracy (28.79% vs. 31.92%).
General Benchmarks:
- Consistent improvements were observed across mathematics, science, and coding tasks. For example, on GSM8K, SlowFast achieved a 5.93× speedup with Cache, and on BBH, a 10.13× speedup.
Ablation Studies:
- Hyperparameter analysis confirmed that a maximum of 8 exploratory steps ( $K_{max}$ ), a sliding window of 2 ( $W_{hist}$ ), and a variance threshold of 1.0 provide the optimal speed-quality trade-off.

5. Significance

This work represents a critical step toward the practical deployment of diffusion-based LLMs.

Bridging the Gap: It demonstrates that dLLMs are not just theoretically parallel but can be made practically faster than the dominant autoregressive paradigm through intelligent sampling.
Efficiency vs. Quality: It challenges the notion that parallel decoding inherently sacrifices accuracy, showing that dynamic, principle-guided sampling can maintain high quality.
Scalability: The combination of dynamic sampling and caching offers a scalable path to reducing the inference latency of large generative models, making them viable for real-time applications.

In summary, SlowFast Sampling unlocks the full potential of dLLMs by moving away from static, one-size-fits-all decoding toward a dynamic, adaptive approach that leverages the inherent stability and spatial patterns of token generation.