🤖 machine learning

CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing

This paper introduces CForce, a consistency forcing distillation method that improves the speed-quality trade-off of diffusion large language models by aligning early-stage mask predictions with later-stage refinements through a novel confidence adaptive KL divergence objective.

Original authors: Yuji Ren, Chenkai Xu, Zhuocheng Gong, Jianguo Li, Zhijie Deng

Published 2026-08-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuji Ren, Chenkai Xu, Zhuocheng Gong, Jianguo Li, Zhijie Deng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers don't just read words one by one, like a person turning pages in a book, but can look at a whole sentence and guess the missing pieces all at once. This is the realm of Diffusion Large Language Models (dLLMs). Think of them like a detective trying to solve a mystery where the crime scene is covered in fog. The detective starts with a page full of question marks (masks) and, step by step, clears away the fog to reveal the hidden words. The faster they can clear the fog, the faster they can write a story.

However, there's a catch. If the detective tries to clear too much fog at once to save time, they might guess the wrong words early on. Once a wrong guess is made, it's hard to fix, and the whole story can go off the rails. This paper tackles that specific problem: how to make these "fog-clearing" detectives super fast without making them careless. The authors introduce a new training trick called Consistency Forcing (CForce) to help the model trust its early guesses more, even when it's rushing.

The Problem: The Rush Hour Mistake

Picture a group of artists trying to paint a mural together. In a traditional setup, they paint one small section, wait for it to dry, and then move to the next. It's slow but safe. Diffusion models are like a team that tries to paint ten sections at once. This is amazing for speed, but if the first few sections are painted with the wrong colors because the artists were rushing, the rest of the mural might look weird, and fixing it later is a nightmare.

The paper points out that when these models try to be super fast (using "aggressive parallelism"), they often make unreliable guesses in the early stages. These early mistakes then spread like a bad rumor, ruining the final result. The goal wasn't just to make the model faster; it was to make the early guesses reliable enough to handle that speed.

The Solution: The "Time-Travel" Coach

Enter Consistency Forcing (CForce). Imagine a coach who watches a student athlete practice. Usually, the coach might just say, "Do it again." But CForce is a special kind of coach that uses a "time-travel" trick.

Here is how it works:

The Self-Play: The model is asked to generate a story on its own, creating a full path from a blank page to a finished sentence. This is its "self-rollout."
The Stages: Instead of looking at every tiny step, the coach breaks this path into "stages." Think of it like watching a movie in fast-forward, but pausing only when a significant chunk of the plot has been revealed.
The Lesson: The coach takes an early stage (where the story is still foggy and uncertain) and compares it to a later stage (where the story is clearer and more complete). The coach tells the model: "Hey, the guess you made when the story was foggy should look a lot like the guess you make when the story is clear."

The model is trained to align its early, shaky predictions with its later, confident ones. It's like telling a student, "Your first draft of the essay should already have the right main ideas, because your final draft will definitely have them."

The Secret Sauce: Confidence and Anchors

To make this training work perfectly, the authors added two clever tools:

Confidence Adaptive KL Divergence: This is a fancy way of saying, "Listen harder when you're sure." If the later stage of the story is very confident about a word, the model is pushed hard to match that prediction. If the later stage is still unsure, the model is allowed to be a bit more flexible. It's a dynamic teacher that knows when to be strict and when to be lenient.
The CE Anchor: This acts like a safety net. When a word is finally revealed (painted on the mural), the model gets a little extra nudge to make sure it's exactly right. This prevents the model from drifting too far off course at the critical moment a word is committed.

What They Found: Speed Without the Crash

The team tested this method on two types of models: one that just writes new text (non-edit) and one that can also go back and fix mistakes (edit-capable).

For the "Fixer" Models: The results were impressive. When using CForce, the model could generate 9.08 tokens (words or parts of words) per forward pass, up from 6.94, while actually getting more accurate (jumping from 85.57% to 86.41% accuracy). It was faster and smarter at the same time.
For the "Writer" Models: Here, there was a trade-off, but a good one. The model could generate 6.42 tokens per pass (up from 3.60), which is a huge speed boost. While the accuracy dipped slightly compared to the slow, careful version, it was still much better than other fast methods.

The paper suggests that this method works because it teaches the model to be consistent with itself. By forcing the early, low-context guesses to match the later, high-context reality, the model learns to make better decisions right from the start.

The Bottom Line

This paper doesn't claim to have solved all the problems of AI speed, but it offers a very promising new way to handle the "rush hour" of text generation. By using the model's own future self to guide its present self, Consistency Forcing helps diffusion models run faster without tripping over their own feet. It's a reminder that sometimes, the best way to move forward quickly is to make sure you're looking in the same direction as the person you'll be later.

Technical Summary: CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing

1. Problem Statement

Diffusion Large Language Models (dLLMs) accelerate text generation by predicting multiple masked tokens in a single forward pass, achieving high parallelism. However, under aggressive parallel decoding strategies (where many tokens are committed simultaneously based on confidence thresholds), early denoising stages often suffer from unreliable predictions due to limited context. These early errors propagate to subsequent stages, degrading overall generation quality.

Existing acceleration methods face two primary limitations:

Inference-time approaches (e.g., caching, speculative decoding) do not modify model parameters to improve inherent prediction reliability.
Training-based approaches often rely on constructed, teacher-generated, or privileged trajectories. These targets differ from the student model's actual inference states (low-context, threshold-decided paths), leading to a training-inference mismatch.

The core challenge is to improve the reliability of early-stage predictions under high-parallelism decoding without relying on external teachers or altering the fundamental decoding algorithm.

2. Methodology: Consistency Forcing (CForce)

CForce is a distillation method designed to align the predictions of early, low-context denoising stages with those of later, more informative stages within the model's own decoding trajectory.

2.1 Staged Trajectory Construction

Instead of training on every native decoding step, CForce constructs "staged" trajectories from self-rollouts of a pretrained dLLM:

Self-Rollout: The model generates a trajectory using confidence-based parallel decoding (revealing tokens above a threshold $\tau$ ).
Stage Partitioning: The trajectory is downsampled into stages based on the cumulative number of newly revealed tokens (stage size $S$ ), rather than native step counts. This ensures adjacent stages represent significant context shifts.
Adjacent Pairs: Training pairs consist of an earlier stage ( $\bar{s}_i$ ) and a later stage ( $\bar{s}_{i+1}$ ). The later stage serves as a stop-gradient target for the earlier stage.

2.2 Training Objective

The method employs a composite loss function to enforce consistency:

Confidence Adaptive KL Divergence (CAD):
- Applied to positions that remain masked in both stages ( $U_i$ ).
- Dynamically interpolates between Forward KL and Reverse KL based on the confidence ( $c_j$ ) of the later-stage prediction.
- Forward KL: Minimizes distributional drift (stable).
- Reverse KL: Encourages mode-seeking (sharpening predictions) but is weighted by confidence to avoid collapse on uncertain targets.
- Formula: $L_{CAD} = \frac{1}{|U_i|} \sum_{j \in U_i} [D_{KL}(p^{tar}_j \| p^{stu}_j) + c_j D_{KL}(p^{stu}_j \| p^{tar}_j)]$
Cross-Entropy (CE) Anchor:
- Applied to newly revealed positions ( $\Delta_i$ ) between stages.
- Anchors the student's prediction to the clean target token immediately before commitment, stabilizing the transition from masked to revealed states.
Curriculum Learning:
- To handle the difficulty of predicting tokens revealed far ahead, the model is gradually exposed to the full gap between stages. The fraction of revealed tokens used in the target state increases from 10% to 100% over training steps.

2.3 Extension to Edit-Capable Models

For models supporting Token-to-Token (T2T) refinement (e.g., LLaDA 2.1), CForce leverages later-stage corrections:

Edit Signals: Positions where tokens are revised in later stages ( $\bar{E}_i$ ) are included in the CAD domain.
Supervision: These T2T corrections provide richer supervision for earlier Mask-to-Token (M2T) predictions, allowing the model to learn from its own ability to self-correct.

2.4 Theoretical Justification

The paper provides a theoretical bound showing that the early-stage prediction error is upper-bounded by the sum of adjacent-stage distributional drift and the token error at the reveal boundary. The CAD term minimizes the drift, while the CE anchor minimizes the boundary error, thereby approximately minimizing the early-stage prediction error.

3. Key Contributions

Reframed Acceleration: Shifts the focus of dLLM acceleration from merely reducing sampling steps to improving the reliability of early-stage predictions under aggressive threshold decoding.
Consistency Forcing: Introduces a distillation method that aligns adjacent stages on the model's own decoding trajectory using a same-model stop-gradient target, eliminating the need for frozen teachers.
Novel Objective: Proposes Confidence Adaptive KL Divergence (CAD) to balance distributional stability and mode-seeking, combined with a CE anchor for token commitment.
Edit-Capable Integration: Demonstrates how T2T refinement signals in edit-capable models can supervise earlier M2T drafting, improving performance in both edit and non-edit settings.

4. Experimental Results

Experiments were conducted on LLaDA 2.0 (non-edit) and LLaDA 2.1 (edit-capable) variants across four benchmarks: GSM8K, MATH500, MBPP, and HumanEval.

Edit-Capable Models (LLaDA 2.1-mini):
- CForce increased average Tokens Per Forward (TPF) from 6.94 to 9.08.
- Average accuracy improved from 85.57% to 86.41%.
- Gains were particularly strong on code generation benchmarks (MBPP, HumanEval).
Non-Edit Models (LLaDA 2.0-mini):
- CForce shifted the model to a higher-parallelism operating point, increasing average TPF from 3.60 to 6.42.
- While average accuracy was slightly lower than the base model (due to the inherent speed-quality trade-off), it significantly outperformed the CAP baseline at high parallelism.
- Under fixed, aggressive TPF budgets (e.g., TPF=8), CForce achieved the highest accuracy among all variants.
Throughput:
- CForce improved wall-clock throughput by 42.82% over the base LLaDA 2.0-mini and 22.84% over the base LLaDA 2.1-mini on average.
Ablation Studies:
- Stage Size: $S=8$ provided the best balance between parallelism and accuracy.
- Target Type: Same-model stop-gradient targets outperformed frozen-teacher baselines.
- KL Form: CAD outperformed pure Forward KL (higher parallelism) and pure Reverse KL (higher accuracy), achieving a superior trade-off.

5. Significance and Claims

The paper claims that Consistency Forcing effectively addresses the reliability bottleneck in high-parallelism dLLM decoding. By forcing early-stage predictions to align with later, more informed states derived from the model's own trajectory, CForce enables:

Higher Parallelism: Models can commit more tokens per forward pass without catastrophic error propagation.
Improved Speed-Quality Trade-off: The method pushes the operating frontier, allowing for faster generation with maintained or improved quality, especially in edit-capable scenarios where self-correction signals are available.
General Applicability: The approach is effective for both standard masked diffusion and edit-capable models, utilizing the specific strengths of each (M2T drafting and T2T refinement).

The authors note that the method currently relies on pre-collected trajectories from a fixed initialization, which may diverge from the student's distribution as training progresses. Future work is suggested to explore online trajectory collection to adapt the forcing objective to the evolving model.