💬 NLP

SelFusion: Self-distillation for Diffusion Language Models

The paper proposes SelFusion, a novel self-distillation framework for Diffusion Language Models that employs bidirectional knowledge distillation between hard and easy masking modes to dynamically correct token-level errors, thereby significantly improving generation quality and often surpassing external teacher models.

Original authors: Hyeongsoo Lim, Jinyoung Kim, Eunseo Seo, Minho Jang, Jiwon Yoon

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hyeongsoo Lim, Jinyoung Kim, Eunseo Seo, Minho Jang, Jiwon Yoon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a persistent tension exists between speed and quality. For years, the most powerful language models have relied on a method called autoregression, where the computer generates text one word at a time, reading its own previous output to decide the next word. This sequential process is like a person writing a sentence by looking back at every word they just wrote before adding the next; it is reliable and produces high-quality results, but it is inherently slow. A newer generation of models, known as diffusion language models, attempts to solve this bottleneck by generating many words simultaneously. Instead of writing a sentence from start to finish, these models start with a block of scrambled, hidden words and gradually refine them into a coherent response, much like a sculptor revealing a statue from a block of stone. While this parallel approach offers the promise of near-instantaneous responses, these models have historically struggled to match the clarity and accuracy of their slower, sequential counterparts.

Researchers have long tried to bridge this gap using a technique called knowledge distillation, where a smaller, faster model learns by mimicking the behavior of a larger, more accurate teacher. However, when scientists applied this standard method to the new diffusion models, the results were disappointing. The attempt to force the fast, parallel model to copy the slow, sequential one often failed, leading to outputs that were no better, and sometimes worse, than if the model had learned on its own. The fundamental issue was a mismatch in how the two types of models "thought" about words. The slow models were very certain about their choices, while the fast models were more uncertain and spread their confidence across many possibilities. Trying to force the uncertain model to copy the certain one was like asking a student to mimic a teacher who speaks a different language; the student could not grasp the underlying logic, only the surface noise.

To overcome this, a team of researchers at Chung-Ang University proposed a new approach called SelFusion, which abandons the idea of an external teacher entirely. Instead of looking outside the model for guidance, the system teaches itself by running two different versions of the same task simultaneously. Imagine a student taking a test where one version of the exam has many questions hidden, forcing the student to guess based on limited clues, while a second version of the same test has fewer questions hidden, providing more context. The researchers created these two "modes" within a single model: a "hard mode" that works with many hidden words and an "easy mode" that works with fewer. The easy mode, having more information, generally produces better predictions, but it is not perfect. It can sometimes become overconfident, sticking to a wrong answer with high certainty just because it had more context to work with.

The innovation lies in how the system handles these two modes. Rather than simply having the easy mode teach the hard mode, the researchers designed a dynamic, two-way exchange. The system constantly checks which mode is actually correct for each specific word. If the easy mode is right, it guides the hard mode. If the hard mode happens to be right while the easy mode is confidently wrong, the hard mode takes the lead. This bidirectional flow ensures that the model learns from the most reliable signal available at any given moment. To prevent the easy mode from being too stubborn about its mistakes, the researchers added a small calibration step that gently lowers its confidence when it is wrong, ensuring it does not mislead the learning process. This entire process happens within a single model, requiring no external teacher and only a tiny addition to the model's size.

When tested on various instruction-following tasks, this self-teaching method proved remarkably effective. The model trained with SelFusion consistently outperformed other methods that relied on external teachers, whether those teachers were the slow, high-quality models or other diffusion models. In many cases, the self-taught model even surpassed the performance of the much larger, slower models it was originally meant to emulate. For instance, on specific benchmarks, the new method achieved significantly higher scores than the best external teacher available, demonstrating that a model can learn to be better than its own initial capabilities by refining its internal logic. The researchers found that this approach not only improved the quality of the generated text but also maintained the speed advantage of the diffusion method, offering a practical path forward for creating fast, high-quality language tools.

The study also highlighted why previous attempts to combine these technologies had failed. By analyzing the internal workings of the models, the team confirmed that the difference in how fast and slow models distribute their confidence was the primary barrier. The slow models were too rigid, and the fast models were too scattered for a direct copy to work. The self-distillation method succeeded because it did not try to force a direct copy; instead, it allowed the model to find the right balance between its own uncertain guesses and its more informed predictions. This approach suggests that the future of fast language generation may not depend on finding a bigger, better teacher, but rather on teaching the model to trust its own best instincts while remaining humble enough to correct its mistakes. The results indicate that with the right internal framework, the speed of parallel generation and the quality of sequential reasoning can finally coexist.

Technical Summary: SelFusion – Self-distillation for Diffusion Language Models

Problem Statement

Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) Large Language Models (LLMs) by leveraging parallel decoding mechanisms to alleviate inference latency bottlenecks. However, DLMs currently suffer from degraded generation quality compared to AR models, limiting their practical applicability. While Knowledge Distillation (KD) has been a successful strategy for improving LLMs, empirical observations in this work reveal that applying conventional KD to DLMs yields marginal gains or even degrades performance.

Two primary failure modes were identified:

LLM-to-DLM Distillation: There is a significant distribution mismatch between AR teachers (which assign near-100% probability to top-1 tokens) and DLM students (which exhibit flatter distributions, e.g., ~60% probability at 50% masking). This mismatch hinders effective logit-level knowledge transfer.
DLM-to-DLM Distillation: Existing DLM teachers often lack sufficient generation quality to provide beneficial supervision, and sequence-level KD remains limited in performance gains.

Methodology: SelFusion

To address these challenges without relying on external teacher models, the authors propose SelFusion, a novel self-distillation framework that enables effective KD within a single DLM.

Core Architecture

SelFusion leverages the inherent noising process of DLMs to create two distinct forward modes within the same model:

Easy Mode: Operates with a lower masking ratio, exposing more context. This mode is expected to yield more accurate predictions and serve as a "teacher" for the harder mode.
Hard Mode: Operates with a higher masking ratio (following the original random masking scheme), acting as the "student."

Key Components

Bidirectional KD:
Contrary to the assumption that the easy mode is always superior, the authors observed that the easy mode can be overconfident on incorrect tokens. Therefore, SelFusion employs a dynamic, token-level bidirectional distillation strategy. The distillation direction ( $D_t$ ) is determined by evaluating the correctness and confidence of both modes:
- If both are correct: The mode with higher probability for the token is the target.
- If both are wrong: The mode with higher probability for the ground-truth token is the target.
- If one is correct: The correct mode serves as the target.
  This ensures the model learns from the most reliable signal at the token level.
RMSNorm-based Logit Calibration:
To mitigate the overconfidence of the easy mode (which tends to produce peaked distributions even on incorrect tokens), the authors insert an RMSNorm layer between the final hidden state and the language modeling head of the easy mode. This selectively suppresses confidence on incorrect predictions while preserving it for correct ones, facilitating more effective knowledge transfer.
Objective Function:
The total loss ( $L_{SelFusion}$ ) is a joint optimization of:
- Diffusion loss for the hard mode ( $L^{(h)}_{diff}$ ).
- Diffusion loss for the easy mode ( $L^{(e)}_{diff}$ ).
- Bidirectional KD loss ( $L_{bkd}$ ) calculated via KL divergence between temperature-scaled distributions.

Key Contributions

Novel Self-Distillation Framework: SelFusion addresses a gap in existing DLM distillation research, which has primarily focused on the pretraining phase, by providing an effective self-distillation approach for post-training scenarios without requiring an external teacher model.
Bidirectional Dynamic Targeting: The introduction of token-level bidirectional KD dynamically selects the distillation direction based on correctness, overcoming the limitations of fixed unidirectional distillation.
Overconfidence Mitigation: The application of RMSNorm-based logit calibration specifically targets the overconfidence issue in low-masking scenarios, a critical factor in stabilizing self-distillation.
Parameter Efficiency: The framework introduces only 1,280 additional parameters (the RMSNorm layer) while eliminating the computational cost of training a separate teacher model.

Experimental Results

The authors evaluated SelFusion on multiple instruction-following benchmarks (Dolly, Self-Inst, Vicuna, Sinst, Uinst) using SMDM architectures (472M and 1476M parameters).

Performance Gains: SelFusion substantially outperformed all baselines, including conventional logit-level and sequence-level KD with external LLM and DLM teachers.
Surpassing Teachers: In many configurations, the student model trained with SelFusion surpassed the performance of the LLM teacher (e.g., achieving higher ROUGE-L scores on Self-Inst and Vicuna).
Ablation Studies: Removing bidirectional KD or RMSNorm resulted in consistent performance degradation across all benchmarks, confirming the necessity of both dynamic direction selection and logit calibration.
Efficiency: While SelFusion requires approximately 2 $\times$ more per-iteration computation than standard Supervised Fine-Tuning (SFT), it reduces total training computation by roughly 2 $\times$ compared to KD methods that require a separately trained teacher.
Inference Speed: DLMs maintain a significant speed advantage over AR models; even with 32 diffusion steps, DLMs achieved a 1.5 $\times$ speedup over LLMs.

Significance and Claims

The paper claims that SelFusion provides a practical path toward improving DLM generation quality. By decomposing a single model into easy and hard modes, it creates a "student-friendly" teacher distribution that aligns better with the student's learning dynamics than external AR teachers.

The authors emphasize that SelFusion effectively transfers knowledge within a single model, overcoming the distribution mismatch inherent in cross-architecture distillation. The work suggests that self-distillation, when properly engineered with bidirectional logic and calibration, can bridge the performance gap between DLMs and AR models without the overhead of external teacher models.

Limitations

The authors note several limitations:

The current availability of DLM backbones is limited, constraining validation across a broader set of architectures.
Evaluation is currently limited to English instruction tuning, leaving broader domains and multilingual settings for future work.