💬 NLP

Unlocking Prompt Infilling Capability for Diffusion Language Models

This paper demonstrates that by extending supervised finetuning to include full-sequence masking rather than just response-only masking, masked diffusion language models can effectively unlock the capability to infill prompts, achieving performance that matches or surpasses manually designed templates.

Original authors: Yoshinari Fujinuma, Keisuke Sakaguchi

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yoshinari Fujinuma, Keisuke Sakaguchi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Blind" Artist

Imagine you have a brilliant artist (the AI model) who can paint a picture by looking at a messy, scribbled canvas and slowly cleaning it up until a perfect image appears. This is how Diffusion Language Models work. Unlike traditional AI that writes one word at a time from left to right (like a typist), this artist can look at the whole picture at once and fix any part of it, whether it's the sky, the trees, or the people.

The Problem:
Even though this artist can fix any part of the picture, the way they were trained in school (Supervised Fine-Tuning) was very strange.

The Old Training: The teacher gave the artist a perfect, clean instruction card (the Prompt) and a messy, scribbled answer card (the Response). The teacher said, "Only fix the answer card. Leave the instruction card alone."
The Result: The artist became a master at fixing answers, but if you handed them a messy instruction card and asked them to fix it, they froze. They had never been taught that they were allowed to touch the instructions. They thought, "I'm only supposed to fix the answer!"

The Paper's Solution:
The researchers realized this wasn't a flaw in the artist's brain (the architecture); it was a flaw in the training manual. They introduced a new rule: Full-Sequence Masking.

The New Training: Now, the teacher gives the artist a messy instruction card and a messy answer card. The artist has to fix both.
The Result: The artist learns that they can fix the instructions too! This unlocks a superpower called Prompt Infilling.

How It Works in Real Life: The "Fill-in-the-Blank" Magic

Imagine you are a teacher trying to grade student essays. You have a template for the grading rubric, but you left some parts blank because you want the AI to figure out the best way to grade based on a few examples you give it.

Before this paper:
You give the AI: "Grade this essay based on [BLANK] criteria."
The AI panics. It tries to guess the criteria but often just writes nonsense or stops writing because it was never trained to fill in the "grading rules" part of the sentence.

After this paper:
You give the AI the same template. Because it was trained to fix both the instructions and the answers, it looks at your examples, understands what kind of grading you want, and fills in the blanks perfectly. It might even invent a clever new way to score the essay that you didn't think of!

Why This Matters: Three Key Takeaways

1. It's a Training Issue, Not a Hardware Issue

The paper proves that the AI models already had the ability to fix prompts; they just never practiced it. It's like having a car with a turbo button that was never connected to the engine. The researchers just connected the wire (changed the training method), and suddenly the car goes much faster.

2. The "Self-Improving" Prompt

The AI can now look at a few examples of a task and write its own "instruction manual" for how to do that task best.

Analogy: Imagine you give a chef a few examples of your favorite dish. Instead of you writing a recipe card for them, the chef looks at the examples, writes their own perfect recipe card, and then cooks the dish using that new card. The result is often better than if you had tried to write the recipe yourself.

3. It Works Everywhere

The researchers tested this on math problems and on judging the quality of summaries.

Math: The AI could fill in the missing parts of a math problem description to solve it better than standard methods.
Grading: When the AI had to act as a judge, it could create a scoring system that matched human judges better than any human-designed system they tried.

The "Secret Sauce" (Full-Sequence Masking)

The paper compares three ways of training:

The Old Way (Response-Only): Only fix the answer. (Result: The AI is bad at fixing instructions).
The New Way (Full-Sequence): Fix everything, including the instructions. (Result: The AI becomes a master at understanding and improving its own instructions).
The Hybrid Way: Fix everything first, then practice just the answers a little bit more. (Result: The best of both worlds).

The Bottom Line

This paper is a wake-up call for the AI community. We often think we need to build bigger, more complex AI brains to do new things. But sometimes, we just need to change how we teach them. By letting these models practice "filling in the blanks" on their own instructions, we unlock a level of intelligence and flexibility that was sitting there, locked away, waiting for the right key.

In short: They didn't build a better car; they just taught the driver how to use the turbo button.

1. Problem Statement

Masked Diffusion Language Models (dLMs) generate text through iterative bidirectional denoising, theoretically allowing them to fill in any masked position within a sequence based on surrounding context. This "infilling" capability is a distinct advantage over autoregressive models (which generate strictly left-to-right).

However, current state-of-the-art dLMs (e.g., LLaDA, Dream) fail to utilize this capability for prompt infilling.

The Limitation: During Supervised Fine-Tuning (SFT), these models are trained using Response-Only Masking. In this convention, the prompt ( $P$ ) remains fully visible (clean), while only the response ( $R$ ) is masked and denoised.
The Consequence: The model learns to denoise responses given clean prompts ( $P(R|P)$ ) but never learns to denoise prompts given responses ( $P(P|R)$ ).
The Gap: This creates a training-inference gap. While the model's architecture is bidirectional and theoretically capable of infilling prompts, the training signal is missing. Consequently, when users attempt to use dLMs to infill masked regions of a prompt template (e.g., to generate task-specific instructions or scoring rubrics from few-shot examples), the models fail, often outputting end-of-sequence tokens or garbage instead of meaningful content.

2. Methodology

The authors propose a simple but fundamental modification to the training paradigm to bridge this gap.

A. Full-Sequence Masking (FS) during SFT

Instead of masking only the response, the authors introduce Full-Sequence Masking during the supervised fine-tuning stage.

Mechanism: For every training example $X = (P, R)$ , a masking ratio $t$ is sampled. Tokens are randomly selected from the entire sequence (both prompt and response) to be masked.
Training Objective: The model is trained to recover masked tokens in both the prompt and the response simultaneously. The loss function applies cross-entropy over all masked positions:
$\mathcal{L}_{\text{full-seq}} = -\mathbb{E} \left[ \frac{1}{|M_t|} \sum_{i \in M_t} \log P(X^0_i | X^t_i) \right]$
where $M_t$ is the set of all masked positions.
Optional Refinement: The authors also test a two-stage approach: Full-Sequence Masking followed by a Response-Only Masking refinement stage (FS+RO) to specialize the model for standard generation while retaining infilling capabilities.

B. Inference Procedure: Prompt Infilling

Once trained with Full-Sequence Masking, the model is used for prompt infilling at inference time:

Input: A partially masked prompt template and a set of few-shot examples (input-output pairs).
Process: The dLM iteratively denoises the masked tokens in the prompt template, conditioned on the few-shot examples.
Selection: Multiple candidate prompts are generated. These are validated by running them on the few-shot examples (using either the dLM or an external LLM) to select the prompt that yields the best performance.
Application: The selected "infilled" prompt is fixed and used for all subsequent test inputs.

3. Key Contributions

Identification of the Training-Inference Gap: The paper demonstrates that the inability of dLMs to infill prompts is not an architectural limitation but a result of the standard "Response-Only" SFT convention.
Full-Sequence Masking Strategy: The authors introduce FS masking as the key ingredient to unlock bidirectional prompt infilling capabilities.
Prompt Infilling Protocol: They establish a protocol where models generate task-adapted prompts (e.g., scoring rubrics, instructions) from few-shot examples, eliminating the need for manual prompt engineering.
Complementarity: The study shows that infilled prompts are complementary to existing automated prompt optimization methods (like COPRO and GEPA), yielding further gains when combined.

4. Experimental Results

The authors evaluated their approach on GSM8K (math reasoning), SummEval (summarization evaluation), and BigGen-Bench (LLM-as-a-Judge).

Baseline Failure: Public checkpoints trained with Response-Only Masking (RO) failed at prompt infilling.
- On GSM8K, infilling dropped performance from 73.1% (ICL) to 64.8% for LLaDA and from 79.3% to 54.3% for Dream.
- On SummEval, public checkpoints achieved near-zero correlation with human judges when infilling prompts.
Success with Full-Sequence Masking:
- SummEval: The FS+RO model achieved a Spearman correlation of 0.535 with human judgments using infilled prompts, significantly outperforming the public baseline (0.087) and the RO-only model (0.465).
- BigGen-Bench: The FS model achieved the highest performance (0.548 Spearman) with the lowest variance, proving robust generalization.
- Perplexity Analysis: Models trained with FS showed significantly lower diffusion perplexity on infilled prompts, confirming better fit to the data distribution.
Prompt Compression & Transfer:
- Infilled prompts achieved higher accuracy on GSM8K (76.4%) than 16-shot In-Context Learning (73.1%) while using ~90% fewer tokens (315 vs. 3333 tokens).
- Prompts infilled by a strong FS+RO model successfully transferred to weaker models (RO and Public checkpoints), significantly boosting their performance.
Synergy with Optimization: When combined with automated optimizers (COPRO, GEPA), infilling further improved performance and reduced variance, particularly for COPRO.

5. Significance and Implications

Training Paradigm Shift: The paper argues that many capabilities assumed to require architectural changes (like bidirectional generation) are actually blocked by suboptimal training practices. Exposure to the correct training signal (full-sequence masking) is sufficient to unlock these features.
Automated Prompt Engineering: The method enables models to "self-generate" high-quality, task-specific prompts (e.g., dynamic scoring rubrics with non-integer scores) directly from data, reducing reliance on manual engineering or expensive external LLMs for prompt optimization.
Efficiency: By distilling reasoning patterns into compact, infilled prompts, the approach offers a more token-efficient alternative to standard In-Context Learning (ICL).
Broader Impact: This finding suggests that the "bottleneck" for many diffusion model capabilities is the training data masking strategy, not the model architecture itself.

In conclusion, the paper successfully demonstrates that Full-Sequence Masking during SFT is the critical missing link to enabling Diffusion Language Models to perform effective prompt infilling, leading to state-of-the-art performance in LLM-as-a-Judge tasks and efficient prompt compression.