🤖 machine learning

BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation

BARD is a novel framework that efficiently converts pretrained autoregressive vision-language models into large-block diffusion models with up to 3x faster decoding throughput by employing progressive block merging and stage-wise intra-diffusion distillation, thereby achieving state-of-the-art performance while avoiding the quality degradation typically associated with direct autoregressive-to-diffusion conversion.

Original authors: Baoyou Chen, Hanchen Xia, Peng Tu, Haojun Shi, Shan Mu, Weihao Yuan, Siyu Zhu

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Baoyou Chen, Hanchen Xia, Peng Tu, Haojun Shi, Shan Mu, Weihao Yuan, Siyu Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, hardworking librarian (the Autoregressive Model) who is incredibly good at reading books and answering questions about them. However, this librarian has one major flaw: they can only write one word at a time, and they must wait for the previous word to be finished before starting the next. If you ask them to write a long essay, it takes them forever because they are working in a strict, linear line.

Now, imagine a new type of writer (the Diffusion Model) who works differently. Instead of writing word-by-word, this writer starts with a blank page full of scribbles and "noise." They look at the whole page at once and fix many words simultaneously, refining the text in big chunks. This is much faster, but there's a catch: if you try to turn your brilliant librarian into this new type of writer overnight, they get confused. They lose their ability to reason, their answers become nonsense, and they forget how to read the pictures in the books.

BARD is the solution to this problem. It's a clever "bridge" that helps the slow, word-by-word librarian transform into the fast, chunk-by-chunk writer without losing their intelligence.

Here is how BARD does it, using three simple metaphors:

1. The "Staircase" Instead of a "Cliff" (Progressive Block Merging)

If you tried to jump from writing one word at a time to writing 32 words at once, you would likely fall. That's what happens when you try to convert these models directly; the jump is too big, and they crash.

BARD builds a staircase.

Step 1: It teaches the librarian to write in tiny groups of 4 words at a time. They get used to this new rhythm.
Step 2: Once they are comfortable, it asks them to write in groups of 8.
Step 3: Then 16, and finally 32.

By slowly increasing the size of the "chunks" they write, the model adapts smoothly. It never has to make a scary, giant leap. It just takes one small step up the stairs at a time.

2. The "Master Coach" vs. The "Confused Teacher" (Stage-Wise Distillation)

Usually, when you train a new student (the big chunk model), you might try to teach them using the original librarian (the word-by-word model) as the teacher. But this doesn't work well because they speak different languages! The librarian talks about "what comes next," while the new student talks about "fixing what's already there." It's like trying to teach a swimmer using a book on how to run; the advice doesn't match the action.

BARD solves this by creating a special "Master Coach."

First, it trains a small, stable version of the new writer (the "Anchor") who is already good at the chunk-by-chunk style.
Then, as the student grows bigger (writing 16 or 32 words at once), this same Master Coach stays on the sidelines to guide them.
Because the Coach speaks the same "chunk language" as the student, the guidance is perfect. This ensures that as the student gets faster, they don't lose their smarts.

3. The "Red Pen" Practice (Mixed Noise Scheduler)

Standard training for these fast writers only asks them to fix words that are completely hidden (masked). It's like a teacher covering a word with a black box and asking, "What goes here?"

BARD adds a twist: it also asks the model to fix words that are visible but wrong.

Imagine a sentence where the word "cat" is clearly written, but it should be "dog."
BARD trains the model to spot that error and correct it, even though the word is already there.
This teaches the model to be a better editor. It learns not just to fill in blanks, but to revise and improve its own work, making the final answer much more accurate.

The Result: Speed Without Sacrifice

The result of the BARD framework is a model that is 3 times faster than the original librarian because it writes in big chunks, but it is just as smart (or even smarter) because it was trained carefully using the staircase and the Master Coach.

In the paper's tests, these new models (BARD-VL) beat all other "fast" models and even outperformed the original "slow" models on many tasks like reading charts, understanding documents, and solving visual puzzles.

In short: BARD is the training program that teaches a slow, careful thinker how to become a fast, parallel thinker without making them forget how to think at all.

1. Problem Statement

Autoregressive Vision-Language Models (VLMs) are currently the dominant paradigm for multimodal understanding, offering strong reasoning and generation capabilities. However, their token-by-token causal decoding is inherently sequential, creating a fundamental inference bottleneck that limits throughput and latency.

Diffusion-based VLMs (dVLMs) offer a promising alternative by enabling parallel decoding (refining blocks of tokens simultaneously). However, existing attempts to convert strong pretrained autoregressive VLMs into large-block diffusion models face two major challenges:

Quality Degradation: Directly converting an autoregressive model to a large-block diffusion model often leads to significant performance drops due to the abrupt shift in decoding patterns and optimization difficulty.
Misaligned Distillation: Standard knowledge distillation from an autoregressive teacher to a diffusion student is ineffective because the teacher predicts "next tokens" under clean causal states, while the student performs "same-position denoising" under corrupted states. This mismatch leads to poor alignment.

2. Methodology: The BARD Framework

The authors propose BARD, a bridging framework that converts a pretrained autoregressive VLM into a same-architecture, decoding-efficient large-block dVLM without sacrificing multimodal capability. The framework consists of three core components:

A. Progressive Supervised Block Merging

Instead of jumping directly from causal decoding (block size 1) to a large block size (e.g., 32), BARD employs a staged approach:

Anchor Construction: The pretrained autoregressive model is first converted into a stable small-block diffusion anchor (e.g., block size $B=4$ ).
Progressive Expansion: The block size is gradually doubled in stages ( $4 \to 8 \to 16 \to 32$ ). At each stage, adjacent blocks from the previous stage are merged.
Benefit: This smooth transition allows the model to adapt to larger parallelism incrementally, preserving the token prediction and reasoning patterns learned at smaller scales.

B. Stage-Wise dVLM Distillation

To recover performance lost when increasing block sizes, BARD introduces a specific distillation strategy:

Fixed Anchor Teacher: Unlike standard approaches that use the original autoregressive model as a teacher, BARD uses the fixed small-block diffusion anchor ( $\theta^{(1)}$ ) as the teacher for all subsequent stages.
Rationale: The anchor is already a strong diffusion model operating in the same regime (corrupted states, same-position denoising) as the student. This ensures the logits are directly comparable, avoiding the KL mismatch inherent in AR-to-diffusion distillation.
Process: After training a student model at a larger block size ( $B_k$ ), it is distilled from the fixed anchor to recover capability.

C. Practical Training Improvements

Mixed-Noise Scheduler: Standard masked diffusion only trains the model to fill in masked tokens. BARD introduces visible-token corruption (randomly replacing visible tokens with other tokens) alongside masking. This trains the model to revise incorrect visible tokens during the denoising process, improving robustness and iterative correction.
Memory-Friendly Packed Training: To handle the high memory cost of multimodal sequences, BARD uses a packed sequence layout $[Q, x_1, x_t]$ , where the multimodal context $Q$ is shared between the clean response ( $x_1$ ) and the noisy response ( $x_t$ ). This eliminates redundant context duplication, significantly reducing memory overhead and allowing longer context windows.

3. Key Contributions

BARD Framework: A novel, simple, and effective recipe for bridging autoregressive VLMs to large-block diffusion VLMs, solving the stability and quality gap issues.
Discovery of Distillation Mismatch: The paper empirically demonstrates that direct AR-to-diffusion distillation is poorly aligned and can hurt performance, whereas diffusion-to-diffusion distillation (using a small-block anchor) is consistently effective.
Efficiency vs. Quality Trade-off: The method achieves up to 3× decoding throughput speedup compared to the source autoregressive model while maintaining or improving multimodal capabilities.
Data Efficiency: The framework is highly data-efficient, requiring only $\le 4.4$ million training samples to transfer capabilities from a frontier model (Qwen3-VL) to a dVLM.

4. Experimental Results

The authors trained BARD-VL models at 2B, 4B, and 8B scales, converting from Qwen3-VL.

Performance vs. Autoregressive Models:
- 4B Scale: Outperformed Qwen3-VL-4B on 5/7 benchmarks (e.g., +5.1 on MMMUval, +8.0 on MMEsum).
- 8B Scale: Outperformed Qwen3-VL-8B on 6/7 benchmarks (e.g., +14.0 on MMEsum).
- The models retained the strong reasoning capabilities of the source model while gaining parallel decoding.
Performance vs. Existing Diffusion VLMs:
- BARD-VL established a new State-of-the-Art (SOTA) among open-source dVLMs of comparable scale.
- BARD-VL 8B outperformed LLaDA-V-8B on all 7 benchmarks.
- BARD-VL 4B surpassed Dimple-VL and Dream-VL on the majority of benchmarks.
Ablation Studies:
- Block Merging: Direct adaptation to large blocks ( $B=32$ ) caused significant degradation; progressive merging recovered most of this loss.
- Distillation: Diffusion distillation from the anchor was significantly superior to AR distillation, especially at larger block sizes (e.g., at $B=32$ , diffusion distillation improved MMStar by +7.7 over no distillation, while AR distillation hurt performance).
- Noise Scheduler: The mixed-noise approach provided the best balance between reasoning stability and the ability to revise errors.

5. Significance

This work represents a critical step toward making diffusion models a viable, high-performance alternative to autoregressive models for multimodal tasks.

Bridging the Gap: It effectively narrows the capability gap between diffusion and autoregressive VLMs, proving that diffusion models can match or exceed the performance of their autoregressive counterparts.
Scalability: The proposed "progressive block merging + stage-wise distillation" recipe offers a scalable path to training highly parallel multimodal models without the instability of one-step conversion.
Practical Deployment: By achieving 3× throughput speedup with minimal data requirements, BARD makes efficient, real-time multimodal generation more feasible for practical applications like document understanding and complex reasoning agents.