🤖 AI

Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models

This paper proposes a collaborative deployment pipeline for the Wan2.2 video diffusion model that combines few-step distillation with low-bit quantization to significantly reduce computational costs while maintaining or even surpassing the visual quality of the original full-precision baseline.

Original authors: Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, Xianglong Liu

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong, Shiqiao Gu, Yang Yong, Jinyang Guo, Xianglong Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a master chef (the Wan2.2 AI model) who is famous for creating incredibly delicious, high-definition video meals. However, there's a catch: to cook a single meal, this chef needs to take 40 tiny, precise steps, and they require a massive, expensive kitchen (huge computer memory) to do it. This makes it too slow and costly to serve to regular customers.

The paper presents a new recipe to make this chef faster and cheaper to run without ruining the taste of the meal. They do this using two main tricks: teaching the chef to cook faster and packing the ingredients more tightly.

Here is how they did it, explained simply:

1. The "Two-Chef" Kitchen (Dual-Expert Architecture)

First, you need to understand the chef's unique style. This isn't just one chef; it's a team of two specialists working in shifts:

The "Big Picture" Chef (High-Noise Expert): Works at the start of the process. They decide where the objects go, how the camera moves, and the general layout of the scene.
The "Detail" Chef (Low-Noise Expert): Works later. They add the fine textures, lighting, and smooth the motion.

The Problem: Most compression methods treat the kitchen like a single, giant room. But because this kitchen has two distinct shifts, squishing everything together causes confusion. The "Big Picture" chef gets mixed up with the "Detail" chef's tools.

The Solution: The researchers treated the two shifts separately. They calibrated (tuned) the tools for the "Big Picture" chef specifically for that job, and the "Detail" chef's tools specifically for theirs. This ensures that when the "Big Picture" chef is working, they aren't accidentally using the wrong tools meant for the "Detail" phase.

2. The "Speed-Run" Training (Few-Step Distillation)

Normally, the chef takes 40 steps to finish a dish. The researchers wanted to cut this down to just 20 steps.

The Wrong Way: You can't just tell the chef, "Stop after step 20." If you do, the dish will be half-cooked and taste bad.
The Right Way (Distillation): They trained a "student chef" to learn the entire 40-step process but compress it into a 20-step routine. The student learns to skip the boring, repetitive parts and jump straight to the important changes. Now, the student can produce a nearly identical meal in half the time.

3. The "Tight Packing" (Low-Bit Quantization)

Even with the faster student chef, the kitchen is still too big for a small apartment (limited computer memory). They needed to shrink the ingredients.

The Analogy: Imagine the ingredients are measured in huge, heavy buckets (high precision). To save space, they switched to small, lightweight cups (low-bit quantization).
The Risk: If you just swap the buckets for cups without checking, you might lose important flavor (data) or spill the sauce (errors).
The Fix: They didn't just swap the buckets; they re-measured the ingredients while the student chef was cooking the 20-step meal. This ensured the small cups were the perfect size for the specific ingredients the student was actually using.

4. Protecting the "Foundation" (Entrance Layer Protection)

In any construction project, if the foundation is shaky, the whole building falls.

The Strategy: The researchers realized that the very first steps of the cooking process (where the layout is set) are the most sensitive. If you mess up the first step, the rest of the meal is ruined.
The Action: They kept the tools for the very first steps in the "heavy buckets" (high precision) and only switched to the "light cups" for the later steps. This protects the foundation while still saving space on the rest of the kitchen.

5. The Result: A Better Meal, Faster

They tested this new setup using a "Food Critic" (called VBench) that judges videos on five things:

Does the character look the same throughout? (Subject Consistency)
Does it look pretty? (Aesthetic Quality)
Is the image sharp? (Imaging Quality)
Does it match the description? (Overall Consistency)
Is the movement smooth? (Motion Smoothness)

The Findings:

The Sweet Spot: They tested cooking in 4, 8, 20, and 40 steps.
The Winner: The 20-step version was the champion. It was much faster than the original 40-step version, but it actually tasted better (scored higher on the critic's list) than the original full-size chef!
The Surprise: Even with the "light cups" (compressed data), the 20-step compressed model performed just as well as, or slightly better than, the uncompressed 20-step model.

Summary

The paper shows that by training a faster student, packing the data tightly, and protecting the most important first steps, you can run a massive video AI on much smaller, cheaper computers without losing quality. In fact, by finding the right balance (20 steps), they made the AI perform better than the original slow version.

It's like taking a luxury limousine, teaching the driver a shortcut, and swapping the heavy leather seats for lightweight fabric, only to find the car is now faster, cheaper to run, and still rides just as smoothly.

Technical Summary: Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models

1. Problem Statement

Large video diffusion models, such as the Wan2.2-T2V-A14B, achieve high visual quality but face significant deployment barriers due to two primary bottlenecks:

Computational Cost: The iterative denoising process requires a large number of sampling steps, making inference expensive in terms of latency.
Memory Footprint: The models possess a large resident parameter count (approx. 27B total, with ~14B active per step in the dual-expert architecture), exceeding the memory budgets of many deployment scenarios.

Existing compression strategies often fail when applied directly to these models. Simply reducing sampling steps without retraining leads to poor quality, while quantizing the original long-step trajectory often introduces activation-distribution mismatches during inference, causing temporal error propagation and degraded video fidelity. Furthermore, Wan2.2 utilizes a dual-expert Mixture of Experts (MoE) architecture (high-noise and low-noise experts), which complicates standard quantization approaches that treat the model as a dense trunk.

2. Methodology

The paper proposes a deployment-oriented compression pipeline that integrates few-step distillation and low-bit quantization in a co-designed manner. The pipeline follows a specific sequence: model adaptation $\rightarrow$ few-step distillation $\rightarrow$ low-bit quantization $\rightarrow$ calibration construction $\rightarrow$ generation $\rightarrow$ evaluation.

Key Technical Components:

Dual-Expert Aware Architecture Handling:
- The pipeline respects Wan2.2's specific routing mechanism, where a high-noise expert handles global layout/motion and a low-noise expert refines texture/details.
- Branch-wise Calibration: Instead of pooling activations, the method captures and calibrates statistics separately for the high-noise and low-noise branches. This prevents the quantizer from mixing incompatible activation distributions.
- Entrance-Layer Protection: Early layers and routing-sensitive blocks are kept in higher precision or calibrated more aggressively. This is critical because errors in the initial high-noise phase (layout/motion initialization) propagate through the entire sequence, whereas later errors are often local.
Few-Step Distillation via Distribution Matching:
- The method employs Distribution-Matching Distillation (DMD) to learn a shorter denoising trajectory rather than merely truncating the sampler.
- The student model is optimized to match the teacher's distribution across the reduced number of steps, with a stage-aware objective ( $L_{distill} = \lambda_1 L_{high} + \lambda_2 L_{low}$ ) that respects the distinct roles of the high-noise and low-noise phases.
Co-Designed Quantization:
- Trajectory-Aligned Calibration: A core innovation is performing quantization calibration on the distilled few-step student rather than the original long-step model. This ensures the quantizer observes the activation statistics it will encounter during actual inference, reducing distribution mismatch.
- HiF4 (HiFloat4) Representation: The pipeline utilizes a 4-bit block floating-point format (HiF4) designed for low-bit inference. This format features a three-level scaling hierarchy (Global E6M2, Level-2 E1, Level-3 E1) and S1P2 signed amplitude. This design provides a wide global dynamic range (approx. $2^{-50}$ to $2^{18}$ ) to handle heterogeneous activation ranges across denoising phases, while preserving local detail through micro-exponents.
- Granularity: Weights are quantized per-channel, and activations are quantized per-token to track prompt- and frame-level variations.
Engineering and Evaluation:
- Memory Management: To avoid Out-Of-Memory (OOM) errors on single GPUs (e.g., 80GB H100), calibration and replacement are performed layer-by-layer, with auxiliary buffers loaded on demand.
- Closed-Loop Evaluation: The system uses VBench to evaluate five specific dimensions: Subject Consistency (SC), Aesthetic Quality (AQ), Imaging Quality (IQ), Overall Consistency (OC), and Motion Smoothness (MS). The evaluation protocol ensures fair comparison by using identical prompts, resolutions, and sampling schedules across baselines.

3. Key Contributions

Co-Design of Distillation and Quantization: The paper demonstrates that calibrating quantization on the distilled few-step trajectory (rather than the original long-step model) significantly reduces activation-distribution mismatch, allowing the compressed model to maintain quality close to the full-precision student.
MoE-Aware Compression Strategy: The method introduces a branch-wise calibration and entrance-layer protection scheme specifically tailored for the dual-expert structure of Wan2.2, preventing the mixing of high-noise and low-noise activation statistics.
HiF4 Integration for Video Diffusion: The application of HiF4's hierarchical scaling is shown to be effective for video diffusion models, balancing the need for a wide global range (to avoid overflow in high-noise phases) with local precision (to preserve texture in low-noise phases).
Empirical Validation of Step Counts: The study systematically evaluates 4, 8, 20, and 40-step settings, identifying the optimal trade-off point for quality and efficiency.

4. Experimental Results

Experiments were conducted on the Wan2.2-T2V-A14B model using the OpenS2V dataset for calibration and VBench for evaluation.

Performance vs. Baselines:
- At 4 steps, the quantized model maintains a small error relative to the same-step full-precision model but trails the original long-step baseline.
- At 8 steps, the quantized model (Avg Score: 0.7002) begins to exceed the original full-precision baseline (0.6976).
- At 20 steps, the quantized model achieves the best performance (Avg Score: 0.7074), outperforming both the original full-precision baseline and the same-step full-precision distilled model (0.7051).
- At 40 steps, performance plateaus, indicating diminishing returns from further increasing the trajectory length.
Metric Analysis (20-Step Setting):
- The quantized model improved upon the original baseline in Subject Consistency (0.9629 vs 0.9495), Aesthetic Quality, Imaging Quality, and Motion Smoothness.
- Overall Consistency remained comparable to the same-step model.
- The results indicate that the distribution-aligned calibration successfully recovers low-bit losses without sacrificing temporal stability.
Error Sensitivity:
- The analysis reveals that single-frame perturbations primarily affect Imaging and Aesthetic Quality, while accumulated errors across steps manifest as Subject Drift and Motion Jitter. The proposed method's protection of entrance layers effectively mitigates the latter.

5. Significance and Claims

The paper claims that the proposed pipeline offers a practical path for deploying high-quality video diffusion models under limited GPU memory and latency budgets.

Quality-Efficiency Trade-off: The authors identify the 20-step setting as the optimal configuration, providing sufficient denoising capacity to recover temporal and visual quality while requiring significantly fewer steps than conventional samplers.
Strategic Shift: The work argues that quantization should not be a "lossy afterthought" but an integral part of the acceleration strategy. By co-designing distillation and quantization, the method turns low-bit conversion into a stable component of the inference pipeline.
Deployment Readiness: The use of entrance-layer protection and branch-wise calibration addresses the specific failure modes of dual-expert video models (e.g., subject drift and motion instability), making the compressed model suitable for interactive generation and resource-constrained batch processing.

The paper concludes that while the current pipeline focuses on post-training quantization, the framework establishes a verification loop where metric-specific failures (e.g., drops in Subject Consistency) provide direct debugging signals for refining calibration and step counts.