💻 computer science

DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization

The DOLLAR framework introduces a novel few-step video generation method that combines variational and consistency distillation with a memory-efficient latent reward optimization approach, achieving state-of-the-art quality and diversity in 10-second videos while accelerating sampling speeds by up to 278.6 times.

Original authors: Zihan Ding, Chi Jin, Difan Liu, Haitian Zheng, Krishna Kumar Singh, Qiang Zhang, Yan Kang, Zhe Lin, Yuchen Liu

Published 2026-05-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zihan Ding, Chi Jin, Difan Liu, Haitian Zheng, Krishna Kumar Singh, Qiang Zhang, Yan Kang, Zhe Lin, Yuchen Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a master chef (the Teacher Model) who can cook a perfect, complex 10-second video dish. The problem is, this chef is incredibly slow. To get the dish right, they taste, adjust, taste again, and adjust 50 times before serving it. If you want a video every day, this process takes forever and uses up all your kitchen's electricity (computing power).

The paper introduces DOLLAR, a new method to train a Student Chef who can cook the exact same delicious dish in just 4 steps (or even 1 step), without losing any flavor or quality.

Here is how they did it, using three simple tricks:

1. The "Taste-Test" and the "Consistency" Drill

Usually, when you try to teach a student to cook fast, you run into two problems:

Problem A (The "Bland" Student): If you just tell the student to copy the master's final dish, they might get the taste right but start making the exact same dish every single time (no variety).
Problem B (The "Messy" Student): If you tell them to be creative and fast, the food might look good but taste terrible or inconsistent.

The authors solved this by combining two training methods:

Variational Score Distillation (VSD): This is like the student tasting the master's dish and trying to match the flavor profile perfectly. It ensures the video looks high-quality.
Consistency Distillation (CD): This is like a drill where the student practices making the dish in a straight line. It ensures that no matter how they start cooking, they end up with a consistent result. This keeps the videos diverse and prevents them from becoming "bland" copies.

By mixing these two, the student learns to be both high-quality and diverse, but much faster.

2. The "Secret Sauce" (Latent Reward Optimization)

Even with a fast student, sometimes the video isn't quite what you want. Maybe you want it to look more "cinematic" or have better lighting. Usually, to fix this, you'd have to send the video to a giant, slow "Food Critic" (a Reward Model) who checks every single pixel. This is slow and requires a massive kitchen (computer memory).

The authors invented a Latent Reward Model (LRM).

The Analogy: Instead of sending the finished, heavy video to the Critic, they teach a tiny, pocket-sized "Mini-Critic" to judge the ingredients (the latent space) before the video is even fully cooked.
The Benefit: This Mini-Critic is tiny, fast, and doesn't need to see the whole video to give feedback. It tells the student chef, "Your lighting is a bit off," and the student adjusts immediately. This allows the student to improve beyond the Master Chef's original skills, specifically for things like aesthetics or text alignment, without needing a supercomputer.

3. The Result: A Super-Fast Kitchen

The paper claims that with this method:

Speed: They can generate a 10-second video in 4 steps instead of 50. This is up to 278 times faster than the original method. It's almost like real-time generation.
Quality: The student videos score higher on standard tests (called VBench) than the original Master Chef, as well as other top competitors like Gen-3 and Kling.
Efficiency: Because they use the "Mini-Critic" (Latent Reward Model) instead of the giant one, they save a huge amount of computer memory. You don't need a supercomputer to run this; it fits on standard high-end GPUs.

Summary

Think of DOLLAR as a training program that takes a slow, perfectionist video generator and turns it into a lightning-fast artist. It does this by:

Teaching the artist to be both accurate and creative at the same time.
Giving them a tiny, smart assistant to provide instant feedback on quality, so they don't need to wait for a slow, giant computer to check their work.

The result is a system that creates high-quality, diverse videos in seconds rather than minutes, making "real-time" video generation a reality.

Technical Summary: DOLLAR (Distillation and Latent Reward Optimization)

Problem Statement

Diffusion probabilistic models have achieved significant breakthroughs in text-to-video (T2V) generation; however, their practical application is hindered by computational inefficiency. Standard diffusion models require hundreds of iterative sampling steps to generate high-quality videos, making real-time or near real-time generation infeasible. While reducing the number of sampling steps (few-step generation) improves efficiency, it typically results in a trade-off: a significant degradation in video quality, loss of diversity (mode collapse), or a failure to align with specific user preferences. Furthermore, existing distillation methods often struggle to surpass the performance of the teacher model or adapt to specific downstream requirements without costly re-training.

Methodology

The authors propose DOLLAR, a framework that combines distillation and latent reward optimization to achieve high-quality, diverse, few-step video generation. The methodology consists of three core components:

1. Hybrid Distillation Strategy (VSD + CD)

To address the limitations of existing distillation methods, DOLLAR integrates Variational Score Distillation (VSD) and Consistency Distillation (CD):

Variational Score Distillation (VSD): Aligns the student's sample distribution with the teacher's by minimizing the Kullback-Leibler (KL) divergence. While effective for quality, VSD alone can lead to mode collapse (reduced diversity).
Consistency Distillation (CD): Ensures consistent sample predictions along the diffusion trajectory, promoting diversity. However, CD alone can lower sample fidelity and produce overly smooth outputs.
Integration: The authors combine these losses to leverage the high fidelity of VSD and the diversity of CD. They employ a generalized CD approach using a multi-step teacher denoising function ( $Denoise_m$ ) rather than a single-step approximation, which improves generation quality. The student model is initialized from the teacher and trained using a conjugate velocity prediction objective.

2. Latent Reward Model (LRM) Fine-Tuning

To further enhance performance beyond the teacher's capabilities and align with specific metrics (e.g., aesthetics, text alignment), the authors introduce a Latent Reward Model (LRM):

Mechanism: Instead of backpropagating gradients through large, pixel-space reward models and decoders (which is memory-intensive and requires differentiable rewards), DOLLAR trains a compact, differentiable reward model directly in the latent space.
Training: The LRM is trained to approximate the scores of a pre-trained pixel-space reward model (e.g., HPSv2, PickScore) using generated latent samples.
Optimization: The diffusion student model is fine-tuned using the gradients from the LRM. This approach supports non-differentiable reward metrics, significantly reduces memory usage (by bypassing the decoder and large reward models), and allows for efficient optimization.

3. Multi-Objective Training

The final training objective combines the distillation losses and the reward fine-tuning loss:
$\mathcal{L}(\theta) = \mathcal{L}_{VSD}(\theta) + \beta_{CD}\mathcal{L}_{CD}(\theta) + \beta_{FT}\mathcal{L}_{FT}(\theta; \phi)$
Where $\mathcal{L}_{FT}$ is the negative expected reward from the LRM. This allows the model to simultaneously optimize for distribution matching, consistency, and specific reward metrics.

Key Contributions

Efficient Few-Step Distillation: The introduction of a distillation method combining VSD and CD losses, enabling the generation of high-quality, diverse 10-second videos (128 frames) in as few as 4 steps.
Generalized Consistency Distillation: An improvement over standard CD by utilizing a multi-step teacher denoising function, which enhances the effectiveness of the distillation process.
Latent Reward Optimization: A novel, memory-efficient fine-tuning approach using a compact latent reward model. This method removes the requirement for differentiable reward models and eliminates the need to backpropagate through large pixel-space models and decoders, making reward-based tuning feasible for large video models.

Experimental Results

The method was evaluated on large-scale T2V settings (128 frames, 12 FPS, 192x320 resolution) using the VBench benchmark and human evaluations.

Performance vs. Baselines: The distilled 4-step student model achieved a VBench Total Score of 82.57 (using HPSv2 reward), surpassing the teacher model (80.25) and state-of-the-art baselines including Gen-3 (82.32), Kling (81.85), and T2V-Turbo (81.01).
Quality and Diversity: The student model outperformed the teacher in 9 out of 16 VBench metrics. The combination of CD and LRM successfully mitigated the mode collapse often seen in pure VSD distillation, maintaining high sample diversity (measured by Vendi score).
Efficiency: The 4-step student model achieved a 15.6x acceleration compared to the 50-step teacher model. A 1-step configuration achieved up to 278.6x acceleration, enabling near real-time generation.
Human Evaluation: In pairwise comparisons, the DOLLAR model (4-step) was preferred over the teacher model (50-step DDIM) by 51.1% of human evaluators for general preference and showed significantly higher visual quality scores.
Reward Tuning: Fine-tuning with LRM successfully improved specific metrics (e.g., text-video alignment, lighting) without the prohibitive memory costs associated with direct reward gradient methods like ReFL or DRaFT.

Significance and Claims

The paper claims that DOLLAR significantly advances the state of the art in video generation by resolving the trade-off between generation efficiency and quality. By combining distillation with latent reward optimization, the method enables:

Real-time Potential: Accelerating diffusion sampling by up to 278x, paving the way for real-time video generation applications.
Superiority over Teachers: Demonstrating that a distilled student model can exceed the performance of its teacher model, challenging the notion that student performance is strictly upper-bounded by the teacher.
Scalable Fine-Tuning: Providing a practical solution for aligning video generation models with diverse, non-differentiable reward metrics without the computational burden of pixel-space backpropagation.

The authors note that while the method achieves substantial improvements, challenges remain regarding prompt length bias (models perform better on long, descriptive prompts) and the potential for reward over-optimization, which can degrade overall visual quality if specific metrics are pursued too aggressively.