Technical Summary: Continuous-Time Distribution Matching for Few-Step Diffusion Distillation

Problem Statement

Diffusion and flow-matching models have revolutionized text-to-image generation but suffer from a severe computational bottleneck due to their iterative sampling processes, often requiring tens to hundreds of network evaluations. While diffusion distillation paradigms (e.g., Consistency Distillation, Distribution Matching Distillation) have emerged to accelerate this process, existing Distribution Matching Distillation (DMD) methods face structural limitations. Specifically, vanilla DMD relies on sparse supervision at a few predefined discrete timesteps to maintain strict alignment between training and inference schedules. This restricted discrete-time formulation, combined with the mode-seeking nature of the reverse KL divergence, often leads to visual artifacts, over-smoothed outputs, and a loss of fine-grained details. Consequently, current state-of-the-art DMD methods frequently require complex auxiliary modules, such as Generative Adversarial Networks (GANs) or reward models, to restore visual fidelity.

Methodology: Continuous-Time Distribution Matching (CDM)

The authors propose Continuous-Time Distribution Matching (CDM), a framework that migrates the DMD paradigm from discrete anchoring to continuous optimization. The method consists of two primary continuous-time designs:

1. Dynamic Continuous Time Schedule

Unlike traditional DMD, which restricts backward simulation to a fixed set of discrete anchors matching the inference schedule, CDM employs a dynamic continuous schedule.

• Mechanism: At each training iteration, the simulation length $N$ is randomly sampled from a uniform distribution $U\{1, N_{max}\}$. The timesteps $t_1, \dots, t_N$ are generated as a strictly decreasing continuous sequence within $(0, 1]$.
• Effect: This decouples the training optimization from the fixed inference schedule. The student model is trained over the full continuous time space rather than a few fixed points, receiving gradient signals from a wider range of trajectories. Theoretical analysis suggests this encourages the student to learn a smoother velocity field across the continuous domain, rather than overfitting to sparse discrete anchors.

2. Off-Trajectory Alignment via Velocity-Driven Extrapolation

To address the numerical truncation errors inherent in few-step generation (where large integration steps cause trajectories to drift off the ideal manifold), CDM introduces a novel CDM loss ($L_{CDM}$).

• Mechanism: Instead of restricting supervision to on-trajectory latents, the method performs active off-trajectory matching. It takes a first-order Euler step along the student's predicted velocity field to extrapolate an off-trajectory latent $x'_{t_i} = x_{t_i} + (t'_i - t_i)v_{t_i}$.
• Objective: The loss enforces distribution matching on these extrapolated points. By anchoring the target to the local clean data estimate at the extrapolated point, the method creates a self-consistency constraint for the student's vector field. This effectively penalizes invalid velocity predictions outside the main trajectory, mitigating inter-anchor inconsistency and allowing the student to self-correct integration errors.

Training Objective

The full training objective unifies three components:
$$L = L_{CA} + L_{DM} + L_{CDM}$$

• $L_{CA}$ (CFG Augmentation): Ensures text-image alignment.
• $L_{DM}$ (Distribution Matching): Aligns the student's marginal distribution with the teacher's CFG-free distribution.
• $L_{CDM}$ (Continuous-Time Matching): Regularizes the velocity field across the continuous time domain via off-trajectory supervision.

Key Contributions

1. Empirical Insights: The authors reveal that anchoring training to fixed discrete timesteps is unnecessary and that the Distribution Matching (DM) loss is not merely a regularizer but the primary driver aligning the student to the teacher's CFG-free distribution.
2. CDM Framework: The proposal of a unified framework that combines dynamic continuous scheduling with a velocity-driven off-trajectory alignment objective. This shifts optimization to the continuous-time domain without relying on adversarial training or external reward models.
3. State-of-the-Art Performance: Extensive experiments demonstrate that CDM achieves superior visual fidelity and semantic adherence in few-step generation (4 NFE) across different architectures (SD3-Medium and Longcat-Image), outperforming existing baselines like DMD2, D-DMD, and Hyper-SD.

Experimental Results

The paper evaluates CDM on SD3-Medium and Longcat-Image at 1024×1024 resolution with 4 Network Function Evaluations (NFE).

• Quantitative Performance: On SD3-Medium, CDM achieves the best scores in Aesthetic (6.075), DPGBench (85.26), PickScore (21.95), and HPSv3 (9.561), outperforming the strong baseline D-DMD. Notably, the 4-NFE CDM student matches or surpasses the 100-NFE teacher on several metrics (e.g., DPGBench and HPSv3).
• Qualitative Improvements: Visual comparisons show CDM produces sharper textures, richer fine-grained details, and better adherence to complex prompts compared to baselines, which often suffer from over-smoothing.
• Ablation Studies:
  • Removing $L_{CA}$ leads to structural collapse.
  • Removing $L_{DM}$ or $L_{CDM}$ individually degrades prompt adherence or visual fidelity, respectively.
  • Reverting to a fixed time schedule or using Gaussian noise perturbation instead of velocity-driven extrapolation results in lower performance, validating the necessity of the continuous and velocity-based designs.

Significance and Claims

The paper claims that CDM is the first work to migrate the DMD distillation framework from discrete schedules to a continuous optimization space. Its significance lies in demonstrating that high-fidelity few-step generation can be achieved without relying on complex auxiliary objectives (such as GANs or reward models) that are typically required to fix the artifacts of discrete-time distillation.

By explicitly simulating and correcting truncation drifts back to the target data manifold through continuous supervision, CDM establishes new state-of-the-art results for 4-step image generation. The authors position this work as a step toward more accessible diffusion distillation, enabling high-quality generation on commodity hardware by reducing inference costs by an order of magnitude while maintaining visual quality.