Technical Summary: FlowConsist

Problem Statement

Fast flow models, including Consistency Models and MeanFlow, aim to accelerate generative sampling by learning to directly predict Ordinary Differential Equation (ODE) path integrals, enabling one-step or few-step generation. However, the authors argue that current training paradigms suffer from two fundamental issues that degrade performance:

1. Systematic Trajectory Drift: Current methods construct conditional velocities by randomly pairing noise ($\epsilon$) and data ($x$) samples. While the expectation of these conditional velocities equals the marginal velocity, individual conditional paths intersect multiple distinct marginal trajectories. Training on these conditional velocities causes the optimization target to drift away from the true, singular marginal ODE path, leading to inconsistent trajectories.
2. Error Accumulation: Approximation errors introduced by the model accumulate over large time spans. In fast flow models that rely on self-predicted velocities for training, these errors compound, causing significant deviations between the generated samples and the ground-truth data distribution, particularly in single-step (1-NFE) generation.

Methodology

The authors propose FlowConsist, a training framework designed to enforce trajectory consistency and correct error accumulation without requiring a pre-trained teacher model.

1. Enforcing Trajectory Consistency (Replacing Conditional Velocities)

The core theoretical insight is that relying on conditional velocities ($v_t = \epsilon - x$) introduces a variance term ($L_{var}$) in the loss function, which forces the model to suppress gradients in high-variance directions and prevents faithful fitting of curved trajectories.

• Theoretical Shift: Instead of using the randomly paired conditional velocity as a surrogate for the marginal velocity, FlowConsist replaces the conditional velocity in the consistency objective with the marginal velocity predicted by the model itself ($u_\theta$).
• Implementation: The authors derive a new training objective (Eq. 7) that combines a standard Flow Matching term with a trajectory consistency regularization term.
  • The objective minimizes the difference between the predicted average velocity and a target derived from the model's own marginal velocity prediction.
  • Crucially, the conditional velocity is only used in a term that converges to the marginal velocity in expectation once the model is trained, effectively eliminating the systematic drift during the optimization process.
• Classifier-Free Guidance (CFG): The method incorporates CFG by treating the guidance scale $\omega$ as a secondary input, allowing the model to learn a unified representation that can be adjusted during inference.

2. Trajectory Rectification (Addressing Error Accumulation)

To address the monotonic divergence caused by cumulative approximation errors over time steps, the authors introduce a trajectory rectification strategy.

• Marginal Velocity Alignment: Unlike previous methods that only correct the initial noise, FlowConsist aligns the marginal distribution of model-generated samples with the real data distribution at every time step along the trajectory.
• Teacher-Free Mechanism: Instead of relying on a pre-trained teacher model (as in DMD) to estimate real marginal velocities, FlowConsist utilizes an auxiliary model ($G_\psi$) trained to estimate the marginal velocity of samples generated by the main model ($F_\theta$).
• Correction Process: The main model is trained to align its predicted marginal velocity with the velocity estimated by the auxiliary model. This effectively "pulls" the model's predictions back toward the real data distribution, rectifying the cumulative errors that would otherwise accumulate in a single-step mapping.

Key Contributions

1. Identification of Trajectory Drift: The paper provides a theoretical proof (Theorem 1 & 2) demonstrating that conditional velocities in Flow Matching inherently possess non-zero covariance, leading to inevitable trajectory drift when used as instantaneous velocity estimates in fast flow models.
2. Novel Training Objective: The authors propose a principled alternative (Eq. 7) that replaces conditional velocities with self-predicted marginal velocities, ensuring the optimization remains anchored to a consistent marginal trajectory.
3. Trajectory Rectification Strategy: A teacher-free method (Eq. 11) is introduced to align marginal distributions at every time step, mitigating the accumulation of approximation errors over long time intervals.
4. State-of-the-Art Performance: The method achieves a new state-of-the-art on ImageNet 256×256 with a Fréchet Inception Distance (FID) of 1.52 using only 1 sampling step (1-NFE), without distillation from a teacher model.

Experimental Results

• Dataset & Setup: Experiments were conducted on ImageNet 256×256 using a DiT architecture and a pre-trained VAE latent space.
• Ablation Studies:
  • Replacing the standard MeanFlow loss with the proposed marginal trajectory consistency loss (Eq. 7) improved FID from 5.86 to 4.99.
  • Adding the marginal velocity alignment loss (Eq. 11) further reduced FID to 4.31.
  • The method demonstrated robustness across different CFG scales and integration ratios of the two loss components.
• Comparison: FlowConsist-XL/2 achieved an FID of 1.52 (1-NFE), outperforming the previous best non-distilled fast flow model (iMF-XL/2 at 1.72) and approaching the performance of distilled methods. It also outperformed multi-step diffusion models in terms of efficiency (1 step vs. 250+ steps) while maintaining competitive quality.

Significance

The paper claims that FlowConsist offers a new perspective for fast flow modeling by addressing the root causes of performance limitations in existing frameworks: the theoretical mismatch between conditional and marginal velocities, and the practical accumulation of approximation errors. By rectifying these issues, the authors demonstrate that high-fidelity, single-step generation is achievable through refined training objectives alone, bridging the gap between teacher-free and teacher-distilled paradigms. The work underscores that trajectory consistency is a core requirement for fast flow models to successfully execute linear mappings between time steps.