Technical Summary: VDE – Training-Free Accelerating Rectified Flow Model via Velocity Decomposition and Estimation

1. Problem Statement

Rectified Flow (RF) models have achieved state-of-the-art performance in image, video, and 3D generation. However, their practical deployment is hindered by high inference latency, as they typically require dozens of iterative sampling steps.

Existing training-free acceleration methods primarily rely on a "cache-and-reuse" paradigm. These approaches store intermediate features (e.g., attention outputs, residuals) from previous steps and reuse them to skip redundant computations. The authors identify a fundamental flaw in this strategy: cache-input mismatch. Because the cached features are static while the model's input evolves dynamically during the sampling trajectory, reusing stale features leads to an output-input mismatch. This mismatch propagates errors, resulting in degraded visual fidelity, such as softened textures and structural inconsistencies.

2. Methodology: Velocity Decomposition and Estimation (VDE)

The paper proposes Velocity Decomposition and Estimation (VDE), a training-free method that shifts the acceleration paradigm from "caching-and-reusing" to "decomposing-and-estimating." Instead of reusing static features, VDE constructs a lightweight estimation function to predict the model's velocity directly from the current input.

Core Mechanism

The method is built upon the observation that the velocity vector $v_t$ predicted by the model exhibits robust temporal regularities when decomposed relative to the current latent input $x_t$.

1. Velocity Decomposition:
   The velocity $v_t$ is uniquely decomposed into two components:

   • A parallel component aligned with the input $x_t$.
   • An orthogonal component perpendicular to $x_t$.

   Mathematically, this is expressed as:
   $$v_t = \alpha_t x_t + \beta_t \|x_t\| u_t$$
   Where:

   • $\alpha_t$ and $\beta_t$ are scalar coefficients.
   • $u_t$ is a unit vector representing the orthogonal direction ($u_t \perp x_t$).

2. Temporal Regularities (Empirical Findings):
   Through extensive analysis, the authors observe two stable patterns during the "stable phase" of sampling (after an initial warm-up):

   • Predictable Coefficients: The scalar coefficients $\alpha_t$ and $\beta_t$ evolve smoothly with strong local linearity between consecutive steps. This allows them to be reliably estimated via linear extrapolation from recent historical values.
   • Stable Orthogonal Direction: The unit vector $u_t$ remains nearly constant over short intervals (cosine similarity $> 0.99$). This permits the reuse of the most recent orthogonal direction without significant error.

3. The VDE Algorithm:
   VDE operates in an anchor-and-estimate mode:

   • Anchor Steps: At periodic intervals (e.g., every $n$ steps), a full model forward pass is executed. The velocity is decomposed to obtain the ground-truth tuple $(\alpha_t, \beta_t, u_t)$.
   • Estimation Steps: For steps between anchors, the velocity is estimated without model computation:
     • Coefficients ($\hat{\alpha}_t, \hat{\beta}_t$): Estimated via linear extrapolation based on the coefficients from the two most recent anchor steps.
     • Direction ($\hat{u}_t$): Reused from the most recent anchor step.
     • Synthesis: The estimated velocity $\hat{v}_t$ is reconstructed analytically using the current input $x_t$ and the estimated components:
       $$\hat{v}_t = \hat{\alpha}_t x_t + \hat{\beta}_t \|x_t\| \hat{u}_t$$

This approach is inherently input-adaptive, as the estimated output is freshly constructed from the current input $x_t$, eliminating the reliance on stale features.

3. Key Contributions

• Problem Identification: The paper identifies the "cache-input mismatch" as the root cause of fidelity loss in existing caching-based acceleration methods and reframes training-free acceleration as an input-output mapping estimation problem.
• Novel Paradigm: It introduces VDE, which replaces static feature reuse with dynamic decomposition and estimation. This fundamentally eliminates the output-input mismatch caused by stale caches.
• Empirical Discovery: The authors demonstrate that decomposing velocity into parallel and orthogonal components reveals strong temporal regularities (local linearity of coefficients and stability of direction) that enable precise, low-cost estimation.
• Performance: Extensive experiments show VDE achieves substantial speedups (2.04× to 3.22×) with minimal loss in visual quality, outperforming state-of-the-art baselines like TeaCache and EasyCache.

4. Experimental Results

The authors evaluated VDE on image generation models (FLUX.1 [dev], Qwen-Image) and a video generation model (Wan2.1).

• Image Generation (FLUX.1):
  • VDE-fast achieved a 3.01× speedup (16 NFEs) while improving SSIM from 0.7240 (EasyCache-fast) to 0.8267 and reducing LPIPS from 0.3197 to 0.1997.
  • VDE-slow achieved a 2.21× speedup with nearly identical visual quality to the T=50 baseline, outperforming the best baseline (EasyCache-slow) by 19.5% in SSIM and 30.3% in PSNR.
• Image Generation (Qwen-Image):
  • VDE-slow achieved a 2.04× speedup with an LPIPS of 0.069, representing a 52.2% reduction in error compared to the best baseline.
  • It matched the ImageReward score of the full T=50 baseline (1.295), indicating high perceptual fidelity.
• Video Generation (Wan2.1):
  • VDE achieved a 2.50× speedup (40 NFEs) while maintaining a VBench score of 80.43, comparable to the original model (81.30) and superior to other acceleration methods.
• Qualitative Analysis: Visual comparisons show that while caching methods often exhibit texture degradation and structural inconsistencies, VDE preserves global composition and fine details (shapes, shading) comparable to the T=50 baseline.

5. Significance and Claims

The paper claims that VDE offers a new perspective on accelerating generative models. By shifting from feature reuse to dynamic estimation, it effectively mitigates the cache-input mismatch that plagues current training-free methods.

The authors emphasize that VDE is:

• Training-free: It requires no additional data or optimization cycles.
• Input-Adaptive: It constructs outputs based on the current input state, ensuring high fidelity.
• Scalable: It generalizes well across different resolutions, aspect ratios, and model architectures (DiT-based image and video models).
• Efficient: The decomposition and estimation involve only straightforward arithmetic operations, adding negligible computational overhead compared to the savings from skipped forward passes.

The work concludes that VDE provides a superior speed-fidelity trade-off, enabling the practical deployment of high-quality rectified flow models in real-time or compute-constrained scenarios.