BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching

This paper proposes BWCache, a training-free method that accelerates DiT-based video generation by dynamically caching and reusing block features across diffusion timesteps based on a similarity threshold, achieving up to a 6$\times$ speedup while maintaining visual fidelity.

The Gist

Imagine you are directing a massive, high-tech movie production. The script is your text prompt (e.g., "A cat riding a skateboard at sunset"), and the actors are the pixels on your screen.

In the world of AI video generation, the current stars are called Diffusion Transformers (DiTs). Think of them as incredibly talented but extremely slow actors. To create a single second of video, they have to perform a long, repetitive dance called "denoising." They start with a screen full of static (like TV snow) and slowly, step-by-step, clean it up until a clear image appears.

The Problem:
This dance is sequential. The actor has to do Step 1, then Step 2, then Step 3, all the way to Step 30. It takes a long time, and the computer gets exhausted (high latency), making it hard to use for real-time applications.

The Old Solutions:

• The "Architect" Approach: Some tried to build a smaller, faster stage (pruning or distilling the model). But this often made the actors forget their lines, resulting in blurry or weird videos.
• The "Lazy" Approach: Others tried to skip steps entirely. But if you skip too many, the movie looks like a glitchy mess.

The New Solution: BWCache (Block-Wise Caching)
The authors of this paper, Hanshuai Cui and his team, realized something brilliant while watching these actors rehearse. They noticed that in the middle of the performance, the actors barely move.

The "U-Shape" Discovery

Imagine the actor's energy levels over the course of the 30 steps:

1. Start (The Beginning): The actor is frantic, making huge changes to the static noise. Everything is chaotic.
2. Middle (The Middle): The scene is mostly set. The actor is just making tiny, almost invisible adjustments. They are essentially standing still, holding the pose.
3. End (The Finale): The actor goes into overdrive again to add the final, crisp details (the sparkle in the eye, the texture of the fur).

The authors found that during that Middle phase, the computer is doing a lot of work for almost no change in the picture. It's like a chef chopping vegetables for a soup that has already been simmering for an hour; the chopping is redundant.

How BWCache Works: The "Freeze-Frame" Trick

Instead of forcing the computer to recalculate every single step, BWCache introduces a smart manager with a clipboard.

1. The Check-In: At every step, the manager looks at the "actors" (the internal parts of the AI called DiT blocks).
2. The Similarity Test: The manager asks, "Did the actor move much since the last step?"
   • If YES (Big Change): The manager says, "Keep working! We need to see this." The computer calculates the new frame.
   • If NO (Tiny Change): The manager says, "Hold that pose! You're just repeating yourself." The computer skips the calculation and just re-uses the image from the previous step.
3. The Safety Net: To make sure the video doesn't get "stuck" or blurry from reusing the same image too long, the manager forces a "rehearsal" every few steps to refresh the details.

Why This is a Game-Changer

Think of it like watching a movie on a streaming service.

• Without BWCache: Your internet has to download every single frame, even the ones where the camera is just panning slowly across a static wall.
• With BWCache: The service realizes, "Hey, this wall isn't moving." It sends you the same image data for the next few seconds, saving massive amounts of data and time.

The Results:

• Speed: They made the video generation 2.6 times faster.
• Quality: Because they are smart about when to skip (only when the image is truly stable), the video looks just as good as the slow version. It doesn't look blurry or glitchy.
• No Training Needed: You don't have to teach the AI a new way to act. You just give it this new "manager" (the caching system) to run the show. It works with existing models like Open-Sora, HunyuanVideo, and others.

In a Nutshell

BWCache is like a smart traffic light for AI video generation. It knows exactly when the traffic (computation) is heavy and needs to flow, and when the road is clear enough to let cars (frames) coast without an engine running. It saves time and energy without causing a traffic jam (quality loss).

Technical Summary

1. Problem Statement

Diffusion Transformers (DiTs) have become the state-of-the-art architecture for video generation (e.g., Sora, Open-Sora). However, they suffer from high inference latency due to their inherently sequential denoising process, which requires computing multiple DiT blocks at every timestep.

• Limitations of Existing Methods:
  • Architectural Modifications: Methods like distillation, pruning, or quantization often degrade visual quality and require expensive retraining.
  • Coarse-Grained Caching: Caching at the timestep level (e.g., TeaCache) often loses essential information because feature variations are not uniform across the entire model at a given step.
  • Fine-Grained Caching: Caching at the attention level often fails to provide significant acceleration.
  • Feature Reuse Assumptions: Many existing methods naively assume high similarity between adjacent timesteps, leading to degraded details when features actually change significantly.

The core challenge is identifying which features to cache (granularity) and when to reuse them (triggering mechanism) without compromising visual fidelity or requiring model retraining.

2. Methodology: BWCache

The authors propose BWCache (Block-Wise Caching), a training-free, plug-and-play acceleration method that dynamically caches and reuses features from DiT blocks across diffusion timesteps.

A. Key Observations & Analysis

Through extensive analysis of DiT blocks across various models (Open-Sora, Latte, Wan 2.1, etc.), the authors identified two critical patterns:

1. U-Shaped Feature Variation: The relative $L_1$ distance of block features between adjacent timesteps follows a U-shaped curve.
   • Early Timesteps: High variation (rapid noise prediction changes).
   • Middle Timesteps: Low variation (high redundancy/stability).
   • Late Timesteps: High variation (transition from structured noise to high-fidelity details).
2. Block-Level vs. Timestep-Level: Block-level features exhibit significant variation depending on the prompt and scene dynamics, whereas timestep-level aggregation often masks these differences. This suggests that block-wise caching is superior to timestep-level caching for adaptive control.

B. Core Components

1. Similarity Indicator (Trigger Mechanism):
   Instead of blindly reusing features, BWCache calculates the Aggregated Relative $L_1$ Distance between the current timestep's block features and the cached features from the previous timestep.
   $$\frac{1}{N} \sum_{n=1}^{N} L1_{rel}(h_{t,i}) < \delta$$
   Where $N$ is the number of DiT blocks, $L1_{rel}$ is the relative distance, and $\delta$ is a similarity threshold.

   • If the average difference is below $\delta$, the system skips computation and reuses the cached block features.
   • If the difference exceeds $\delta$, the blocks are recomputed, and the cache is updated.
   • Adaptability: This allows the method to adapt to scene dynamics; static scenes trigger more reuse, while dynamic scenes trigger fewer.

2. Periodic Recomputation (Drift Mitigation):
   To prevent "latent drift" (where continuous reuse causes error accumulation and loss of fine details), BWCache employs a reuse interval ($R$). Even if the similarity indicator suggests reuse, blocks are periodically recomputed (e.g., every 10% of timesteps) to refresh the features.

3. Final Step Protection:
   The authors observed that the final diffusion steps are critical for high-fidelity detail recovery. Therefore, BWCache enforces that the last $k/2$ steps (where $k$ is the step where caching was triggered) are always recomputed, ensuring the final output quality is not compromised.

3. Key Contributions

• Granularity Analysis: Demonstrated that block-level caching is more effective than timestep-level caching for DiTs, as it captures specific feature redundancies that global metrics miss.
• Training-Free Acceleration: BWCache requires no architectural changes or fine-tuning, making it applicable to any pre-trained DiT-based video model.
• Dynamic Triggering: Introduced a lightweight similarity indicator that dynamically decides when to cache based on actual feature variance, balancing speed and quality.
• Comprehensive Evaluation: Validated the method across five diverse state-of-the-art models (Open-Sora, Open-Sora-Plan, Latte, Wan 2.1, HunyuanVideo) and various video resolutions/lengths.

4. Experimental Results

The experiments were conducted on NVIDIA A800 GPUs, comparing BWCache against baselines like PAB, TeaCache, and FasterCache.

• Speedup: BWCache achieves up to 2.6× speedup in inference latency compared to the original models.
  • Example: On HunyuanVideo, latency dropped from 1122s to 433s (2.60×).
  • Example: On Wan 2.1, latency dropped from 912s to 457s (2.00×).
• Visual Quality: Unlike many acceleration methods that degrade quality, BWCache maintains or slightly improves visual metrics (VBench, LPIPS, SSIM, PSNR).
  • On Open-Sora, BWCache achieved a 1.61× speedup while maintaining a VBench score of 80.03% (vs. 80.33% for the original), significantly outperforming TeaCache (79.16%) and PAB (78.10%).
• Scalability: The method scales effectively across multiple GPUs (up to 8 GPUs) using Dynamic Sequence Parallelism (DSP), achieving up to 17.2× speedup on long, high-resolution videos.
• Memory Efficiency: While caching adds memory overhead, BWCache is more memory-efficient than high-memory methods like ProfilingDiT and TaylorSeer, which often fail (OOM) on long videos.

5. Significance

BWCache addresses a critical bottleneck in the deployment of generative video AI: latency. By leveraging the inherent redundancy in DiT block features during the middle stages of diffusion, it offers a practical, training-free solution that significantly reduces inference time without sacrificing the high visual fidelity required for real-world applications. The method's ability to adapt to different scene dynamics (static vs. dynamic) via a simple similarity threshold makes it a robust and generalizable acceleration technique for the next generation of video generation models.