🤖 AI

Motion-Aware Caching for Efficient Autoregressive Video Generation

The paper proposes MotionCache, a motion-aware caching framework that dynamically adjusts denoising step frequencies based on pixel-level motion characteristics to significantly accelerate autoregressive video generation while maintaining high visual quality.

Original authors: Jing Xu, Yuexiao Ma, Songwei Liu, Xuzhe Zheng, Shiwei Liu, Chenqian Yan, Xiawu Zheng, Rongrong Ji, Fei Chao, Xing Wang

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jing Xu, Yuexiao Ma, Songwei Liu, Xuzhe Zheng, Shiwei Liu, Chenqian Yan, Xiawu Zheng, Rongrong Ji, Fei Chao, Xing Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an artist trying to paint a long, moving movie scene, one frame at a time. In the world of AI video generation, this "painting" process is called denoising. The AI starts with a static cloud of static (noise) and slowly refines it into a clear video.

Doing this for every single pixel in every single frame is incredibly slow and expensive, like trying to repaint the entire canvas from scratch for every tiny movement in the movie.

The Problem: The "All-or-Nothing" Mistake

Previous methods tried to speed this up by using a cache—a shortcut where the AI says, "Hey, this part of the picture didn't change much, so I'll just copy the last version instead of painting it again."

However, these old methods were like a clumsy foreman who only gave two orders:

"Paint the whole room!" (Compute everything).
"Don't paint anything!" (Skip the whole room).

The problem is that in a video, some things move fast (like a girl riding a bike), while other things stay still (like the background trees). The old methods treated the whole scene the same. If they skipped the room, they missed the moving bike. If they painted the whole room, they wasted time repainting the still trees.

The Solution: MotionCache

The paper introduces MotionCache, a smarter system that acts like a traffic controller for the AI's attention. Instead of looking at the whole room, it looks at every single "pixel" (or token) individually.

Here is how it works, using simple analogies:

1. The "Motion Proxy" (The Heat Map)
The AI needs to know what is moving without actually doing the heavy math first. MotionCache uses a clever trick: it looks at the difference between frames.

Analogy: Imagine taking two photos of a room one second apart. If a chair moved, the difference between the photos is huge. If a lamp stayed still, the difference is zero.
MotionCache uses this "difference" as a heat map. Hot spots (high difference) mean "Paint this now!" Cold spots (low difference) mean "Just copy the old paint."

2. The Two-Phase Strategy
The paper proposes a "Coarse-to-Fine" approach, which is like building a house:

Phase 1 (The Foundation): At the very beginning, the AI doesn't know exactly where the bike or the trees will go. It's all blurry. So, MotionCache forces the AI to paint the entire scene for a few steps. This ensures the structure (the skeleton of the video) is solid.
Phase 2 (The Details): Once the structure is set, the AI switches to the smart traffic controller mode. It only paints the moving parts (the bike wheels, the girl's hair) and skips the static parts (the sky, the trees).

3. The "Accumulator" (The Patience Meter)
Sometimes, a static object might need a tiny touch-up eventually. MotionCache keeps a "patience meter" for every pixel.

If a pixel is static, the meter fills up very slowly.
If a pixel is moving, the meter fills up fast.
Once the meter hits a limit, the AI paints that pixel and resets the meter. This ensures that even static backgrounds get a tiny refresh if they start to look "stale," preventing errors from piling up.

The Results

The authors tested this on two powerful video models (SkyReels-V2 and MAGI-1).

Speed: It made the video generation 6 times faster on one model and 2 times faster on the other.
Quality: Unlike the old methods that made videos look blurry or glitchy (like a bike rider suddenly having six fingers), MotionCache kept the video looking sharp and natural. It preserved the details of the moving objects while skipping the boring, still parts.

In a Nutshell

MotionCache is like a smart construction crew that knows exactly which workers to send where. It doesn't waste money painting the walls of a room that hasn't changed, but it makes sure the moving furniture gets painted perfectly. This allows AI to create long, high-quality videos much faster without losing the details that matter.

Technical Summary: Motion-Aware Caching for Efficient Autoregressive Video Generation

Problem Statement

Autoregressive video generation models, which leverage the Causal Diffusion-Forcing (CDF) framework to enable long-duration synthesis, face significant computational bottlenecks. While these models reduce memory complexity from quadratic to linear by processing video in chunks, the iterative denoising process remains prohibitively slow for high-resolution, long-duration videos. Existing acceleration strategies, such as layer-level or step-level feature caching, often rely on coarse-grained skipping (e.g., skipping entire timesteps or chunks). These methods fail to account for fine-grained temporal redundancy and intra-chunk frame discrepancies. Specifically, they treat all tokens within a chunk uniformly, ignoring the fact that static backgrounds tolerate aggressive skipping while high-motion regions require frequent updates to prevent error accumulation and structural degradation.

Methodology: MotionCache

The authors propose MotionCache, a motion-aware caching framework that transitions from coarse-grained chunk skipping to fine-grained, token-level adaptive caching. The methodology is grounded in a theoretical analysis linking caching errors to residual instability.

1. Theoretical Foundation

The paper establishes that the approximation error in feature caching is strictly bounded by the residual instability (the difference between the true residual and the cached residual) across timesteps.

Residual Inconsistency Principle: The error $\epsilon$ is proportional to the magnitude of the vector difference between the true residual $R_{t-1}$ and the cached residual $R_t$ .
Motion Proxy: Since true residuals are inaccessible prior to inference, the authors prove that the intra-chunk frame difference serves as a mathematically grounded upper bound for residual instability. This allows the system to use lightweight frame differences as a proxy for identifying dynamic regions.

2. Core Mechanisms

MotionCache operates via a coarse-to-fine inference schedule:

Phase 1: Coarse-Grained Structure Construction (Warm-up):
- In the initial denoising steps, global semantic structure is unstable. The NDCG scores for token importance are volatile.
- To prevent semantic layout disruption, the model enforces a chunk-wise decision policy. The entire chunk is either fully computed or fully skipped, ensuring structural integrity before fine-grained adaptation begins.
- This phase lasts for $K$ full computations.
Phase 2: Fine-Grained Detail Refinement:
- Once the global structure stabilizes, the system switches to token-wise adaptive caching.
- Motion-Aware Token Importance: An importance map is generated for each token based on the $L_1$ difference between adjacent frames within the chunk. A soft-mapping function normalizes these scores, applying a floor value ( $\alpha$ ) to ensure static background tokens still accumulate a baseline update probability, preventing them from being completely frozen.
- Importance-Weighted Accumulation: An error accumulator tracks the estimated residual change for each token. Tokens with high motion (high importance weights) accumulate error rapidly and trigger a recomputation once a threshold is exceeded. Static tokens accumulate slowly, allowing them to reuse cached residuals for longer periods.
- Execution: Only active tokens (those exceeding the threshold) are forwarded for computation; inactive tokens retrieve cached residuals.

Key Contributions

Theoretical Analysis of Caching Error: The paper provides a rigorous proof that caching error is bounded by residual instability and identifies Heterogeneous Temporal Redundancy and Intra-Chunk Frame Discrepancy as fundamental properties of autoregressive models that coarse-grained methods overlook.
Motion-Induced Residual Instability: A theoretical link is established proving that frame differences serve as an upper bound for residual instability, validating the use of frame differences as a lightweight, high-fidelity proxy for token importance.
MotionCache Framework: A novel acceleration framework implementing a dual-stage schedule (warm-up + token-wise adaptation) that dynamically allocates computational resources, prioritizing high-motion regions while efficiently reusing residuals for static backgrounds.
State-of-the-Art Performance: Extensive evaluations demonstrate that MotionCache outperforms existing methods (TeaCache, FlowCache) in both speed and quality.

Experimental Results

The method was evaluated on two state-of-the-art autoregressive models: SkyReels-V2 and MAGI-1.

SkyReels-V2 (540p, 177 frames):
- MotionCache achieved a 6.28× speedup (slow configuration) and 7.26× (fast configuration).
- Compared to FlowCache (6.26× speedup), MotionCache maintained superior visual fidelity with a PSNR of 23.46 vs. 21.83, and a VBench score of 82.84% vs. 82.70%.
- TeaCache provided only a 2.20× speedup with significantly lower quality (PSNR 18.39).
MAGI-1 (720p, 168 frames):
- MotionCache achieved a 1.64× speedup (slow) and 2.07× (fast).
- It preserved visual quality with a VBench score of 77.25% (slow) and 74.59% (fast), significantly outperforming FlowCache-fast (73.42%) and TeaCache-fast (68.81%), which suffered from severe quality degradation.
- PSNR improved to 19.71 (slow) compared to FlowCache's 18.16.

Qualitative results highlighted that MotionCache effectively preserved fine-grained details (e.g., texture on clothing, animal features) and structural consistency, whereas competing methods exhibited texture drifting, structural misalignment, and hallucinations.

Significance and Claims

The paper claims that MotionCache establishes a new state-of-the-art for efficient autoregressive video generation. By moving beyond the "all-or-nothing" constraints of coarse-grained caching, MotionCache enables a paradigm where computational resources are dynamically allocated based on motion dynamics. The authors assert that this fine-grained, motion-centric approach is essential for preserving the structural and aesthetic integrity of complex video content while achieving significant inference speedups, thereby paving the way for the real-time deployment of autoregressive video generation models. The work is presented as a solution to the specific limitations of current caching strategies in handling the spatial and temporal heterogeneity inherent in video generation.