💻 computer science

Grounded Forcing: Bridging Time-Independent Semantics and Proximal Dynamics in Autoregressive Video Synthesis

This paper introduces Grounded Forcing, a novel framework that enhances long-term coherence and controllability in autoregressive video synthesis by integrating a Dual Memory KV Cache, Dual-Reference RoPE Injection, and Asymmetric Proximity Recache to simultaneously address semantic forgetting, visual drift, and instruction switching challenges.

Original authors: Jintao Chen, Chengyu Bai, Junjun hu, Xinda Xue, Mu Xu

Published 2026-04-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jintao Chen, Chengyu Bai, Junjun hu, Xinda Xue, Mu Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to tell a story to a friend, but your friend has a very short attention span. Every time you tell them a new sentence, they immediately forget the first one you said. If you try to tell a long story, they only remember the last few words and have no idea who the main character is or what the setting looks like.

This is exactly the problem with current AI video generators when they try to make long videos. They suffer from three main issues:

Forgetting: They lose track of who the characters are after a while.
Drifting: The video starts to look weird or "glitchy" because the AI gets confused about how much time has passed.
Stubbornness: If you tell the AI to change the story halfway through (e.g., "Now the man runs"), the AI either ignores you or changes the character's face entirely because it's too jumpy.

The paper introduces a new method called Grounded Forcing to fix these problems. Think of it as giving the AI a "super-brain" with three special tools to keep the story coherent, no matter how long it gets.

The Three Magic Tools

1. The "Dual-Notebook" System (Dual Memory KV Cache)

The Problem: Imagine the AI is writing a story in a notebook. Usually, it only keeps the last few pages visible. If the story gets too long, it has to tear out the old pages to make room for new ones. But if it tears out the page with the hero's face, the hero changes in the next scene.

The Solution: Grounded Forcing gives the AI two notebooks:

The "Action Notebook" (Local Temporal Memory): This is a small, fast notebook where the AI writes down what's happening right now (running, jumping, talking). It gets wiped clean and rewritten constantly to keep the motion smooth.
The "Identity Notebook" (Global Consistency Memory): This is a special, protected notebook where the AI keeps the "Golden Rules" of the story: Who is the hero? What does the background look like? This notebook is never thrown away. Even if the action changes, the AI keeps checking this notebook to make sure the hero still looks like the hero.

Analogy: It's like a director filming a movie. The camera (Action Notebook) follows the actor's movements, but the director (Identity Notebook) constantly reminds the crew, "Remember, this is still Iron Man, not a random guy in a suit."

2. The "Timeless Anchor" (Dual-Reference RoPE Injection)

The Problem: AI models usually count time like a stopwatch: 1 second, 2 seconds, 3 seconds... forever. But the AI was only trained on short videos (maybe 5 seconds). When the stopwatch hits 100 seconds, the AI gets confused because it's never seen numbers that big. The video starts to warp and melt.

The Solution: The authors invented a way to reset the stopwatch for different parts of the video.

For the Action Notebook, the AI uses a "relative stopwatch." It only cares about the last few seconds (e.g., "0 to 20 seconds ago"). This keeps the motion smooth.
For the Identity Notebook, the AI uses a "timeless anchor." It tells the AI, "This character exists at Time Zero." No matter how long the video gets, the AI treats the character's identity as if it's always at the start.

Analogy: Imagine you are walking down a long road. Your feet (the action) are moving relative to the ground right in front of you. But your memory of your home (the identity) is a fixed point on a map that doesn't move, no matter how far you walk. This prevents the AI from getting lost in "time."

3. The "Smooth Transition" (Asymmetric Proximity Recache)

The Problem: Imagine you are driving a car. If you suddenly yell, "Turn left!" to the driver, and the driver instantly spins the car 180 degrees, you'd crash. But if they ignore you, you miss the turn. Current AI is like a bad driver: it either spins wildly or ignores you.

The Solution: Grounded Forcing uses a "sliding scale" for changes.

When you give a new instruction (e.g., "Now the cat flies"), the AI changes the immediate future frames quickly to follow your command.
However, for the distant past frames, it changes them very slowly or not at all. This creates a smooth bridge between the old story and the new one.

Analogy: Think of it like a dimmer switch on a light, not an on/off button. When you change the prompt, the AI slowly fades the old story out and fades the new story in, rather than snapping a picture and switching instantly. This keeps the character's face and the background consistent while still letting the action change.

The Result

By combining these three tools, Grounded Forcing allows the AI to generate a 1-minute (or even longer) video where:

The characters stay the same (no identity swapping).
The video doesn't get weird or glitchy over time.
You can change the story mid-video (e.g., "The grandma is now a superhero"), and the AI adapts smoothly without breaking the illusion.

In short, it turns a forgetful, jittery AI into a reliable storyteller that can handle long, complex, and interactive movies without losing its mind.

1. Problem Statement

The paper addresses the critical limitations of autoregressive (AR) video synthesis when scaling from short clips to infinite-horizon, interactive long-form generation. While AR models offer low-latency streaming capabilities compared to bidirectional diffusion models, they suffer from three intertwined challenges that degrade performance over time:

Semantic Forgetting: Due to context window limitations (sliding windows), the model loses access to initial semantic anchors (e.g., character identity, scene layout) as the video progresses, leading to identity swaps or background inconsistencies.
Visual Drift: Standard Rotary Positional Embeddings (RoPE) bind tokens to absolute temporal indices. In infinite generation, these indices exceed the training distribution (Out-of-Distribution), causing attention mechanisms to degenerate, resulting in motion jitter and quality collapse.
Controllability Loss: Existing methods for switching prompts (e.g., changing instructions mid-video) often rely on uniform cache refreshing. This causes "semantic shock" (abrupt visual jumps) or instruction ignoring, failing to balance immediate adherence to new prompts with the preservation of long-term context.

Current solutions typically address these issues in isolation (e.g., anchoring only to the first frame), which fails in dynamic scenarios where semantics must evolve (e.g., a character transforming).

2. Methodology: Grounded Forcing

The authors propose Grounded Forcing, a framework that decouples time-independent semantics (global identity) from proximal dynamics (local motion) using three synergistic mechanisms:

A. Dual Memory KV Cache

To solve semantic forgetting, the authors architecturally split the Key-Value (KV) cache into two distinct components:

Local Temporal Memory (LTM): A standard sliding window storing recent frames to capture high-frequency motion and proximal dynamics. These frames are evicted as new ones arrive.
Global Consistency Memory (GCM): A persistent, sparse set of keyframes that anchor the video's semantic identity (e.g., subject identity, core style). These are protected from standard eviction.
Diversity-Aware Update Strategy: The GCM is not static. It dynamically updates based on latent diversity. When a new frame is generated, its importance is calculated based on its dissimilarity to existing anchors. If a new frame represents a novel semantic state (e.g., a character transformation), it replaces the most redundant anchor in the GCM, ensuring the memory spans the evolving semantic space without bloating.

B. Dual-Reference RoPE Injection

To suppress visual drift caused by positional extrapolation, the authors introduce Dual-Reference RoPE Injection (DR-RoPE):

Raw Key Storage: Instead of caching position-encoded keys, the model stores raw keys ( $K_{raw}$ ) in both LTM and GCM.
Time-Invariant Global Anchors: For the GCM, positional information is injected with a fixed temporal index ( $t=0$ ). This renders global memory tokens position-agnostic, allowing the model to attend to them as timeless semantic references regardless of the video's duration.
Relative Local Context: For the LTM, temporal indices are mapped relative to the current sliding window (e.g., $[0, 21]$ ). This ensures local motion dynamics always remain within the model's trained perceptual range, preventing OOD positional degradation.
Multi-Shot Support: The system supports cinematic scene cuts by resetting the LTM while retaining the GCM, preventing temporal contamination between distinct scenes.

C. Asymmetric Proximity Recache (APR)

To resolve controllability loss during prompt switching, the authors propose Asymmetric Proximity Recache:

Instead of a uniform or binary cache flush, APR applies a proximity-weighted interpolation between the old cache (previous semantics) and the new cache (new prompt).
Mechanism: The update weight $\alpha_t$ $α_{t}$ is a function of temporal distance from the current generation step.
- Recent Frames (High Proximity): High $\alpha_t$ allows immediate adherence to new instructions (e.g., changing motion).
- Distant Frames (Low Proximity): Low $\alpha_t$ preserves the original semantic anchors (identity, background).
This creates a "semantic bridge," enabling smooth transitions without the abrupt visual jumps associated with uniform cache refreshing.

3. Key Contributions

Systematic Decoupling: Identified and solved the entanglement of static semantics and dynamic motion in AR video generation.
Dual Memory KV Cache: Introduced a structural innovation separating local motion (LTM) from global identity (GCM) with a diversity-aware update mechanism to prevent semantic degradation.
Positional Stability: Developed Dual-Reference RoPE Injection to confine positional embeddings within the training manifold while making global semantics time-invariant.
Seamless Interaction: Created Asymmetric Proximity Recache (APR) to enable fine-grained, smooth prompt switching that balances responsiveness with context retention.

4. Experimental Results

The method was evaluated on the Wan2.1-T2V-1.3B backbone, comparing against state-of-the-art baselines like LongLive, Rolling Forcing, and Infinity-RoPE.

Quantitative Performance (240s videos):
- Achieved State-of-the-Art (SOTA) in Subject Consistency (0.9163) and Background Consistency (0.9265).
- Maintained high Dynamic Degree (0.60) and Motion Smoothness, proving it does not sacrifice motion quality for stability.
- Outperformed baselines in Aesthetic Quality and Temporal Flickering metrics.
Qualitative Results:
- Demonstrated successful multi-shot generation (e.g., Man $\to$ Hulk $\to$ Silver Wyvern) where baselines failed due to identity drift or forgetting.
- Showed smooth transitions in single-shot sequences (e.g., Iron Man $\to$ Laser $\to$ Explosion) without visual artifacts.
Ablation Studies: Removing any of the three core components (Dual Mem, DR-RoPE, APR) resulted in significant drops in Subject Consistency and Background Consistency, validating the necessity of the full system.
User Study: In a 5-point Likert scale evaluation, Grounded Forcing scored highest in all categories, particularly in Subject Consistency (3.66 vs. ~3.0 for baselines) and Text Adherence.

5. Significance

Grounded Forcing represents a paradigm shift in autoregressive video synthesis. By moving beyond simple sliding windows or rigid first-frame anchoring, it provides a robust foundation for interactive, infinite-horizon world simulators.

Real-Time Viability: It maintains the low-latency benefits of AR models while solving the long-term coherence issues that previously limited them to short clips.
Interactive Control: It enables complex user interactions (dynamic prompt switching, multi-shot storytelling) without sacrificing narrative flow or character identity.
Future Impact: The framework paves the way for truly immersive, long-form generative video applications where users can steer the narrative in real-time over extended durations.