Technical Summary: Resampling Forcing for Autoregressive Video Diffusion

Problem Statement

Autoregressive (AR) video diffusion models offer a promising paradigm for world simulation by predicting future frames conditioned on past observations, strictly adhering to the causal nature of physical dynamics. However, these models suffer significantly from exposure bias. During training, models typically rely on "teacher forcing," where they condition on ground-truth history frames. During inference, however, they must condition on their own previously generated (and imperfect) outputs. This train–test mismatch leads to error accumulation, where small artifacts in early predictions are amplified across the autoregressive rollout, often resulting in catastrophic video collapse over long horizons.

Existing solutions to mitigate this mismatch generally rely on post-training strategies that require auxiliary models:

1. Distillation-based approaches (e.g., Self Forcing): Train a bidirectional teacher model first, then distill it into a causal student. This introduces a dependency on a non-causal teacher that can leak future information, compromising strict temporal causality.
2. Adversarial approaches (e.g., AAPT): Jointly optimize a generator and a discriminator. This requires complex online discriminator training and limits scalability.

Furthermore, extending AR generation to long videos exacerbates attention complexity, as the historical context grows indefinitely, creating computational bottlenecks for both training and inference.

Methodology: Resampling Forcing

The authors propose Resampling Forcing, an end-to-end, teacher-free framework designed to train autoregressive video diffusion models from scratch. The core philosophy is to train the model to be robust against input perturbations (simulated errors) while maintaining a clean prediction objective.

1. Self-Resampling Mechanism

To simulate inference-time model errors during training, the method introduces a self-resampling scheme:

• Process: Instead of conditioning on clean ground-truth history, the model conditions on "degraded" history frames.
• Implementation:
  1. Ground-truth frames are corrupted to a sampled timestep $t_s$ (introducing noise).
  2. The online model itself (with current weights) is used to autoregressively complete the remaining denoising steps from $t_s$ to 0, producing a degraded, noise-free frame $\tilde{x}$.
  3. This degraded frame $\tilde{x}$, which contains model-induced errors, is then used as the condition for predicting the next frame.
• Key Design: Gradients are detached from this resampling process to prevent "shortcut learning" (where the model learns to ignore the input rather than correct it). This forces the model to learn to correct its own imperfections.
• Timestep Sampling: The corruption timestep $t_s$ is sampled from a logit-normal distribution, shifted by a parameter $s$ to balance history faithfulness and error correction flexibility. A warm-up phase using standard teacher forcing is employed initially before switching to Resampling Forcing.

2. History Routing for Efficient Attention

To address the computational cost of attending to an ever-growing history in long-horizon generation, the authors introduce a parameter-free history routing mechanism:

• Dynamic Retrieval: Instead of attending to all previous frames (dense causal attention) or a fixed sliding window, the model dynamically retrieves the top-$k$ most relevant history frames for each query token.
• Selection Metric: Relevance is determined by the dot product between the query token and a mean-pooled descriptor of the history frames' keys.
• Complexity: This reduces attention complexity from linear $O(L)$ to constant $O(k)$ as the video length $L$ increases, while maintaining a receptive field larger than a fixed window by allowing diverse context combinations.

3. Training Objective

The model is trained using a sparse causal mask that enforces temporal causality (frames only attend to past frames) while enabling parallel training via a per-frame diffusion loss. The loss is computed between the model's prediction and the ground-truth clean frame, conditioned on the degraded (resampled) history.

Key Contributions

1. Teacher-Free End-to-End Training: The first framework to train autoregressive video diffusion models from scratch without relying on bidirectional teachers or discriminators, avoiding future information leakage.
2. Resampling Forcing Algorithm: A novel self-resampling strategy that simulates inference-time errors during training, effectively mitigating error accumulation and stabilizing long-horizon generation.
3. Parameter-Free History Routing: A dynamic mechanism that selects relevant historical context, enabling efficient long-video generation with near-constant attention complexity and negligible quality loss compared to dense attention.
4. Native Long-Video Training: The ability to train natively on long sequences (e.g., 15 seconds) rather than relying on distillation from short clips, leading to superior temporal consistency.

Experimental Results

The method was evaluated on the Wan2.1-1.3B architecture, trained on 5s and 15s videos.

• Qualitative Performance:
  • Stability: Unlike baselines (e.g., Pyramid Flow, CausVid) that exhibit progressive degradation in color and texture over time, Resampling Forcing maintains stable visual quality in 15-second videos.
  • Causality: In physical simulations (e.g., pouring milk), the model strictly adheres to causal laws (monotonic liquid level rise), whereas distillation-based baselines (e.g., LongLive) often violate causality due to future information leakage from their teachers.
• Quantitative Performance (VBench):
  • The method achieves temporal and visual quality scores comparable to or better than state-of-the-art distilled baselines (Self Forcing, CausVid) across 0–5s, 5–10s, and 10–15s segments.
  • It outperforms baselines in temporal consistency on longer videos, attributed to native-length training.
• Efficiency:
  • The history routing mechanism achieves 75% attention sparsity (top-5 out of 20 frames) with negligible quality drop compared to dense attention.
  • It eliminates the need for a massive 14B-parameter bidirectional teacher required by competing distillation methods.

Significance and Claims

The paper claims that Resampling Forcing advances the scalability of autoregressive video world models by solving the fundamental exposure bias problem without auxiliary models. By simulating errors during training, the model learns to correct its own drift, enabling stable generation over long horizons. The integration of dynamic history routing further addresses the memory and compute bottlenecks of long-context attention. The authors position this work as a viable path toward training robust, strictly causal video models from scratch, offering a practical alternative to distillation-based pipelines that compromise temporal causality for quality.