Technical Summary: Flex-Forcing

Problem Statement

Current video generation methods are constrained by a rigid inference paradigm, forcing a trade-off between two dominant approaches:

1. Bidirectional Diffusion Models: These leverage full-context attention to model all frames jointly, achieving high visual fidelity and strong long-range temporal coherence (e.g., camera motion, scene transitions). However, they suffer from high computational overhead and slow inference, limiting their scalability for real-time or long-video generation.
2. Autoregressive Models: These generate frames sequentially using causal conditioning and KV caching, enabling efficient, streaming, and real-time inference. However, the lack of global context during generation leads to error accumulation, causing drift in object appearance and motion, and resulting in weaker global temporal consistency.

Existing work typically optimizes for one paradigm while compromising the other, lacking a unified mechanism to jointly realize the advantages of both within a single framework.

Methodology: Flex-Forcing

The authors propose Flex-Forcing, a unified training and inference framework that enables a single video diffusion model to operate seamlessly under bidirectional, autoregressive, or hybrid regimes. The core innovation lies in a Flexible Chunking mechanism defined over two orthogonal axes:

1. Flexible Chunking Strategy

• Temporal Axis (Frame Chunking): The video is partitioned into contiguous chunks of varying sizes. Within a chunk, frames are modeled bidirectionally (full attention). Between chunks, generation remains autoregressive (conditioned on previous chunks). This allows the model to adapt chunk granularity to different device budgets and video lengths.
• Denoising Timestep Axis: The chunking strategy is dynamic across the denoising trajectory. Early steps (high noise) utilize larger chunks to capture global structure, while later refinement steps (low noise) utilize smaller chunks to focus on local details. This creates a hierarchical, pyramid-like structure where chunk size decreases as noise levels drop.

2. Training Objective

The framework trains a single model to handle mixed causal and non-causal contexts through:

• Stochastic Chunking: During training, the model is exposed to random chunk partitions ranging from strictly causal (frame-by-frame) to fully bidirectional. This is achieved via an asymmetric distillation pipeline (inspired by CausVid and Self-Forcing) that injects causality while preserving non-causal attention capabilities.
• Noise-Level Alignment (K-Projection): A critical challenge arises because the same query token may attend to key-value states from different noise levels (clean past tokens vs. noisy future tokens). Standard self-attention treats these uniformly, causing performance degradation. The authors introduce a K-Projection, a lightweight, timestep-dependent linear projection that maps clean key states from past frames into the "noisy" latent space of the current timestep. This ensures all attended keys reside in a consistent noise representation space, stabilizing flexible inference.

3. Inference Flexibility

At test time, the trained model can be configured to:

• Operate in Bidirectional Mode for maximum global consistency.
• Operate in Autoregressive Mode for maximum efficiency and streaming.
• Operate in Hybrid Mode, adjusting chunk sizes and noise-level dependencies to find an optimal Pareto frontier between quality and speed.

Key Contributions

1. Unified Framework: The first approach to demonstrate that bidirectional and autoregressive paradigms can coexist within a single trained model, allowing test-time control over the inference regime.
2. Flexible Chunking & Alignment: The introduction of flexible chunking over both temporal frames and denoising steps, coupled with a train-test consistent objective and the K-Projection mechanism to bridge different causal contexts.
3. Any-Order, Any-Timestep Editing: Leveraging the unified framework, the authors enable new editing capabilities:
   • Any-Timestep Editing: Restricting edits to low-level refinement timesteps while preserving high-level planning, decoupling global structure from local details.
   • Any-Order Editing: Conditioning on both past and future clean tokens allows arbitrary temporal segments to be re-edited after full-video generation without re-generating the entire sequence.

Experimental Results

Extensive experiments were conducted on 5-second and 30-second video benchmarks (using Wan2.1 as the base).

• Performance-Efficiency Trade-off: Flex-Forcing achieves a markedly improved Pareto frontier compared to baselines like Self-Forcing. It consistently outperforms causal models in quality while offering faster inference than rigid bidirectional models.
• 5-Second Videos: In configurations matching the baseline (1.3B parameters), Flex-Forcing achieved higher VBench scores (e.g., 85.07 vs. 84.31 for Self-Forcing) and higher FPS (25.8 vs. 24.9). It also outperformed few-step distilled diffusion models.
• 30-Second Videos: On long-video benchmarks, Flex-Forcing outperformed Infinity-RoPE and Self-Forcing across most metrics, particularly in Dynamic Degree and Motion Smoothness, demonstrating superior long-horizon planning and reduced error accumulation.
• User Preference: User studies indicated a strong preference for Flex-Forcing over Self-Forcing in both visual quality and prompt alignment for 5s and 30s videos.
• Ablation: The K-Projection was shown to be essential; without it, performance degraded significantly as chunk sizes increased, whereas with it, performance remained stable and improved.

Significance and Claims

The paper claims that Flex-Forcing reframes bidirectional and autoregressive generation not as competing paradigms, but as controllable inference dimensions that can be smoothly adjusted based on requirements. By unifying these approaches, the method:

• Reduces the computational cost and environmental footprint of maintaining separate specialized architectures.
• Democratizes high-fidelity, long-form video generation for resource-constrained environments by allowing dynamic speed-quality trade-offs at runtime.
• Provides precise, localized editing tools (any-order, any-timestep) for creative and industrial applications, promoting adaptive generative systems.

The authors acknowledge limitations, noting that while the strict left-to-right constraint is relaxed, training-inference mismatch can still accumulate errors in very long videos, and the method's effectiveness relies heavily on pre-training priors (specifically bidirectional encoding).