💻 computer science

DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation

The paper proposes DCARL, a novel divide-and-conquer autoregressive framework that combines a dedicated keyframe generator for global structural consistency with an interpolation generator for local coherence, enabling high-fidelity, stable long-trajectory video generation up to 32 seconds while outperforming state-of-the-art baselines in visual quality and camera adherence.

Original authors: Junyi Ouyang, Wenbin Teng, Gonglin Chen, Yajie Zhao, Haiwei Chen

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junyi Ouyang, Wenbin Teng, Gonglin Chen, Yajie Zhao, Haiwei Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to draw a massive, 32-second-long movie scene on a single, continuous strip of paper. You start with a sketch of the first frame, and you have to keep drawing the next frame based on the one before it, over and over again, until the movie is done.

This is what current AI video generators try to do. They are like a student who is great at drawing one picture but gets tired and confused after a while. By the time they reach the 30-second mark, their drawing has drifted off the page. The car they were drawing might suddenly turn into a tree, or the road might twist into a spiral that defies physics. This is called "visual drift." The AI loses track of where it is supposed to be going.

DCARL is a new method that solves this problem by changing the strategy. Instead of trying to draw the whole movie in one continuous, exhausting line, it uses a "Divide-and-Conquer" approach. Think of it like building a long bridge.

The Problem: The "Drifting" Artist

If you ask a standard AI to generate a long video, it acts like a person trying to walk in a straight line while blindfolded, taking one step at a time based only on where they just stepped.

Step 1: They take a step forward.
Step 2: They take another step, but they are slightly off-course from Step 1.
Step 3: They take another step, compounding the error from Step 2.

By Step 100, they are miles away from the straight line they were supposed to follow. In video terms, the camera might spin wildly, or the scenery might melt into nonsense.

The DCARL Solution: The "Architect and the Mason"

DCARL splits the job into two distinct roles, like a construction project with an Architect and a Mason.

1. The Architect (The Keyframe Generator)

First, the AI doesn't try to draw every single frame. Instead, it acts as an Architect who draws a few crucial "blueprints" or Keyframes spread out across the 32-second timeline.

Imagine you are planning a road trip from New York to Los Angeles. The Architect doesn't draw every mile of the road. Instead, they drop pins on a map at major cities: New York, Chicago, Denver, Los Angeles.
These "pins" (Keyframes) are generated all at once, looking at the whole journey together. This ensures that the destination is correct and the route makes sense globally. The AI knows exactly where the road should be at these major checkpoints.

2. The Mason (The Interpolation Generator)

Once the Architect has placed the pins, the Mason comes in to fill in the gaps.

The Mason's job is to draw the smooth road between New York and Chicago, then between Chicago and Denver, and so on.
Because the Mason has the "pins" (the Keyframes) as strict boundaries, they can't wander off. They know, "I must start at the Chicago pin and end at the Denver pin."
The Secret Sauce: To make the road look natural and not just a stiff line connecting two dots, the Mason is given a little bit of "noise" (randomness) and told to imagine the motion. This prevents the AI from just copying the pictures of the pins (which would look like a stuttering video) and forces it to actually animate the movement between them.

Why This Works Better

In the old way (Pure Autoregressive), the AI was like a person trying to walk a tightrope without a safety net. One small wobble at the start became a massive fall by the end.

With DCARL:

The Safety Net: The Keyframes act as safety nets. Even if the Mason makes a small mistake while drawing the road between Chicago and Denver, the next Keyframe (Denver) pulls the video back to the correct path. The error can't grow forever because it gets "reset" at every major checkpoint.
The Seamless Stitch: The paper also mentions a clever trick to make sure the road doesn't look like it was cut and pasted together. The Mason overlaps the end of one section with the start of the next, blending them so smoothly that you can't tell where one section ended and the next began.

The Result

The paper shows that this method allows the AI to generate 32-second videos that stay perfectly on track.

Visual Quality: The video looks crisp and real, not blurry or melted.
Camera Control: If you tell the AI, "Move the camera forward and turn left," it actually does that for the whole 32 seconds. It doesn't forget the instruction halfway through.
Stability: The video doesn't drift into a hallucination. The car stays on the road, and the buildings stay upright.

In short: DCARL stops the AI from trying to remember the whole movie at once. Instead, it gives the AI a map with major landmarks (Keyframes) and asks it to fill in the details between them, ensuring the journey stays on course from start to finish.

1. Problem Statement

The paper addresses the critical challenge of long-trajectory video generation (e.g., generating 30+ seconds of continuous video) for world modeling, autonomous driving, and terrain reconstruction. Existing approaches face two primary limitations:

Video Diffusion Models (VDMs): While they produce high-fidelity images, they struggle with scalability. Generating long sequences collectively incurs prohibitive computational costs, and they often fail to maintain consistency over extended durations.
Autoregressive (AR) Models: These models allow for infinite rollout but suffer from exposure bias and error accumulation. As the model generates frames sequentially based on its own previous outputs, small prediction errors compound, leading to severe visual drift (degradation of quality) and trajectory inconsistency (failure to adhere to camera pose instructions).

Current attempts to mitigate drift (e.g., Diffusion Forcing, Self-Forcing) often fail to explicitly bound error propagation, resulting in unstable long-term generation and poor controllability.

2. Methodology: DCARL Framework

The authors propose DCARL, a novel Divide-and-Conquer framework that combines the structural stability of global planning with the high-fidelity generation of VDMs. The approach is formulated as a two-stage autoregressive pipeline:

A. Stage 1: Global Keyframe Generation (Structural Anchors)

Instead of generating every frame sequentially, the model first synthesizes a sparse set of Keyframes ( $K$ ) that span the entire video timeline.

Mechanism: A dedicated Keyframe Generator ( $G_K$ ) uses a flow-matching process (DiT-based) to generate these frames jointly from the initial image, camera trajectory, and text caption.
Innovation: To preserve spatial fidelity, the model treats keyframes as a batch of independent images rather than a temporally compressed video sequence. This avoids the lossy compression inherent in standard temporal VAEs, ensuring the keyframes serve as high-quality structural anchors.
Theoretical Basis: The authors prove (Proposition 1) that purely autoregressive generation leads to error divergence (linear or exponential growth). By anchoring the generation to globally synthesized keyframes, the error accumulation is bounded, preventing catastrophic drift.

B. Stage 2: Semi-Autoregressive Interpolation (Local Refinement)

Once the global structure is established, a Interpolation Generator ( $G_I$ ) fills in the dense frames between the keyframes.

Mechanism: The generator synthesizes video segments in an autoregressive manner but is conditioned on:
1. Global Context: A subset of surrounding keyframes (both past and future relative to the segment).
2. Local Context: The immediately preceding generated frames.
Key Technical Modules:
- Motion-Inductive Noisy Conditioning: To prevent the model from simply "copy-pasting" pixels from the clean keyframes (which causes stuttering), the keyframe latents are perturbed with noise during both training and inference. This forces the model to learn actual motion dynamics rather than relying on visual shortcuts.
- Seamless Boundary Consistency: To eliminate visual fractures at segment junctions, the model uses a latent substitution strategy. It overlaps segments and replaces the first few frames of a new segment with the latent representation of the previous segment's end, ensuring strict temporal continuity.

3. Key Contributions

Divide-and-Conquer Paradigm: The paper introduces a framework that decouples global structural planning (keyframes) from local detail refinement (interpolation), effectively solving the unbounded error accumulation problem in autoregressive video generation.
Theoretical Error Bounding: The authors provide a mathematical proof demonstrating that their keyframe-anchored approach bounds the cumulative generation error, contrasting it with the divergent nature of pure autoregressive models.
Novel Conditioning Strategies:
- Spatial-Structural Preservation: Avoiding temporal compression for keyframes to maintain high-resolution geometric cues.
- Motion-Inductive Noisy Conditioning: A technique to break identity mapping shortcuts and ensure dynamic motion.
- Seamless Boundary Consistency: A latent substitution method to ensure smooth transitions between generated segments.
Dual-Generator Architecture: A specialized design separating the tasks of global planning and local interpolation, optimized for long-horizon tasks.

4. Experimental Results

The method was evaluated on large-scale datasets (OpenDV-YouTube and nuScenes) for generating videos up to 32 seconds long.

Quantitative Performance:
- Visual Quality: DCARL achieved significantly lower FID (Fréchet Inception Distance) and FVD (Fréchet Video Distance) compared to state-of-the-art baselines (Diffusion Forcing, Self-Forcing, Deep-Forcing, Vista, SEVA). For example, on ODV-YT, DCARL achieved an overall FID of 19.2 vs. 35.0 for Diffusion Forcing.
- Camera Control: The method showed superior adherence to camera trajectories, with lower ATE (Absolute Trajectory Error) and ARE (Absolute Rotation Error).
- Stability: Unlike baselines where performance degraded sharply as video length increased, DCARL maintained consistent quality across 0–32 seconds.
Zero-Shot Generalization: The model demonstrated robust performance on the unseen nuScenes dataset without additional fine-tuning, outperforming all baselines in both fidelity and controllability.
Ablation Studies:
- Removing keyframe conditioning led to severe drift and loss of camera control.
- Using temporal compression for keyframes degraded visual quality and geometric accuracy.
- Applying noise to keyframes during both training and inference was crucial for preventing motion stagnation.

5. Significance

DCARL represents a significant advancement in generative world modeling. By effectively bounding error accumulation in autoregressive systems, it enables the synthesis of long-duration, high-fidelity, and controllable video sequences (up to 32 seconds) that were previously unattainable with existing methods.

This work bridges the gap between the structural stability required for long-horizon tasks and the visual fidelity of diffusion models. It provides a scalable solution for applications requiring realistic, long-term simulations, such as autonomous driving training, robotics planning, and interactive world models, where maintaining geometric consistency and camera control over time is paramount.