DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation

1. Problem Statement

Recent video generative models have achieved high visual fidelity and short-term temporal coherence but struggle with long-term consistency across three distinct dimensions:

1. Intra-clip World Knowledge Consistency: Ensuring objects, attributes, and physical interactions within a single video clip are semantically and physically plausible (e.g., avoiding contradictions in object behavior).
2. Inter-clip Camera Consistency: Maintaining precise, stable, and geometrically correct camera motion and view transitions across different clips.
3. Inter-shot Element Consistency: Preserving the identity of characters, scenes, and styles across multiple shots in long-form narratives (long-range coherence).

The Core Challenge: Enforcing all three types of consistency within a single monolithic diffusion model is difficult. It leads to representational entanglement, where the model must simultaneously reason over semantics, geometry, and long-range temporal dependencies, often resulting in suboptimal trade-offs between controllability, coherence, and computational efficiency.

2. Methodology: Divide-and-Conquer Diffusion Model (DCDM)

The authors propose DCDM, a system-level framework that decomposes video consistency modeling into three dedicated components while sharing a unified Diffusion Transformer (DiT) backbone.

A. Intra-clip World Knowledge Consistency (Semantic Reasoning)

• Goal: Ensure semantic and physical coherence within a single clip.
• Mechanism: LLM-based Prompt Extension (PE).
  • User prompts are often under-specified. DCDM employs a Large Language Model (LLM, e.g., Qwen3) to parse the input prompt and explicitly infer implicit details (entities, attributes, spatial relations, actions).
  • The LLM generates a refined prompt ($\tilde{p}$) which is then embedded and fed as textual conditioning ($c_{text}$) into the DiT backbone.
  • Benefit: This separates semantic reasoning from visual generation, grounding the video in explicit world knowledge to reduce contradictions.

B. Inter-clip Camera Consistency (Geometric Control)

• Goal: Achieve precise and stable camera motion control.
• Mechanism: Temporal Camera Representation in Noise Space.
  • Classification: An LLM classifies the prompt's camera motion into predefined categories (e.g., pan left, zoom in, static).
  • Noise Warping: Instead of conditioning only at the text level, DCDM constructs a camera representation in the noise space. It starts with standard Gaussian noise ($z_1$) and propagates it through time using a reprojection operator ($W_{t \leftarrow t-1}$) derived from camera intrinsics/extrinsics.
  • Blending: The warped noise is blended with independent Gaussian noise to preserve stochasticity while enforcing temporal coherence: $z_t = \sqrt{\lambda}\tilde{z}_t + \sqrt{1-\lambda}\epsilon_t$.
  • Initialization Injection: This structured noise replaces the initial noise input ($x_T$) of the diffusion process, anchoring the entire denoising trajectory to the desired camera motion.
  • Enhancement: A Text-to-Image initialization strategy is used, where a reference image is generated first to reduce appearance ambiguity before conditioning the DiT on both the image and camera representation.

C. Inter-shot Element Consistency (Narrative Coherence)

• Goal: Maintain character identity and style across long sequences (minutes) without excessive computational cost.
• Mechanism: Sparse Inter-shot Self-Attention.
  • Problem: Full attention across all frames in a long video is computationally prohibitive ($O(N^2)$).
  • Solution: The model uses a windowed cross-attention mechanism to localize prompts to specific shots. For inter-shot consistency, it employs a sparse self-attention pattern:
    • Intra-shot: Dense bidirectional attention within a shot to ensure smooth motion.
    • Inter-shot: A global context summary is created by selecting a small subset of key-value tokens (e.g., from the first frame) from each shot. These summaries form a global cache ($KV_{global}$).
    • Attention Logic: Queries from the current shot attend to both local tokens and the global summary tokens.
  • Complexity: Reduces complexity from $O((N_s L_{shot})^2)$ to approximately $O(N_s \cdot L_{shot} \cdot S)$, where $S$ is the number of summary tokens ($S \ll L_{shot}$).

3. Key Contributions

1. Unified Framework: Proposes DCDM, a system-level architecture that decomposes video consistency into three complementary dimensions (intra-clip, inter-clip, inter-shot) rather than forcing them into a monolithic model.
2. Semantic Grounding: Introduces an LLM-based prompt extension mechanism to enrich implicit semantics, significantly improving intra-clip world knowledge consistency.
3. Geometric Control: Develops a novel temporal camera representation injected into the noise space, enabling precise and stable camera motion control that text-only conditioning cannot achieve.
4. Efficient Long-Range Modeling: Introduces sparse inter-shot self-attention, enabling minute-scale video generation with strong narrative coherence and persistent memory of characters/scenes while maintaining computational efficiency.

4. Results

• Benchmark: Validated on the test set of the AAAI'26 Consistency in Video Generative Models (CVM) Competition.
• Performance: The framework achieved Top Team status in the competition.
• Qualitative Evidence:
  • Intra-clip: Generated videos showed plausible physics and consistent object attributes (e.g., a man riding a specific animal without morphing).
  • Inter-clip: Demonstrated precise execution of complex camera moves (e.g., dolly in, truck right) with stable geometry.
  • Inter-shot: Successfully maintained character identity and scene style across multiple shots in long narratives (e.g., a multi-shot story about a "tracker" in a cyberpunk setting).

5. Significance

The paper demonstrates that system-level modeling is superior to monolithic approaches for complex video generation tasks. By "dividing and conquering" the problem:

• It decouples semantic reasoning (handled by LLMs), geometric control (handled by noise-space warping), and narrative coherence (handled by sparse attention).
• It provides a scalable path toward generating controllable, consistent, and long-form video content that adheres to cinematic structures, addressing a critical bottleneck in current generative AI video research.
• The approach offers a blueprint for future video models to handle multi-dimensional constraints without sacrificing efficiency or quality.