Technical Summary: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation

1. Problem Statement

Long-horizon video generation, particularly in the Video-to-Video (V2V) regime, faces two primary, intertwined challenges: drift and continuity issues.

• Drift: As generation proceeds over time, errors compound, leading to a degradation in video quality (e.g., shifting colors, textures, and contrast).
• Continuity Issues: These manifest as a lack of object permanence or inconsistent rendering of transient content (e.g., an object reappearing with a different style or color).

Current solutions predominantly rely on autoregressive (AR) distillation, which converts bidirectional models into causal generators. While these methods attempt to stabilize sequential rollouts, they suffer from inherent limitations:

1. Horizon-Scaling Dependency: Errors in one chunk condition the next, causing drift to accumulate linearly with time.
2. Sequential Rollout: Generation must occur step-by-step, resulting in linear growth in wall-clock time regardless of available hardware.
3. Reset Artifacts: To manage memory, AR systems often perform KV-cache resets. These resets discard accumulated state, causing "jumpy" discontinuities in object identity and scene state.

Furthermore, existing long-form work focuses heavily on Text-to-Video (T2V). Long-form V2V, where a dense per-frame conditioning signal (e.g., pose, depth, edges) is known in advance, remains underexplored. The authors argue that if the full conditioning $c_{1:T}$ is known, insisting on causal (left-to-right) rollout is unnecessary.

2. Methodology: Anchored Tree Sampling (ATS)

The paper proposes Anchored Tree Sampling (ATS), a training-free, inference-time scheduler that replaces sequential AR rollout with a sparse-to-dense, anchor-bounded imputation strategy organized as a tree.

Core Mechanism

ATS utilizes a "two-sided" primitive (Equation 2), where a model generates a span $x_{a:b}$ conditioned on both a start anchor $A_a$ and an end anchor $A_b$, alongside the intermediate conditioning $c_{a:b}$. This allows the model to fill in content bidirectionally rather than causally.

The generation process is organized hierarchically:

1. Root Call (Sparse Anchors): A single call generates sparse "anchors" (keyframes or short clips) spanning the entire horizon $T$. In the static-camera regime, these anchors are derived directly from the dense conditioning signal without retraining.
2. Recursive Refinement (Intermediate Anchors): The tree is refined by generating new anchors between existing ones. If an interval is too large for the model's context window, intermediate anchors are inserted.
3. Leaf Generation (Dense Frames): The final step synthesizes dense frames between neighboring anchors. These "leaf" spans are bounded by anchors on both sides.

Key Properties

• Top-Down Factorization: The dependency graph is top-down (Root $\to$ Refinement $\to$ Leaves) rather than left-to-right.
• Parallelism: Sibling nodes at the same tree level are conditionally independent given their parent anchors. This allows for parallel execution across multiple GPUs.
• Training-Free: ATS does not modify the base model weights. It treats the pretrained bidirectional generator as a black-box primitive.
• Static-Camera Assumption: The current implementation assumes a static camera where sparse and dense conditioning look nearly identical, allowing the base model to produce clean sparse anchors "out-of-the-box."

Complexity and Latency

• Autoregressive (AR): Latency scales linearly with the number of chunks $K$ ($T_{AR} \approx K \cdot \tau_{call}$).
• ATS: Latency scales logarithmically with the horizon depth $L$ ($T_{tree} \approx (L+1) \cdot \tau_{call}$).
• Drift Mitigation: By bounding every generation step between two anchors, drift is contained within the span of the anchor pair rather than compounding across the entire video.

3. Experimental Results

The authors evaluated ATS on Wan 2.1 + VACE (a bidirectional V2V backbone) against two contemporary autoregressive baselines: LongLive and Reward Forcing.

Setup

• Task: 30-minute V2V generation across five conditioning modalities: inpainting, outpainting, edge, pose, and depth.
• Metrics: Aesthetic Quality (AQ) and Imaging Quality (IQ) from VBench-1.0, measured globally, as "chunk drift" (within-cache degradation), and as "reset jump" (discontinuity at cache resets).

Key Findings

1. Quality Improvement: ATS outperformed both AR baselines in global quality.
   • vs. LongLive: +4.0 AQ / +6.6 IQ.
   • vs. Reward Forcing: +5.7 AQ / +6.2 IQ.
2. Drift Elimination:
   • Chunk Drift: AR methods showed significant quality degradation within cache windows. ATS flattened this, maintaining consistent quality across chunks.
   • Reset Jump: AR methods exhibited large discontinuities at cache resets. ATS eliminated these jumps entirely.
3. Speed: Due to parallelization, ATS achieved 5.3x faster generation than the best AR baseline for a 2000-second video. It achieved faster-than-realtime performance (3.3x) with just 8 GPUs.
4. Long-Horizon Stability: The authors demonstrated stable generation of $\ge$40-minute videos on LTX-2.3 across all five modalities, with no observable horizon-scaling content drift.

4. Limitations and Future Directions

The paper explicitly acknowledges limitations in the current static-camera implementation:

1. Anchor Poisoning: A single poor anchor (e.g., at the root) corrupts its entire subtree. This is less critical for static cameras but problematic for dynamic cameras where sparse generation is harder.
2. Weak Motion Guidance: For weak motion signals (e.g., depth maps), single keyframes are insufficient; multi-frame video anchors are required.
3. Consistency Gaps: ATS guarantees temporal coherence within bounded spans but does not inherently enforce identity for objects that appear between anchors (off-camera entities) unless they are explicitly tracked.
4. Non-V2V Generalization: The current method relies on a dense source signal to generate sparse anchors. It does not yet generalize to pure Text-to-Video (T2V) or dynamic-camera regimes where the model must hallucinate sparse content without a dense guide.

Proposed Path Forward

The authors propose three structural extensions to generalize ATS:

1. Sparse-Generation Specialist: Introduce a dedicated model (similar to DCARL's keyframe generator) to produce sparse anchors for T2V and dynamic cameras, replacing the root call's reliance on dense conditioning.
2. Semantic DAG: Replace the strictly temporal tree with a Directed Acyclic Graph (DAG) indexed by semantic constraints (e.g., character identity, scene changes) rather than just time, allowing non-adjacent leaves to share constraints.
3. Reference-Based Conditioning: Leverage explicit reference inputs (images/clips) to enforce continuity across the entire duration, sidestepping the drift inherent in suffix-based memory.

5. Significance

The paper claims that ATS represents a significant shift in long-horizon generation strategy. By decoupling the scheduling problem from the modeling problem, ATS achieves:

• Drift Prevention: Converting horizon-compounding drift into anchor-bounded drift.
• Continuity: Eliminating the "jumpy" artifacts caused by AR cache resets.
• Efficiency: Reducing the critical path from linear to logarithmic, enabling massive speedups through parallelism.
• Scalability: Demonstrating the first non-avatar framework capable of coherent, drift-free generation approaching the hour scale (40+ minutes) without retraining the base model.

The work concludes that while currently optimized for static-camera V2V, the tree-based, bidirectional paradigm offers a robust foundation for extending long-form generation to dynamic and text-driven regimes.