ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

Technical Summary: ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

Problem Statement
Test-Time Tuning (TTT) has emerged as a powerful paradigm for video editing, aiming to adapt pre-trained diffusion models to specific user-provided source videos to improve appearance, structure, and motion preservation. However, the authors identify a foundational mismatch between the distribution-mapping nature of generative diffusion models and the single-point optimization inherent in standard TTT. This mismatch triggers a phenomenon termed Prior Collapse, a degenerate state where the model discards text conditions and spatial latents, collapsing generations to the source video or entangling features of distinct regions.

The paper characterizes Prior Collapse through two complementary failure modes:

1. Conditioning Collapse: The conditional guidance becomes anchored to the source-conditioned dynamics during tuning. The model gradually weakens text condition pathways, ignoring the target editing instruction and failing to follow new semantic guidance.
2. Spatial Entanglement: Spatial representations become globally coupled during denoising as the model attempts to memorize the global target. This prevents localized control, causing unintended modifications outside the target region and visual corruption.

Existing interventions, such as Low-Rank Adaptation (LoRA), prior-preservation losses, or attention map modifications, are described as sub-optimal, often relying on delicate architectural tuning, auxiliary datasets, or remaining sensitive to hyperparameters. Furthermore, transferring image-based TTT techniques to Video Diffusion Transformers (DiT) without adaptation severely accelerates Prior Collapse due to the complex spatio-temporal priors of video models.

Methodology: ElasticTTT
To resolve the optimization-sampling tension and rescue generative elasticity, the authors propose ElasticTTT, a framework that intervenes across the entire generative pipeline through three novel components:

1. Target Distribution Regularization (TDR):

   • Goal: Prevent Conditioning Collapse during the optimization (training) phase.
   • Mechanism: Instead of optimizing against a rigid, deterministic Dirac delta distribution (the single source video), TDR injects controlled stochasticity into the optimization target. Specifically, the target velocity vector is computed using a perturbed version of the source latent ($\tilde{x}_0 = x_0 + \epsilon_{reg}$), while the input trajectory remains clean.
   • Effect: This replaces the sharp singularity with a continuous local Gaussian manifold. Theoretically, this introduces a geometry-aware variance penalty (Gauss-Newton penalty) into the loss landscape, preventing the model from "lazily" memorizing an infinitely sharp spatial mapping. It forces the model to retain its elastic denoising priors while still converging to the correct expected vector field.

2. Contrastive Classifier-Free Guidance (Contrastive CFG):

   • Goal: Resolve the residual distribution shift and decouple source concepts during the inference (sampling) phase.
   • Mechanism: Standard CFG steers generation using a negative prompt. ElasticTTT extends this to a tri-directional formulation. It explicitly contrasts the target condition ($C_{trg}$) against the source condition ($C_{src}$) used during tuning. The guidance formula includes a term that subtracts the source-conditioned prediction: $v_{\theta}(x_t, t, C_{trg}) - v_{\theta}(x_t, t, C_{src})$.
   • Effect: This actively repels the sampling trajectory away from the "source attractor bias" (the energy well created by TTT), ensuring crisp semantic transitions and restoring the model's ability to follow new instructions without being pulled back to the source video's concepts.

3. Asynchronous Noise Scheduling (Async-NS):

   • Goal: Mitigate Spatial Entanglement during inference.
   • Mechanism: The authors argue that temporal uniformity exacerbates entanglement. Async-NS assigns distinct noise levels and time embeddings to edited regions versus anchored (unedited) regions. A mask defines the edited areas; the edited regions follow a high-noise schedule ($T_e$) to allow flexibility for new concepts, while preserved regions follow a low-noise schedule ($T_p$) to retain maximum source information.
   • Effect: This physically breaks the entanglement of representations by forcing the network to process regions with distinct noise regimes and timestep embeddings. It grants the model localized freedom to overwrite target concepts while safely guiding unedited surroundings back to their physical state, avoiding the need for invasive attention map modifications.

Key Results
The paper evaluates ElasticTTT on a custom testset of 125 text-video pairs covering five task types (subject editing, background editing, subject addition, color adjustment, and general editing), using the Wan2.1 1.3B model as the backbone.

• Quantitative Performance: ElasticTTT achieves state-of-the-art (SOTA) performance across multiple metrics, including VBench (visual quality/temporal consistency), VEBench, and CLIP-T (semantic alignment). It outperforms both training-free methods (e.g., UniEdit, Flow-align) and other TTT-based approaches (e.g., Tune-A-Video, VidTTA). Notably, it achieves high scores in Instruction Adherence (IA) and Source Preservation (SP) simultaneously, whereas baselines often suffer from trade-offs (e.g., high preservation but low instruction adherence).
• Human Evaluation: In a study with 10 participants, ElasticTTT received the highest average scores (3.60/5.0), particularly excelling in Video Quality (3.87) and Instruction Adherence (3.32).
• Ablation Studies: Removing any of the three core components (TDR, Contrastive CFG, or Async-NS) leads to significant performance degradation. Specifically, without TDR, the model fails to add new concepts; without Contrastive CFG, edits are subtle and indistinguishable; without Async-NS, the model fails to isolate target regions, causing simultaneous modification of foreground and background.
• Scalability: The method generalizes effectively to larger base models (Wan2.2-5B), showing even more pronounced improvements in instruction adherence and video quality compared to the 1.3B model.

Significance and Claims
The paper claims that ElasticTTT represents a principled solution to the fundamental tension between instance-specific overfitting (required for TTT) and the stochastic, distribution-preserving nature of diffusion models. By demonstrating that Prior Collapse is a distinct failure mode from standard overfitting, the authors argue that simply preventing overfitting is insufficient; one must actively preserve the "generative elasticity" of the prior.

The significance of the work lies in:

1. Systematic Resolution: It provides a unified framework that addresses both Conditioning Collapse and Spatial Entanglement, two failure modes previously treated with disparate and often sub-optimal heuristics.
2. Theoretical Insight: It offers a mathematical analysis of how TDR injects a geometry-aware penalty to prevent the optimizer from getting trapped in sharp minima, and how Contrastive CFG algebraically cancels the TTT-induced distribution shift.
3. Practical Impact: It achieves SOTA performance in one-shot video editing without requiring external paired datasets or heavy training, making it a viable data engine for synthesizing paired editing data for future large-scale models.

The authors conclude that their work is an important step toward more reliable diffusion-based models and highlights promising directions for designing personalized AI systems that balance instruction adherence with source fidelity.