Technical Summary: Motion Attribution for Video Generation (Motive)

Problem Statement

Despite rapid advancements in video generation models, the specific role of training data in shaping motion (temporal dynamics) remains poorly understood. Existing data attribution methods, primarily designed for image generation, focus on static visual appearance. When naively applied to video, these methods collapse motion into appearance, failing to distinguish between static backgrounds and dynamic trajectories. This creates a gap in understanding which training clips influence temporal coherence, physical plausibility, and specific motion patterns (e.g., velocity, acceleration, deformation). Furthermore, video attribution faces unique challenges: the high computational cost of processing long sequences, biases introduced by variable clip lengths, and the difficulty of isolating temporal relations from spatial content.

Methodology: Motive Framework

The authors introduce Motive (MOTIon attribution for Video gEneration), a gradient-based data attribution framework designed to isolate motion-specific influence from static appearance. The framework operates in the fine-tuning regime, aiming to identify which training clips most strongly influence the temporal dynamics of a generated query video.

Core Components

1. Scalable Gradient Computation:

   • To handle billion-parameter models and large datasets, Motive avoids exact Hessian inversion. Instead, it approximates influence via gradient similarity using an identity preconditioner.
   • Efficiency Strategies: It employs a single-sample variant (fixing a single timestep $t_{fix}$ and noise draw $\epsilon_{fix}$) to reduce variance while maintaining ranking stability. It further reduces storage costs using a Fastfood projection (Johnson–Lindenstrauss transform) to project high-dimensional gradients into a lower-dimensional space ($D' = 512$) before computing cosine similarity.
   • Frame-Length Bias Correction: Raw gradient magnitudes correlate with video duration. Motive normalizes gradients by the number of frames ($F$) prior to projection to ensure fair attribution across clips of varying lengths.

2. Motion-Aware Weighting (Motion Masking):

   • Unlike standard attribution which treats a video as a single unit, Motive reweights gradients based on local motion.
   • Motion Detection: The framework uses AllTracker to extract optical flow and displacement vectors in pixel space.
   • Latent Alignment: Motion magnitudes are downsampled to align with the model's latent grid (where gradients are computed).
   • Motion-Weighted Loss: A motion magnitude mask ($W$) is applied to the loss function. This suppresses gradients from static regions (backgrounds, textures) and emphasizes gradients from dynamic regions (moving objects, camera motion). The resulting influence score traces the generated motion back to training clips that contributed to those specific dynamic behaviors.

3. Data Selection Strategy:

   • For a given query video, Motive computes motion-aware influence scores for all training samples.
   • Subset Selection: To select a fine-tuning subset, the top-$K$ scoring samples are chosen. For multiple queries, a majority voting mechanism aggregates scores, selecting samples that are consistently influential across different motion queries.

Key Contributions

1. Scalable Attribution for Video: A computationally efficient, gradient-based attribution approach that scales to modern, large-scale video datasets and generative models, utilizing single-sample estimators and structured projections.
2. Frame-Length Bias Correction: A novel normalization technique that corrects for the bias where longer videos artificially receive higher influence scores due to accumulated gradient magnitudes.
3. Motion-Centric Attribution: The first framework to explicitly isolate temporal dynamics from static appearance in video generative models using motion-weighted loss masks, enabling the tracing of specific motion patterns to training data.
4. Empirical Validation: Demonstration that using Motive-selected data (only 10% of the dataset) improves motion smoothness and dynamic degree on VBench, outperforming both random selection and motion-unaware attribution baselines.

Experimental Results

The framework was evaluated on text-to-video models Wan2.1-T2V-1.3B and Wan2.2-TI2V-5B, as well as LTX-2B, using the VIDGEN-1M and 4DNeX-10M datasets.

• Quantitative Performance (VBench):
  • Motive achieved the highest Dynamic Degree scores (47.6% on Wan2.1 and 48.3% on Wan2.2), significantly outperforming random selection (41.3%) and whole-video attribution (43.8%).
  • Notably, Motive surpassed full-dataset fine-tuning on dynamic degree metrics while using only 10% of the data.
  • It maintained competitive scores in Motion Smoothness, Subject Consistency, and Aesthetic Quality.
• Human Evaluation:
  • In pairwise comparisons with 17 annotators, Motive-guided models achieved a 74.1% win rate against the pre-trained base model and a 53.1% win rate against full fine-tuning models.
• Ablation Studies:
  • Projection Dimension: A projection dimension of 512 provided the best trade-off between accuracy (Spearman correlation $\rho \approx 74.7\%$ with full gradients) and efficiency.
  • Frame-Length Normalization: Removing this correction caused rankings to correlate strongly with video length ($\rho=78.0\%$) rather than motion quality. Normalization reduced spurious length correlations by 54%.
  • Motion Masking: Models fine-tuned with motion-aware selection outperformed those using whole-video attribution, confirming that isolating motion signals is critical for improving temporal dynamics.

Significance and Claims

The paper claims to present the first framework to attribute motion rather than visual appearance in video generative models and to utilize this attribution for curating fine-tuning data.

The authors emphasize that while large-scale data scaling has driven progress in video generation, the specific influence of data on motion has been a "black box." Motive provides a direct lens to understand why models move the way they do. By isolating temporal dynamics, the framework enables:

• Targeted Data Curation: Selecting a small fraction of high-influence data to significantly improve motion quality and physical plausibility.
• Diagnosis of Failure Modes: Identifying which training clips lead to specific motion artifacts or inconsistencies.
• Efficiency: Achieving performance that matches or exceeds full-dataset fine-tuning with a fraction of the data, making high-quality motion generation more accessible.

The work concludes that as video models scale, data-level understanding of motion will be essential for building more controllable and physically plausible generative systems.