Technical Summary: CineMobile

Problem Statement

The rapid advancement of Diffusion Transformers (DiTs) has enabled high-quality video generation, yet their deployment on mobile devices remains a significant challenge. State-of-the-art models, such as Wan2.1-14B, require substantial computational resources, large parameter counts, and multi-step iterative denoising processes (e.g., 40 steps), resulting in inference times of minutes even on high-end GPUs. This makes real-time, on-device image-to-video (I2V) generation infeasible. Furthermore, existing mobile acceleration techniques primarily target U-Net architectures, leaving DiT models largely unexplored. A critical constraint in mobile I2V is the need to preserve prompt semantics and fine-grained details (such as portrait identity) while achieving aggressive acceleration, a balance that is difficult to strike with standard pruning or quantization methods.

Methodology

CineMobile addresses these challenges through a three-fold optimization strategy designed to compress a DiT-based I2V model while maintaining cinematic motion quality. The framework focuses on three specific cinematic effects: bullet time, dolly zoom, and slow motion.

1. Structured Depth Pruning

To reduce model size without disrupting temporal modeling or identity preservation, CineMobile adapts the Pluggable Pruning with Contiguous Layer Distillation (PPCL) method.

• Depth vs. Width: Unlike width pruning, which alters hidden dimensions and can disturb pretrained temporal representations, CineMobile employs structured depth pruning. It identifies and removes redundant contiguous layers while keeping the backbone's hidden dimensionality unchanged.
• Process: The method uses residual linear probes and CKA similarity to detect redundant layer intervals on latent video sequences. Selected intervals are replaced by a single trainable surrogate block, initialized from the middle of the removed span.
• Distillation: The surrogate blocks are trained to reproduce the teacher model's hidden states at the span output, using portrait and motion data to ensure the retention of identity consistency and motion controllability.

2. Step Distillation via Adversarial Reinforcement Learning

To reduce the number of inference steps from 20+ to 4, CineMobile utilizes a two-stage fine-tuning and distillation pipeline:

• Supervised Fine-Tuning (SFT) Warm-up: Before distillation, the pruned student model undergoes a warm-up stage using flow-matching objectives. It is first fine-tuned on videos from the original dense model to restore generative capabilities, then adapted to target cinematic motion distributions using teacher-generated videos.
• Adversarial Distillation: Building on AdvDMD, the model is distilled into a 4-step generator. This stage integrates Reinforcement Learning (RL) via GRPO (Group Relative Policy Optimization).
  • A frozen "real" score estimator (teacher) and an online "fake" score estimator (student) provide distribution-matching signals.
  • A lightweight discriminator attached to the fake model provides adversarial realism feedback.
  • The discriminator's scores on intermediate latents serve as rewards for GRPO, refining the 4-step denoising trajectory to maximize visual quality and temporal consistency.

3. Hybrid Post-Training Quantization (PTQ)

To meet the strict memory constraints of mobile devices, CineMobile applies a hybrid precision quantization strategy:

• Activation: Kept at 16-bit precision for numerical stability.
• Weights: Most linear layers (attention, text/time embeddings) are quantized to 8-bit (FP8).
• Feed-Forward Networks (FFN): Given that FFNs constitute a large fraction of parameters, their linear projections are quantized to 4-bit.
• Result: This strategy compresses the model footprint to under 1 GB.

Key Results

CineMobile was evaluated on the MediaTek Dimensity 8400 Ultimate 5G platform and compared against the Wan2.1-14B teacher model and other commercial systems.

• Efficiency:

  • Speedup: CineMobile achieves a 40× speedup in generation time compared to the teacher model.
  • Latency: On the MediaTek Dimensity 8400, it generates 49-frame 480p videos with a per-step denoising latency of 20 seconds (total generation time is significantly reduced compared to the teacher's 240s on H200).
  • Compute: It reduces DiT-FLOPs by 71.89× compared to the teacher.
  • Memory: Peak memory usage on the mobile device is 1.8 GB, with the model footprint under 1 GB.

• Quality:

  • VBench Scores: CineMobile achieves scores comparable to the 14B teacher model across three cinematic scenarios (Bullet Time: 88.35 vs 89.27; Dolly Zoom: 89.30 vs 89.96; Slow Motion: 88.05 vs 88.51).
  • Human Evaluation: In comparisons with commercial models (Kling, Hailuo, Vidu), CineMobile demonstrated competitive overall perceptual quality and achieved the strongest performance in camera control, specifically in maintaining smooth orbital motion and subject-background consistency.
  • Ablation: The inclusion of GRPO refinement improved total scores by approximately 0.5 points, primarily enhancing aesthetic and imaging quality.

Significance and Claims

The paper claims that CineMobile bridges the gap between high-fidelity DiT-based video generation and practical mobile deployment. Its primary significance lies in demonstrating that:

1. Structured pruning combined with adversarial step distillation can drastically reduce inference steps and model size without sacrificing the complex temporal dynamics required for cinematic effects.
2. Hybrid quantization (4-bit/8-bit) enables the execution of large-scale video generation models on consumer-grade mobile hardware with memory footprints under 1 GB.
3. On-device cinematic generation is feasible, offering a practical solution for creating bullet time, dolly zoom, and slow motion videos directly on smartphones, preserving subject identity and scene consistency better than existing mobile-focused methods.

The authors position CineMobile not as a replacement for cloud-scale generation but as a viable, efficient alternative for on-device creation, specifically optimized for controllable camera motion rather than just static image quality.