SPDMark: Selective Parameter Displacement for Robust Video Watermarking

1. Problem Statement

The rapid advancement of high-quality video generation models (e.g., Sora, Stable Video Diffusion) has created an urgent need for robust watermarking to track the provenance of AI-generated content and prevent misuse. Existing video watermarking methods face a "trilemma" where they struggle to simultaneously achieve:

1. Imperceptibility: The watermark must not degrade visual quality or temporal coherence.
2. Robustness: The watermark must survive common attacks (compression, cropping, frame drops, reordering).
3. Computational Efficiency: The method should not add significant latency or require heavy retraining.

Limitations of Current Approaches:

• Post-hoc methods: Add latency and cannot leverage generative priors.
• Noise-space methods (e.g., Tree-Ring): Rely on DDIM inversion, which is computationally expensive and fragile against perturbations.
• Model fine-tuning methods: Often apply uniform modulation or single fixed signatures, limiting the ability to detect temporal manipulations (like frame swaps) and often harming video quality.

2. Methodology: SPDMark

SPDMark (Selective Parameter Displacement) is an in-generation watermarking framework. Instead of perturbing pixels or diffusion noise, it embeds watermarks by selectively displacing the parameters of the generative model itself.

Core Framework

• Selective Parameter Displacement (SPD): The method modifies a subset of the generative model's parameters ($\Phi$) based on a watermarking key ($\kappa$). The displaced model is defined as $\Phi + \Delta\Phi$, where $\Delta\Phi$ encodes the key.
• Layer-wise Basis Shifts: To make the problem tractable, the parameter displacement $\Delta\Phi$ is modeled as an additive composition of learned low-rank basis shifts.
  • The generative model's decoder is split into $L$ layers.
  • Each layer $\ell$ has a dictionary of $P$ basis shifts $\{\zeta_{\ell,p}\}$.
  • A binary mask $b(\kappa)$, derived from the watermark key, selects which basis shifts to activate in each layer.
  • Formula: $\Delta\phi_\ell = \sum_{p=1}^{P} b_{\ell,p} \zeta_{\ell,p}$.
• Low-Rank Adaptation (LoRA): To ensure parameter efficiency, the basis shifts $\zeta$ are implemented using LoRA (low-rank matrix decomposition, $A \times B$ where rank $r \ll d$). This allows the system to support arbitrary watermark keys without retraining the entire model; the key simply selects a different combination of pre-learned shifts.

Per-Frame Watermarking & Temporal Alignment

• Unique Messages per Frame: Unlike methods that embed a single signature for the whole video, SPDMark generates a unique watermark message $\kappa_t$ for every frame $t$. This is done using a cryptographic hash function (e.g., HMAC-SHA256) on a base key and the frame index.
• Extraction & Verification:
  • Extractor: A lightweight ResNet-50 extracts the message from each frame independently.
  • Temporal Robustness: Since attacks may drop, swap, or insert frames, the extracted messages may be misaligned. SPDMark uses Maximum Bipartite Matching (Hungarian algorithm) to align the extracted messages with the expected sequence based on Hamming distance.
  • Statistical Hypothesis Testing: A statistical test determines if the alignment is valid (distinguishing true watermarks from random noise) at both the frame and video levels.

Training Objective

The system jointly learns the basis shifts ($\zeta$) and the watermark extractor ($V_\eta$) by minimizing a combined loss function:

1. Message Recovery Loss: Binary cross-entropy to ensure the extractor recovers the correct bits.
2. Imperceptibility Loss:
   • Perceptual Similarity (LPIPS): Ensures the watermarked video looks like the original.
   • Temporal Consistency: Minimizes flickering by ensuring smooth luminance transitions between consecutive frames.

3. Key Contributions

1. SPDMark Framework: A novel in-generation scheme using Selective Parameter Displacement to embed multi-key watermarks in video diffusion models without retraining for each key.
2. LoRA-based Implementation: Practical realization using low-rank basis shifts, enabling efficient, frame-specific watermarking with negligible inference overhead.
3. Temporal Forensics: A mechanism to embed unique messages per frame and use bipartite matching to detect and localize temporal manipulations (frame drops, swaps, insertions).
4. Comprehensive Evaluation: Demonstrated robustness across text-to-video and image-to-video models against a wide range of attacks (photometric, geometric, temporal, and post-processing).

4. Experimental Results

The authors evaluated SPDMark on two models: Stable Video Diffusion (SVD-XT) and ModelScope.

• Watermark Extraction Accuracy:
  • Achieved 99.5% bit accuracy on SVD-XT and 98.8% on ModelScope.
  • Outperformed or matched state-of-the-art baselines (VideoShield, VideoSeal, VidSig) in extraction accuracy.
• Video Quality (Imperceptibility):
  • SPDMark achieved the highest Subject Consistency (SC) and Motion Smoothness (MS) scores among all tested methods, indicating that the parameter displacement preserves the underlying temporal structure better than uniform modulation methods.
  • Visual comparisons showed no noticeable artifacts.
• Robustness:
  • Average Robustness: 93.5% (SVD-XT) and 93.9% (ModelScope) across 16 different attack types.
  • Temporal Attacks: Excelled in handling frame drops, swaps, and insertions, achieving near-perfect alignment recovery (F1 > 0.99).
  • Geometric/Photometric: Significantly outperformed noise-space methods (like VideoShield) in robustness against cropping, rotation, and color jitter.
• Ablation Studies:
  • Increasing payload capacity (more bits per frame) slightly reduced accuracy under fixed training budgets, but additional training steps recovered most of the lost robustness.
  • The method is robust to variations in diffusion steps and guidance scales.

5. Significance

SPDMark represents a significant shift in video watermarking by moving away from noise-space perturbations or post-hoc processing toward parameter-space manipulation. Its significance lies in:

• Scalability: It supports multi-key watermarking without per-key retraining, making it viable for large-scale deployment.
• Forensic Capability: By embedding unique messages per frame, it enables the detection of how a video was tampered with (e.g., identifying exactly which frames were swapped or dropped), a capability missing in many existing schemes.
• Efficiency: It avoids the computational cost of DDIM inversion and adds negligible overhead to the generation process.
• Practicality: It offers a balanced trade-off between detection accuracy, video quality, and robustness, addressing the critical need for provenance tracking in the era of generative AI.