💻 computer science

Dual-Branch Adaptive Diffusion and Motion-Guided Temporal Alignment for Robust Invisible Video Watermarking

This paper proposes a dual-branch adaptive diffusion and motion-guided temporal alignment framework that effectively resolves the trade-off between imperceptibility and robustness in invisible video watermarking by combining content-adaptive local diffusion with coordinate-aware global anchoring to achieve superior performance against local cropping and other attacks.

Original authors: Zhanglei Huang, Chenming Yao, Jianfeng Lu, Qihao Liang, LI LI, Zhiheng Zhang

Published 2026-09-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhanglei Huang, Chenming Yao, Jianfeng Lu, Qihao Liang, LI LI, Zhiheng Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast, flowing river of digital media, video has become the primary way we share stories, news, and memories. Yet, this ease of sharing brings a persistent problem: once a video leaves its creator's hands, it is incredibly difficult to prove who made it or where it came from. If someone edits a clip, crops out a corner, or compresses it for social media, the original owner's claim can vanish. To solve this, scientists have long tried to hide secret messages inside videos, a practice known as invisible watermarking. The goal is to embed a digital fingerprint that is invisible to the human eye but can be retrieved later to verify ownership. However, this task is a delicate balancing act. If the hidden message is too strong, it distorts the picture; if it is too weak or placed in only one spot, a simple cut or crop can erase it entirely. The challenge has been to make the watermark strong enough to survive these attacks without ruining the visual experience.

A team of researchers from Hangzhou Dianzi University and a technology company in Beijing has proposed a new approach to this problem, one that treats the video not as a static image but as a moving, breathing sequence of events. Instead of trying to hide the message in a single, fixed location or spreading it thinly across the entire screen, they developed a system that uses two different strategies working together. Imagine the video as a landscape; the researchers decided to hide parts of the message in the complex, textured areas where the eye is less likely to notice small changes, while simultaneously planting invisible signposts across the entire timeline. This dual strategy allows the system to recover the message even if a large chunk of the video is cut away, a scenario that has historically broken other methods.

The core of their innovation is a framework that adapts to the content of the video itself. The first part of their system acts like a careful guide, analyzing the video frame by frame to find areas rich in detail, such as the leaves of a tree or the fabric of a shirt. It gently pushes the hidden message into these busy regions, where the human eye is naturally distracted and less likely to spot the subtle alterations. This ensures the video looks exactly the same to a viewer. The second part of the system takes a broader view, embedding a coordinate-based map that tells the computer where every part of the video belongs in time and space. This global map acts as a safety net. If someone crops the video, removing the textured areas where the message was hidden, the system can still use the remaining coordinate clues to reconstruct the missing pieces of the secret message.

To handle the fact that videos move and change, the researchers added a third layer of intelligence that pays attention to motion. They realized that video compression, which is used to shrink files for streaming, often struggles to keep track of moving objects. By using a technique that tracks how pixels move from one frame to the next, their system learns to hide the message in ways that are resistant to these compression tricks. It focuses on the areas where motion is most active, ensuring the watermark survives the journey from a high-quality file to a compressed stream. Furthermore, they used a specialized network structure that filters out the background noise, ensuring the hidden message is only placed where it will be most stable and least noticeable.

The team tested their method on two large collections of video clips, one featuring human actions and another with scenes from movies. They subjected the watermarked videos to a battery of harsh treatments, including random cropping, blurring, adding noise, and heavy compression. The results showed that their system maintained a high level of visual quality, with the watermarked videos looking nearly identical to the originals. More importantly, when they tried to extract the hidden message after these attacks, the system succeeded in recovering the correct information in nearly 98 percent of cases on average. This was a significant improvement over previous methods, particularly when it came to surviving random cropping, where older systems often failed to retrieve the message at all.

The researchers found that their dual-branch approach successfully resolved the long-standing trade-off between hiding a message well and making it hard to destroy. By combining a local strategy that hides the message in busy textures with a global strategy that keeps track of the video's structure, they created a watermark that is both invisible and resilient. The study suggests that this method could be a practical tool for protecting copyright in an era where videos are constantly edited, shared, and compressed. While the system was tested on short clips and specific types of attacks, the results indicate a promising path forward for securing digital video content without sacrificing the quality that viewers expect.

Technical Summary: Dual-Branch Adaptive Diffusion and Motion-Guided Temporal Alignment for Robust Invisible Video Watermarking

Problem Statement
Robust invisible video watermarking is critical for copyright protection and source tracing in large-scale video distribution. While existing deep learning-based schemes have improved resistance to common video operations, they face a persistent trade-off between imperceptibility and robustness, particularly under local cropping. Global redundant embedding strategies enhance crop tolerance but often introduce significant visual distortion by modifying too many pixels. Conversely, local embedding preserves visual quality but fails when the retained video region is incomplete, leading to a loss of synchronization. Furthermore, video-specific challenges such as H.264/AVC compression, frame dropping, and temporal misalignment require algorithms that can maintain synchronization across frames without sacrificing visual fidelity.

Methodology
The paper proposes a dual-branch adaptive diffusion framework designed to decouple watermark preprocessing into complementary paths, integrated with motion-guided temporal alignment and an Attention U-Net architecture.

Dual-Branch Watermark Preprocessing:
- Content-Adaptive Local Diffusion Path: This branch iteratively diffuses the watermark message into texture-rich regions of the video. Using a 3D convolutional feature extractor and iterative residual interpolation, the method guides watermark residuals toward areas where perturbations are less perceptible, thereby enhancing imperceptibility.
- Coordinate-Aware Global Path: Inspired by implicit neural representations, this branch generates global watermark features based on normalized spatial-temporal coordinates $(t, h, w)$ . These features act as spatial-temporal anchors, allowing the decoder to recover the watermark from surviving regions even after partial frame cropping.
- Fusion: The local and global features are fused via a learnable weight, providing the encoder with both content-dependent invisibility cues and global synchronization information.
Motion-Guided Temporal Feature Alignment (MGTA):
To address temporal synchronization issues caused by compression and frame-domain distortions, the MGTA block utilizes LiteFlowNet3 to estimate dense optical flow. It performs backward mapping to align non-reference frames with a reference frame using differentiable bilinear sampling. Additionally, a motion-aware attention mechanism uses the L2 norm of the optical flow field to identify motion-rich regions, which are then prioritized for embedding. This simulates motion compensation processes found in video coding, helping the encoder learn residuals that survive compression artifacts.
Encoder and Decoder Architecture:
- Encoder: The system employs an Attention U-Net backbone. Unlike standard U-Nets, attention gates are used to filter shallow encoder features using deep decoder features as gating signals. This suppresses irrelevant background noise and prevents watermark residuals from leaking into smooth, visually sensitive regions.
- Decoder: A high-dimensional temporal feature classifier processes the distorted video (after MGTA alignment) through 3D convolutional layers and an MLP to recover the binary watermark message.
- Training Strategy: A differentiable noise attack layer simulates various distortions, including frame averaging, dropping, swapping, random cropping, Gaussian noise, filtering, and H.264/AVC compression (using a differentiable DiffH264 module). The training objective combines message recovery loss, adversarial loss (for visual realism), mask-guided loss, and perceptual loss (LPIPS), with a dynamic weighting schedule to balance robustness and imperceptibility.

Key Contributions

Dual-Branch Preprocessing: A novel module that fuses content-adaptive local diffusion with coordinate-aware global anchoring, effectively balancing imperceptibility and robustness against random cropping.
Motion-Guided Alignment: A temporal feature alignment block combining LiteFlowNet3-based optical flow with motion-aware attention, enabling the encoder to focus on motion-salient and compression-resistant regions.
Attention U-Net Integration: The use of attention gates to refine skip-connected features, reducing watermark leakage into smooth background areas.
Comprehensive Evaluation: Extensive experiments demonstrating the method's efficacy across frame-domain, signal-processing, cropping, and compression attacks.

Experimental Results
The method was evaluated on the Kinetics-400 and Hollywood2 datasets.

Imperceptibility: On Kinetics-400, the proposed method achieved a Peak Signal-to-Noise Ratio (PSNR) of 37.85 dB and an LPIPS of 5.36, outperforming REVMark and DVMark in overall visual quality.
Robustness: The method achieved an average bit accuracy of 97.69% across seven representative attacks on Kinetics-400. Notably, it improved random crop attack accuracy by approximately 5.50 percentage points over REVMark (88.81% vs. 83.31%), validating the efficacy of the global coordinate path.
Compression Resistance: Under H.264/AVC compression (CRF=22), the method achieved 96.20% accuracy, outperforming both REVMark and DVMark.
Generalization: The model maintained high performance on the Hollywood2 dataset (97.64% average accuracy), indicating strong cross-domain generalization despite differences in lighting and cinematography.

Significance and Claims
The paper claims that the proposed framework successfully addresses the central trade-off between imperceptibility and robustness in video watermarking. By decoupling the embedding strategy into local texture adaptation and global coordinate anchoring, the method avoids the fragility of purely local embeddings while preventing the excessive distortion of dense global tiling. The integration of motion-guided alignment specifically targets the synchronization challenges posed by H.264/AVC compression. The authors assert that the framework offers a promising direction for practical video copyright protection and source tracing in complex distribution environments, demonstrating that the method has not merely memorized training set statistics but has learned robust features applicable to diverse video content.

Limitations
The authors acknowledge several limitations: the evaluation was conducted on 128×128, 8-frame clips rather than full-resolution long-form videos; some baseline comparisons were unavailable due to missing source code or metrics; and the current scope covers common attacks but excludes more complex real-world scenarios such as social-platform transcoding, screen recording, and adaptive watermark removal.

More like this