Towards Robust Content Watermarking Against Removal and Forgery Attacks

1. Problem Statement

The rapid advancement of text-to-image diffusion models (e.g., Stable Diffusion, DALL-E) has created urgent needs for verifying image provenance and copyright attribution. While content watermarking has emerged as a solution, existing methods (such as Tree-Ring, RingID, and Gaussian-Shading) suffer from critical vulnerabilities:

• Static Nature: Most current methods inject a fixed, static watermark pattern (e.g., a specific ring in the Fourier space) regardless of the input prompt. This static nature leaks structural information to adversaries.
• Vulnerability to Adversarial Attacks:
  • Removal Attacks: Attackers can optimize watermarked images to remove the specific static pattern, causing detection failure.
  • Forgery Attacks: Attackers can use a single leaked watermarked image (or an average of many) to infer the static pattern and forge watermarks onto benign, non-watermarked images.
• One-Sided Detection: Existing detection methods often rely on a "one-sided" metric (comparing the latent against a positive pattern), which allows attackers to flip the latent representation to evade detection.

2. Methodology: ISTS (Instance-Specific watermarking with Two-Sided detection)

The authors propose ISTS, a novel paradigm designed to resist both removal and forgery attacks by introducing dynamism and robustness into the generation and detection pipeline.

A. Instance-Specific Watermarking (Dynamic Injection)

Instead of using a fixed pattern for all images, ISTS customizes the watermark for every generation instance based on the input prompt's semantics.

1. Semantic Feature Extraction: For a given prompt, the system first generates a non-watermarked image and encodes it into a semantic feature vector using a CLIP encoder.
2. Dynamic Parameter Selection: A pretrained Semantic-Based Parameter Selector maps these semantic features to specific watermarking parameters:
   • Injection Time ($t$): The specific timestep in the diffusion process where the watermark is injected.
   • Injection Location ($l$): The specific coordinate in the frequency domain where the pattern is added.
3. Watermark Injection: The system performs partial denoising to step $t$, adds the watermark pattern at location $l$ in the frequency domain, and continues the denoising process to generate the final image.
   • Benefit: Since the parameters depend on the prompt's semantics, an attacker cannot predict the injection time or location without knowing the specific semantic mapping, making it impossible to use a single reference image to forge or remove watermarks effectively.

B. Two-Sided Detection

To counter removal attacks that attempt to flip the latent representation, ISTS introduces a two-sided detection metric.

• Mechanism: Instead of only checking for the presence of the pattern $W$ (i.e., $|W - F(z)|$), the detector checks for both the pattern and its inverse (i.e., $|W + F(z)|$).
• Metric: The detection score is the minimum of the distances to the positive pattern and the negative pattern:
  $$d = \min \left( \frac{1}{|M|}\sum |W_i - F(z_T)_i|, \frac{1}{|M|}\sum |W_i + F(z_T)_i| \right)$$
• Benefit: This prevents attackers from simply inverting the latent space to bypass detection, as the detector will still recognize the inverted pattern.

3. Key Contributions

1. Novel Paradigm (ISTS): The first framework to combine instance-specific dynamic injection (time and location) with two-sided detection to simultaneously address removal and forgery attacks.
2. Semantic-Driven Dynamism: The use of a semantic selector ensures that watermark parameters are tightly coupled with image content, breaking the correlation between different images that attackers rely on for averaging attacks.
3. Robust Detection Scheme: The introduction of two-sided detection effectively neutralizes attacks that rely on inducing opposite latent representations.
4. Comprehensive Evaluation: The paper provides extensive experiments against three representative removal attacks (Imprinting, Averaging, VAE) and three forgery attacks, demonstrating state-of-the-art performance.

4. Experimental Results

The authors evaluated ISTS against baselines like Tree-Ring, Gaussian-Shading, RingID, ROBIN, and SEAL on Stable Diffusion 2.1.

• Against Removal Attacks:

  • Existing methods (e.g., Tree-Ring, RingID) saw their detection AUC drop significantly (often below 0.3) under removal attacks.
  • ISTS maintained high robustness, achieving an AUC of 0.821 against Imprinting Removal (compared to 0.267 for Tree-Ring) and 0.990 against Averaging Removal.
  • In worst-case scenarios, ISTS achieved an average AUC of 0.936, significantly outperforming all baselines.

• Against Forgery Attacks:

  • Lower AUC indicates better resistance (harder to forge).
  • Most baselines were easily forged (AUC > 0.95).
  • ISTS achieved a significantly lower AUC of 0.634 against Imprinting Forgery and 0.474 against Averaging Forgery, demonstrating superior resistance.

• Image Quality & Distortion Robustness:

  • ISTS maintained high image quality (PSNR, SSIM, CLIP-Score) comparable to ROBIN and better than SEAL.
  • It demonstrated strong robustness against common image distortions (rotation, noise, blur, cropping, JPEG) and diffusion purification, outperforming Gaussian-Shading and SEAL in these areas.

• Ablation Study:

  • Removing dynamic patterns or dynamic injection steps significantly degraded performance against forgery and removal attacks, confirming the necessity of both components.
  • Removing two-sided detection reduced robustness against Imprinting Removal attacks.

5. Significance

This paper addresses a critical gap in the deployment of generative AI: the fragility of current watermarking schemes against determined adversaries. By moving away from static, one-size-fits-all watermarking to a dynamic, instance-specific approach, ISTS significantly raises the barrier for attackers.

• Practical Impact: It offers a more viable solution for copyright protection and provenance tracking in real-world scenarios where images may be leaked or maliciously altered.
• Theoretical Insight: It highlights that static watermarking patterns inherently leak information that facilitates attacks, and that detection mechanisms must account for potential adversarial manipulation of latent spaces (via two-sided detection).

While the authors note that worst-case robustness is not yet perfect for all attack types, ISTS represents a state-of-the-art advancement in making content watermarking resilient to both removal and forgery.