Do Modern Post-Hoc Watermarking Methods Beat Broken-Arrows?

Technical Summary: Do Modern Post-Hoc Watermarking Methods Beat Broken-Arrows?

Problem Statement
The rapid proliferation of generative AI, particularly diffusion models, has necessitated robust mechanisms for identifying AI-generated content. While regulatory bodies (e.g., EU AI Act, White House executive orders) mandate traceability, a gap exists in the comparative evaluation of modern post-hoc watermarking methods against classic techniques. Modern approaches, which leverage deep neural networks (DNNs) for embedding and decoding, are often assumed to be superior due to their flexibility and ability to handle complex augmentations during training. However, these methods lack the theoretical security foundations of classical schemes, such as the concept of a secret key, and inherit vulnerabilities to adversarial examples. This paper investigates whether modern post-hoc methods consistently outperform classical approaches in a realistic zero-bit detection setting, prioritizing security and robustness over the high-capacity multi-bit payloads often targeted by modern schemes.

Methodology
The authors propose a fair comparison framework evaluating robustness, imperceptibility, and security across three watermarking schemes: two modern post-hoc methods (Videoseal and TrustMark) and one classical method (Broken-Arrows).

• Zero-Bit Conversion: To ensure a fair comparison focused on traceability rather than communication, multi-bit modern decoders were converted into zero-bit detectors. This was achieved by treating the decoder's projection into a subspace as a hypercone detector, comparing the extracted vector against a secret vector using the angle between them.
• Experimental Setup: Experiments utilized 200 natural images (1024×1024) from the MFlickr dataset. A fixed imperceptibility constraint was enforced by scaling the watermark signal to achieve a PSNR of 42dB relative to the original image.
• Security Scenarios: The evaluation tested four attacker knowledge levels:
  1. White-box: Full access to the detector (and key for Broken-Arrows). Attacks included the DDN boundary projection attack for DNNs and optimal closed-form attacks for Broken-Arrows.
  2. Black-box: Access to the detector only via query/response. The CGBA attack was used with a fixed query budget.
  3. Oracle: The attacker uses a proxy (oracle) to guide the attack. Two methods were tested: WmForger (using a preference model) and a modified Watermark In the Sand (WIS) attack (using VAE purification and PSNR as an oracle).
  4. Blind: No access to the detector or oracle. This included classic value-metric transformations (e.g., JPEG compression) and regeneration attacks using diffusion models (SANA, Stable-Diffusion-2.1).
• Metrics: Performance was measured by the Attack Success Rate (ASR) at a fixed false-alarm probability ($\alpha = 10^{-6}$) and the distortion (PSNR) required to achieve success.

Key Contributions

1. Unified Evaluation Framework: The paper establishes a methodology to compare modern and classical watermarking under a zero-bit assumption, converting multi-bit modern decoders into hypercone detectors to align with classical detection theory.
2. Comprehensive Security Analysis: It provides a systematic comparison of robustness and security across four distinct threat models, ranging from full white-box access to blind attacks, highlighting the specific vulnerabilities of DNN-based detectors.
3. Empirical Evidence of Trade-offs: The study challenges the assumed superiority of modern methods by demonstrating that, in a zero-bit setting, classical methods often offer superior security without sacrificing robustness.

Results

• White-box and Black-box Scenarios: Broken-Arrows significantly outperformed both Videoseal and TrustMark. Modern DNN-based detectors were found to be highly brittle; the attack achieved near 100% success with minimal distortion (sub-quantization limits, 60dB) for modern methods, whereas Broken-Arrows required significantly higher distortion (40dB) for 100% success. The authors attribute this to "blind spots" in the pixel-space detection regions of DNNs, which are easily exploited via gradient-based attacks.
• Oracle Scenario: For low-distortion attacks ($PSNR \geq 35$ dB), Broken-Arrows remained the most secure. While Videoseal showed better resilience at higher distortion levels, the WmForger attack was highly effective against TrustMark. The WIS attack (VAE purification) systematically removed mid-frequency wavelet coefficients, impacting Broken-Arrows more than modern methods, though Broken-Arrows remained robust overall.
• Blind Scenario: All three methods were robust against standard value-metric operations and regeneration attacks, requiring at least 30dB distortion to erase the watermark. While Videoseal showed marginal gains in robustness against strong JPEG compression compared to Broken-Arrows, TrustMark performed the worst.

Significance and Claims
The paper concludes that modern post-hoc watermarking methods do not consistently outperform classical approaches in a zero-bit setting. The primary finding is a critical trade-off: while modern DNN-based methods offer no significant gain in robustness under the tested conditions, they exhibit a significant decline in security due to their susceptibility to adversarial perturbations and the lack of a secret key mechanism.

The authors emphasize that modern techniques are primarily optimized for high-capacity multi-bit payloads, a goal not addressed in this zero-bit evaluation. Furthermore, the study acknowledges limitations, noting that it focuses on value-metric operations and does not account for geometric distortions (e.g., cropping, rotation), which modern methods often handle via synchronization techniques. The work serves as a cautionary note that the flexibility of deep learning in watermarking comes at the cost of increased attack surface and reduced security guarantees compared to well-established classical designs.