Watermarking Should Be Treated as a Monitoring Primitive

Technical Summary: Watermarking Should Be Treated as a Monitoring Primitive

Problem Statement
Current evaluations of watermarking for generative models focus almost exclusively on per-sample robustness against adversaries attempting to evade detection or induce false positives. These threat models treat watermark signals as targets for removal or forgery. However, this perspective overlooks a critical capability: the ability of observers to aggregate watermark signals across multiple outputs to infer entity-level information. The paper argues that watermarking is not merely a defensive mechanism for provenance but inherently functions as a monitoring primitive. When deployed with per-entity attribution keys, watermarking introduces persistent statistical structures that enable tracking, linkability, and re-identification over time, capabilities that are not captured by standard robustness metrics.

Methodology
The authors introduce an observer-based threat model that shifts the focus from manipulating individual samples to passively aggregating signals across a stream of outputs. This model distinguishes between two types of observers:

1. Internal Observers: Entities with access to watermark detectors and secret keys. In multi-key deployments, they can directly attribute outputs to specific entities.
2. External Observers: Entities without access to keys or detectors. They rely on observable outputs and statistical learning to infer source identity by exploiting persistent, key-dependent patterns induced by the watermarking process.

The study formalizes entity re-identification as the task of determining whether two outputs originate from the same entity. The authors evaluate this under zero-bit watermarking (where no explicit identity message is encoded) across text and image modalities.

• Models: Qwen2.5-14B for text and Stable Diffusion v2.1 for images.
• Watermarking Methods: Multiple zero-bit schemes including KGW, Unigram, EWD, MorphMark, EXP, IE, SWEET, SEMSTAMP, SynthID, TreeRing, Wind, and Gaussian Shading.
• Experimental Setup: A multi-key deployment where distinct entities are assigned unique keys. The authors use prompt-matched settings to ensure attribution relies on watermark-induced structure rather than prompt differences.
• Metrics:
  • Internal: Top-1 attribution accuracy (TPR@1%FPR).
  • External: Top-1 and Top-3 classification accuracy using learned classifiers (BERT-Base for text, CLIP-RN50 for images) trained on observable features.
• Controls: Baselines include no-watermark generation and shared-key deployments to isolate the effect of key-dependent structure.

Key Contributions

1. Observer-Based Threat Model: The paper formalizes a threat model where observers aggregate signals across outputs to perform entity-level inference, distinguishing between inherent monitoring (internal) and emergent monitoring (external).
2. Inherent Monitoring in Zero-Bit Schemes: The authors demonstrate that monitoring is unavoidable in multi-key deployments even for zero-bit watermarking. Distinct keys induce persistent statistical structures that allow internal observers to attribute outputs without explicit identity encoding.
3. Emergent External Monitoring: The study shows that external observers can learn to identify generating entities over time by aggregating weak signals, even without access to keys or detectors. This capability emerges from the statistical structure introduced by the watermarking design.
4. Dual-Use Tension: The work highlights a fundamental tension between the intended uses of watermarking (attribution, safety) and its unintended capabilities (tracking, privacy leakage), arguing that evaluation protocols must extend beyond per-sample robustness.

Results

• Internal Attribution: Under multi-key zero-bit watermarking, internal observers achieved near-perfect attribution accuracy (TPR@1%FPR) across all tested methods. Performance remained high even as the number of entities increased from 4 to 16, with only mild degradation. This confirms that zero-bit schemes inherently support reliable entity tracking when keys are distinct.
• External Re-Identification: External observers initially performed near random guessing but showed substantial improvement as the number of observed samples per entity increased.
  • For text (KGW) with 16 entities, top-1 accuracy rose from ~11.3% to 73.0% with 4,000 samples per entity.
  • For images (TreeRing), convergence was faster, reaching 91.0% top-1 accuracy with 16 entities and 4,000 samples.
• Control Experiments:
  • No-Watermark: Identification accuracy collapsed to random chance, confirming that the signal is not inherent to the model or prompts.
  • Shared-Key: When all entities used the same key, identification accuracy also collapsed to chance, proving that the monitoring capability relies specifically on key-dependent statistical structure.

Significance and Claims
The paper claims that watermarking should be re-evaluated as a monitoring technology rather than solely a provenance tool. The authors assert that:

• Monitoring is Unavoidable: Once per-entity keys are deployed, internal monitoring is a direct consequence of the system design and cannot be technically prevented without sacrificing attribution utility (e.g., by using shared global keys).
• Emergent Capabilities: External monitoring is an emergent property that depends on watermark design. While distribution-preserving or undetectable schemes may mitigate external risks, they do not eliminate internal monitoring capabilities in multi-key settings.
• Evaluation Gap: Current evaluation protocols significantly understate the monitoring capabilities of watermarking systems. The authors recommend that future evaluations explicitly report attribution accuracy as a function of the number of entities and samples observed over time.

The authors conclude that regulatory and standardization efforts (such as the EU AI Act) are incomplete if they treat watermarking only as a mechanism for provenance. They call for broader discussion on the governance of watermarking systems, emphasizing the need for transparency regarding their inherent ability to enable tracking and inference over time.