Technical Summary: SafeMark for Watermark-Preserving Text-Guided Image Editing

Problem Statement

The rapid advancement of text-to-image models (e.g., Stable Diffusion, DALL-E 3) has enabled powerful, semantics-driven image editing via iterative denoising. While invisible image watermarking has been established as a critical tool for ownership verification and provenance tracking in static generation, its robustness under text-guided diffusion editing remains largely unexplored.

Existing robust watermarking methods (e.g., HiDDeN, VINE, SleeperMark) typically employ an adversarial-defense paradigm, training encoders against predefined distortions (JPEG, resizing) or specific editing behaviors. However, the paper demonstrates that these methods fail to generalize across diverse, heterogeneous diffusion editors. When images are edited using text-guided editors like DiffusionCLIP, Asyrp, or Eff-Diff, watermark bit accuracy degrades significantly (e.g., dropping from ~0.99 to ~0.30 in some cases). The core research question is: Can watermarked images remain editable without compromising watermark integrity? The paper argues that current "robust" watermarks are editor-dependent and that a new approach is needed to enforce integrity during the editing process itself.

Methodology: SafeMark

The authors propose SafeMark, a framework that treats watermark preservation as an explicit objective within the image editing optimization process, rather than relying on the watermark's intrinsic robustness alone.

1. Problem Formulation

SafeMark frames watermark-preserving editing as a constrained optimization problem. Given a watermarked image $x_{orig}$ with message $w$, an editor $E_\theta$, and a prompt $p$, the goal is to find parameters $\theta^*$ that minimize semantic loss while satisfying a watermark accuracy constraint:
$$\min_\theta \mathbb{E}[L_{sem}(x_{edit}, x_{ref})] \quad \text{s.t.} \quad \text{Acc}(\hat{w}, w) \geq \tau$$
where $x_{edit} = E_\theta(x_{orig}, p)$, $\hat{w}$ is the decoded watermark, and $\tau$ is a target accuracy threshold.

2. Information-Theoretic Foundation

The paper provides a theoretical justification for optimizing bit accuracy directly.

• Mutual Information Bound: Watermark recoverability is bounded by the mutual information $I_\theta(W; X_e)$ preserved by the editor channel.
• Bit Accuracy as a Surrogate: The authors prove (via Fano's inequality and Lemma 1) that maintaining high average bit accuracy $\bar{a}$ lower-bounds this mutual information. Specifically, enforcing $\bar{a} \geq \tau$ ensures a minimum level of information preservation required for recovery.

3. Training Objective and Hinge Penalty

Since hard-bit accuracy is non-differentiable, SafeMark relaxes the constraint using a thresholded hinge penalty on a differentiable surrogate (Brier-type loss):
$$L_{wm} = \max(0, \tau - \hat{g}_{Acc}(\hat{w}, w))$$
where $\hat{g}_{Acc}$ is a smooth approximation of bit accuracy. The total training objective combines semantic fidelity and watermark preservation:
$$L_{total} = \lambda_{sem} L_{sem} + \lambda_{wm} L_{wm}$$

• Semantic Alignment: To ensure the edit remains faithful to the user's intent, a reference edit $x_{ref}$ is generated using a frozen, pre-trained editor $E_{\theta_0}$. The semantic loss $L_{sem}$ encourages the trainable editor's output to stay close to this reference ($L_1$ distance).
• Architecture: SafeMark requires no architectural modification to the underlying diffusion editor. It operates by fine-tuning the editor parameters $\theta$ while keeping the watermark encoder/decoder fixed. Gradients flow only through the final denoised output.

Key Contributions

1. Empirical Identification of Editor-Dependence: The paper demonstrates that existing robust watermarking methods fail to generalize across different text-guided editing pipelines, with performance varying drastically depending on the specific editor used.
2. Theoretical Foundation: It establishes an information-theoretic basis for watermark-preserving editing, showing that bit accuracy is a tractable proxy for mutual information preservation, justifying the use of hinge-penalty objectives.
3. SafeMark Framework: A novel method that integrates watermark supervision directly into the editing objective, achieving near-perfect watermark recovery (bit accuracy $\approx 1.0$) across diverse editors and datasets without sacrificing semantic edit quality.
4. Robustness Validation: The authors show that SafeMark-protected images remain robust to common post-edit distortions (rotation, blurring, resizing), confirming that enforcing preservation during editing does not degrade resilience to standard image corruptions.

Experimental Results

The framework was evaluated on real images (LSUN-Church, LSUN-Bedroom, CelebA, AFHQ-Dog) and AI-generated images, using multiple watermarking schemes (HiDDeN, VINE, Stable Signature, SleeperMark) and three distinct diffusion editors (DiffusionCLIP, Asyrp, Eff-Diff).

• Watermark Preservation: While direct editing ("Mani") caused bit accuracy to drop significantly (e.g., VINE on Bedroom images dropped from 0.999 to 0.299 under Asyrp), SafeMark consistently restored accuracy to 1.000 (or near-perfect levels like 0.999) across all settings.
• Semantic Fidelity: SafeMark did not compromise the quality of the edit. Metrics such as FID, Inception Score (IS), and CLIP similarity showed that SafeMark outputs remained perceptually and semantically close to direct edits. The "Comp" metric (comparing SafeMark edits to direct edits) yielded low FID values, indicating the watermark constraint did not force the image back toward the original or collapse the output.
• Post-Edit Robustness: Under distortions like cropping and blurring, SafeMark maintained high bit accuracy (failure rates generally below 0.15), demonstrating that the preserved watermark signal is robust to subsequent processing.

Significance and Claims

The paper claims that semantic editability and watermark integrity are fundamentally compatible. The degradation of watermarks under semantic editing is not an inherent limitation of the editing process but a result of unconstrained editing dynamics.

By shifting the paradigm from "robust watermarking against editing" to "watermark-preserving editing," SafeMark provides a design-time solution for trustworthy image provenance. The authors position SafeMark as a mechanism that allows image editing service providers to offer high-quality, semantics-driven manipulation while guaranteeing that ownership signals remain recoverable. This reframes provenance protection as an integral part of the editing pipeline rather than an external add-on, enabling trustworthy image provenance in generative editing workflows.

The paper modestly acknowledges limitations: SafeMark currently requires white-box access to the editor and a separate fine-tuning step for each architecture, and it may face challenges with extremely aggressive structural edits that fundamentally alter the scene layout beyond the watermark's capacity. However, it establishes a critical proof-of-concept that watermark preservation can be enforced by design.