SHIFT: Stochastic Hidden-Trajectory Deflection for Removing Diffusion-based Watermark

1. Problem Statement

Diffusion-based watermarking methods (e.g., Tree-Ring, RingID, Gaussian Shading) have emerged as a leading solution for tracing AI-generated content. These methods embed watermarks by manipulating the initial noise, intermediate latent variables, or the reverse diffusion trajectory. Verification relies on a critical, shared assumption: Trajectory Consistency. Specifically, defenders assume that the generative path taken to create an image can be faithfully reconstructed (via DDIM inversion) to recover the original noise or latent structure containing the watermark.

Existing attacks (e.g., regeneration, latent-noise forgery) have struggled to break these watermarks because:

• Deterministic regeneration (using DDIM/PNDM) preserves the structural frequency information and latent paths that carry the watermark.
• Optimization-based attacks require expensive, per-image iterative optimization and often fail against semantic watermarks.

The paper identifies that the reliance on trajectory consistency is a fundamental, exploitable vulnerability common to all current diffusion watermarking paradigms.

2. Methodology: SHIFT

The authors propose SHIFT (Stochastic Hidden-Trajectory Deflection), a training-free, watermark-agnostic attack that exploits the difference between deterministic and stochastic sampling. SHIFT operates in two stages to deflect the generative trajectory in latent space, making the reconstructed image statistically decoupled from the original watermark-embedded path while preserving visual quality.

Stage I: Partial Forward Diffusion

• Goal: Attenuate trajectory-specific information without destroying semantic content.
• Process: The watermarked image ($x_w$) is encoded into a latent representation ($z_0$) using a pretrained VAE. The system then performs a controlled forward diffusion step to a target timestep $t_\lambda = \lceil \lambda T \rceil$ (where $\lambda \in (0, 1]$ is a re-noising strength).
• Mechanism: This injects isotropic Gaussian noise, progressively replacing fine-grained trajectory details with noise while retaining the coarse semantic scaffold.

Stage II: Stochastic Reverse Resampling

• Goal: Regenerate the image along a new, random trajectory to break the link to the original watermark.
• Process: Instead of using a deterministic sampler (like DDIM) which would retrace the original path, SHIFT uses a stochastic ancestral sampler (Euler ancestral sampler).
• Mechanism: At each reverse step, fresh Brownian motion noise ($\xi_t$) is injected. This ensures that the reverse trajectory is not uniquely determined by the starting point ($z_{t_\lambda}$). The pretrained score function pulls the sample toward the natural image manifold (preserving semantics), but the path taken is entirely new and unrelated to the original watermark trajectory.
• Output: The final latent is decoded to produce the attacked image ($x_a$).

Key Advantages:

• Requires no model retraining, fine-tuning, or knowledge of the specific watermark key/algo.
• Computationally efficient (single forward pass + standard diffusion steps).
• No adversarial optimization required.

3. Theoretical Justification

The paper provides formal guarantees using Wasserstein distance ($W_2$) analysis:

• Trajectory Decoupling: The authors prove that the noise recovered via DDIM inversion from the attacked image is statistically independent of the original watermark noise.
• Stochasticity is Essential: The paper demonstrates that deterministic resampling fails because the conditional law of the reconstructed latent remains degenerate (uniquely determined by the start point). In contrast, stochastic resampling introduces fresh randomness at every step, breaking the one-to-one correspondence between the partially noised latent and the final trajectory.
• Bound Analysis: Theoretical bounds show that as the re-noising strength ($\lambda$) increases, the source signal contribution decreases, eventually falling below the verification tolerance of the watermarking scheme.

4. Experimental Results

The authors evaluated SHIFT against nine representative watermarking methods across three paradigms (noise-space, frequency-domain, and optimization-based):

• Methods Tested: Tree-Ring, RingID, PRC, WIND, Gaussian Shading, GaussMarker, SFW, SEAL, and ROBIN.
• Attack Success Rate (ASR): SHIFT achieved 95%–100% ASR across all methods, with an average of 97.8%.
  • It outperformed the "Black-box attack" (96.8% avg) and "Forgery attack" (81.3% avg).
  • It successfully defeated highly robust methods like Tree-Ring (98% vs. 72% for black-box) and Gaussian Shading (97% vs. 4% for forgery).
• Image Quality: SHIFT maintained superior semantic consistency and realism compared to existing attacks.
  • CLIP Score: 32.194 (Highest among all methods).
  • FID: 73.471 (Significantly lower/better than Black-box: 106.831 and Forgery: 96.431).
• Trajectory Decoupling: Empirical analysis of $L_1$ and $L_2$ noise distances showed that SHIFT consistently pushed the recovered noise into a range statistically indistinguishable from random noise, effectively decoupling the image from the watermark trajectory.

5. Key Contributions

1. Vulnerability Identification: Identified "trajectory consistency" as a fundamental, shared vulnerability in all diffusion-based watermarking schemes.
2. Novel Attack Framework: Proposed SHIFT, a training-free attack that uses partial forward diffusion and stochastic reverse resampling to deflect trajectories without needing watermark-specific knowledge.
3. Theoretical Proof: Provided formal trajectory-decoupling guarantees via Wasserstein distance analysis, proving why stochastic resampling succeeds where deterministic methods fail.
4. Empirical Superiority: Demonstrated state-of-the-art attack success rates (95–100%) while preserving the highest image quality among all compared attacks.

6. Significance and Future Work

• Security Implications: The paper challenges the current consensus that diffusion watermarks are robust. It suggests that any verification system relying on reconstructing a single deterministic trajectory is inherently fragile against stochastic deflection.
• Future Directions:
  • Designing provenance mechanisms that do not rely on single-trajectory reconstruction.
  • Extending the analysis to video diffusion and hybrid generative models (e.g., autoregressive + flow matching).
  • Investigating defenses against stochastic trajectory deflection.

In summary, SHIFT reveals that the very mechanism (deterministic invertibility) that makes diffusion watermarks verifiable is also their Achilles' heel, and it leverages stochasticity to render these watermarks ineffective without compromising the visual utility of the generated content.