💻 computer science

SPFM-Net: Semantic-Prior-Guided Frequency-Constrained Mamba for Invisible Watermark Attack

This paper proposes SPFM-Net, a semantic-prior-guided and frequency-constrained Mamba framework that effectively attacks invisible watermarks by combining high-ratio masking, a fine-tuned Masked Autoencoder, and a global state-space modeling unit to achieve a superior trade-off between removal effectiveness and visual fidelity.

Original authors: Chunpeng Wang, Yanan Shi, Zhiqiu Xia, Jidong Yang, Suo Gao, Qi Li

Published 2026-07-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chunpeng Wang, Yanan Shi, Zhiqiu Xia, Jidong Yang, Suo Gao, Qi Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, bustling library where everyone is constantly sharing photos. To protect their work, artists and companies often hide secret "invisible watermarks" inside these pictures. Think of these watermarks not as a visible stamp, but like a tiny, invisible whisper of code woven into the fabric of the image itself. It's a digital fingerprint that says, "This belongs to me!" without changing how the picture looks to the human eye.

However, just as a library needs to know if a book is being stolen, researchers need to test if these watermarks are strong enough to survive. This is where "watermark attacks" come in. Instead of trying to steal the image, these attacks try to silence the invisible whisper. The goal is to scrub the secret code out of the photo without ruining the picture itself. It's a delicate balancing act: if you scrub too hard, you erase the watermark but also turn the photo into a blurry mess; if you scrub too gently, the watermark stays hidden, and the photo remains safe. For years, scientists have struggled to find the perfect "eraser" that removes the secret message while keeping the image looking crisp and real.

Enter SPFM-Net, a new digital tool designed to solve this tricky puzzle. The researchers behind this project realized that previous attempts to remove watermarks were like trying to fix a broken clock by only looking at one gear at a time. They focused too much on small, local details and missed the big picture. SPFM-Net changes the game by treating the photo like a giant jigsaw puzzle where some pieces are missing.

Here's how it works, step-by-step:

1. The "Blindfold" Strategy
First, SPFM-Net takes the watermarked image and covers up 90% of it with a digital blindfold. It randomly hides most of the picture, leaving only a few scattered patches visible. This might sound counterintuitive—why hide more of the image? The trick is that invisible watermarks often rely on a continuous pattern to hide. By breaking the image into tiny, disconnected pieces, the network forces itself to stop looking at the "whisper" of the watermark and start focusing on the "voice" of the actual image. It's like trying to guess the plot of a movie when you only see a few random frames; your brain has to rely on the story (the semantic meaning) rather than the specific details of the actors' faces.

2. The "Smart Guessing" Engine
Next, the system uses a pre-trained "brain" (called a Masked Autoencoder) that has already learned what real, natural images look like. Because the network has seen millions of photos before, it knows that a sky should be blue and a face should have two eyes. When it sees the few remaining patches of the watermarked image, it uses this knowledge to "hallucinate" or reconstruct the missing parts. Crucially, it reconstructs the natural image, effectively ignoring the weird, artificial patterns of the watermark. It's like a master chef who, given a few ingredients, can recreate a whole dish without accidentally adding the secret spice the enemy tried to hide.

3. The "Frequency Detective"
Once the image is roughly reconstructed, SPFM-Net gets even more specific. It has a special module called MRFFI that acts like a frequency detective. It looks at the image in different "zoom levels" (from tiny textures to big shapes) to find the leftover traces of the watermark. Since watermarks often hide in specific frequency patterns (like a specific musical note), this module hunts down those notes and mutes them, while leaving the rest of the music (the image details) untouched.

4. The "Long-Distance" Scanner
Finally, the system uses a cutting-edge component called Mamba (part of the GSFM module). Think of this as a scanner that can look at the entire image at once, from one corner to the other. Watermarks are often spread out across the whole picture, like a net. Older tools could only look at small neighborhoods, missing the big net. Mamba connects the dots across the whole image, ensuring that even the faintest, most distant watermark signals are caught and removed.

The Results
The researchers tested SPFM-Net against a variety of modern watermarking tricks, including some that use advanced artificial intelligence to hide the code. The results were impressive. In many cases, the system managed to scramble the watermark so thoroughly that it became impossible to read (achieving a "Bit Error Rate" close to 0.5, which is basically random guessing).

But the real magic is in the quality of the photos. While other methods often turned images into blurry, pixelated nightmares, SPFM-Net kept the pictures looking sharp and natural. In tests, it achieved a PSNR (a score for image quality) of up to 42.76 for certain types of watermarks, which is significantly higher than previous methods. It managed to destroy the watermark while keeping the visual fidelity so high that the human eye barely noticed a difference.

What It Doesn't Do
It's important to note that SPFM-Net isn't a magic wand that works perfectly on every single type of watermark in every situation. For some specific, simple watermarks that are deeply tied to the texture of the image, the system was slightly less aggressive to avoid ruining the picture. In these cases, it chose to keep the image looking perfect rather than risk a tiny bit of the watermark remaining. The authors suggest that this trade-off is actually a feature, not a bug, because in the real world, preserving the visual quality of the photo is often just as important as removing the watermark.

In short, SPFM-Net suggests that by treating watermark removal as a "reconstruction" problem—rebuilding the natural image from scratch rather than just trying to erase the bad parts—we can silence the invisible whispers without breaking the song. It's a clever, new way to look at an old problem, proving that sometimes, to see the truth, you have to ignore most of what's right in front of your eyes.

Technical Summary: SPFM-Net

Problem Statement
Existing invisible watermark attacks face a critical trade-off between removal effectiveness and visual fidelity. Traditional methods, ranging from signal processing operations to locally constrained deep learning networks (e.g., CNNs), struggle to capture the long-range dependencies of globally distributed watermark signals. These approaches often treat watermark removal as a local perturbation or pixel-level restoration task, which fails to distinguish between watermark-related signals and reconstructed image details. Consequently, stronger attacks that effectively remove watermarks often degrade image quality through texture loss and artifacts, while weaker attacks preserve fidelity but leave watermarks recoverable. Furthermore, current architectures lack sufficient semantic guidance to separate watermark features from natural image content and are constrained by limited receptive fields.

Methodology: SPFM-Net
The authors propose SPFM-Net (Semantic-Prior-Guided Frequency-Constrained Mamba), a framework that reformulates invisible watermark attack as a semantic-guided image restoration problem rather than direct perturbation suppression. The architecture operates through the following pipeline:

High-Ratio Masking and Semantic Prior Extraction:
- The input watermarked image is divided into non-overlapping patches and subjected to a high-ratio random masking strategy (90% masking ratio). This disrupts the spatial coherence and joint distribution of watermark signals, preventing the network from relying on local denoising shortcuts.
- A partially unfrozen Masked Autoencoder (MAE) serves as the semantic backbone. The encoder is pre-trained on large-scale natural scenes. During fine-tuning, the first 21 Transformer blocks remain frozen to preserve macro-level semantic priors, while only the final three blocks and normalization layers are updated to adapt to subtle watermark residuals. This forces the model to reconstruct intrinsic image content based on global context rather than superficial local details.
Multi-scale Residual Frequency Feature Interaction (MRFFI):
- This module aggregates watermark-related residual features across multiple receptive fields. It employs three parallel convolutional branches with distinct strides ( $s \in \{1, 2, 4\}$ ) to capture features from local textures, intermediate structures, and global semantics.
- The features are concatenated and processed via a $1 \times 1$ convolution. A channel attention mechanism generates an adaptive weight map to suppress responses from watermark-irrelevant regions while enhancing responses in watermark-related areas.
Mamba-based Global State-space Feature Modeling (GSFM):
- To address the limitation of local receptive fields, a lightweight Mamba-based unit models the long-range dependencies of globally distributed watermark signals.
- It utilizes cascaded dual-residual sequence modeling blocks with 1D depthwise separable convolutions and feed-forward networks. This unit separates watermark-related features from natural image content and suppresses remaining traces by scanning the global sequence.
Decoder and Multi-level Optimization:
- A lightweight decoder reconstructs the watermark-free image using transposed convolutions and cross-level skip connections to preserve high-frequency details from shallow encoder layers.
- The network is optimized using a joint loss function comprising three components:
  - Pixel Loss ( $L_{pixel}$ ): $L_1$ norm for spatial tonal consistency.
  - Frequency Loss ( $L_{fft}$ ): $L_1$ norm on 2D Fast Fourier Transform (FFT) magnitudes to suppress abnormal watermark-induced frequency variations.
  - Edge Loss ( $L_{edge}$ ): $L_1$ norm on Sobel gradients to preserve structural boundaries and reduce artifacts.

Key Contributions

Framework Reformulation: The paper introduces SPFM-Net, which shifts the paradigm from direct perturbation suppression to semantic-guided image restoration, leveraging high-ratio masking to sever watermark signal coherence.
Semantic-Prior-Guided Frequency Modeling: By combining a partially frozen MAE with the MRFFI module, the method effectively separates watermark features from natural content, capturing multi-scale frequency discrepancies to suppress watermarks with reduced visual distortion.
Long-Range Dependency Modeling: The design of the Mamba-based GSFM module enables the capture of long-range dependencies in spatially distributed watermark patterns, addressing the receptive field limitations of traditional CNNs.
Zero-Shot Generalization: The approach demonstrates the ability to perform zero-shot removal on unseen deep watermarks without target-specific fine-tuning, validating the feasibility of semantic-guided reconstruction.

Experimental Results
Extensive experiments were conducted on spatial-domain, transform-domain, orthogonal moment-based, and deep learning-based watermarking schemes (including HiDDeN, Stable Signature, and Yu).

Attack Effectiveness: SPFM-Net achieved Bit Error Rates (BER) of 0.48 for Stable Signature and 0.55 for HiDDeN, approaching the binary random guessing level (0.5). This indicates a near-total disruption of the watermark extraction process.
Visual Fidelity: The method maintained high perceptual quality, achieving 42.76 dB PSNR and 0.99 SSIM on the Stable Signature dataset, significantly outperforming baselines like UnMarker (18.23 dB PSNR) and DiffusionAttack.
Generalization: Despite being trained only on traditional watermarks (LSB, DCT, QPHFMs), SPFM-Net successfully generalized to disrupt advanced deep generative watermarks, confirming that the semantic prior forces the model to treat embedded watermarks as high-frequency anomalies regardless of their generation mechanism.
Ablation Studies: Results confirmed that the progressive integration of MRFFI and GSFM modules significantly increased BER, while the multi-level joint loss (pixel, frequency, and edge) was crucial for maintaining high PSNR and preventing perceptual degradation.

Significance and Claims
The paper claims that SPFM-Net offers a favorable trade-off between watermark attack effectiveness and perceptual fidelity, outperforming existing state-of-the-art methods. By utilizing semantic priors to guide the restoration process and frequency constraints to enforce structural integrity, the method provides a novel "semantic decoupling" perspective for digital watermarking attacks. The authors position this work as a step toward evaluating the security boundaries of existing watermarking methods, particularly against the rapid iteration of generative watermarking and adversarial steganography. The approach is presented as a practical compromise that prioritizes high visual quality while achieving robust removal, making it suitable for realistic security evaluation scenarios.

Technical Summary: SPFM-Net

More like this