🤖 AI

HarmonicAttack: An Adaptive Cross-Domain Audio Watermark Removal

The paper introduces HarmonicAttack, a novel adaptive cross-domain audio watermark removal method that effectively eliminates watermarks from state-of-the-art schemes without requiring access to the target detector, demonstrating superior generalization across unseen datasets and maintaining high perceptual quality.

Original authors: Kexin Li, Xiao Hu, Ilya Grishchenko, David Lie

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kexin Li, Xiao Hu, Ilya Grishchenko, David Lie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a secret stamp on your voice or a song you create with AI. This stamp, called a watermark, is invisible to the human ear but acts like a digital fingerprint. Its job is to prove, "Hey, this was made by a computer, not a human." This is meant to stop bad actors from using fake voices to scam people or spread lies.

But, just like a forger trying to scrape a stamp off a rare painting, there are people trying to remove these watermarks to make the fake audio look and sound completely genuine.

This paper introduces a new tool called HarmonicAttack. Think of it as a highly skilled, automated "eraser" that can wipe these invisible digital stamps off audio files without ruining the sound.

Here is how it works, broken down into simple concepts:

1. The Problem with Old Methods

Previously, trying to remove these watermarks was like trying to guess a password by asking a locked door, "Is this the right key?" over and over again.

The Old Way: Attackers had to keep asking the watermark detector (the "lock") if their changes were working. This was slow, required access to the detector, and often sounded like static or a broken radio when they finally got it to work.
The New Way (HarmonicAttack): This tool doesn't need to ask the detector anything. It's like a master locksmith who has studied thousands of photos of the lock and the key. It learns the pattern of the watermark itself and removes it in one smooth motion, without needing to check the lock every time.

2. How HarmonicAttack "Sees" the Watermark

The authors realized that even though watermarks are hidden, they leave a specific "footprint" in the audio.

The Analogy: Imagine a song is a painting. The watermark is a tiny, almost invisible layer of paint added on top of the original colors.
The Trick: The paper explains that these watermarks are usually hidden in the "busy" parts of the sound (where the music or voice is loudest) because our ears can't hear the tiny changes there (this is called psychoacoustic masking).
The Solution: HarmonicAttack uses a Dual-Path Autoencoder. Think of this as a pair of glasses with two lenses:
1. One lens looks at the sound as a wave over time (like watching a heartbeat monitor).
2. The other lens looks at the sound as a colorful map of frequencies (like a piano roll showing which notes are playing).
  By looking at both views at once, the AI can spot exactly where the "invisible paint" (the watermark) is hiding and scrape it off precisely, leaving the original painting (the audio) untouched.

3. The "Teacher and Student" Training

To make sure the eraser doesn't accidentally erase the singer's voice along with the watermark, the system uses a GAN (Generative Adversarial Network) approach.

The Generator (The Student): This is the part that tries to remove the watermark.
The Discriminator (The Teacher): This is a strict judge that listens to the "cleaned" audio and compares it to the original. If the cleaned audio sounds even slightly weird or robotic, the teacher says, "No, that's not natural! Try again."
The Result: The student gets better and better until it can remove the watermark so perfectly that even the teacher can't tell the difference between the original and the cleaned version.

4. Why It's a Big Deal (The Results)

The paper tested this tool against the strongest, most modern watermarking systems (like AudioSeal, WavMark, and others) using both speech (people talking) and music.

It Works Everywhere: The team trained the AI on one type of data (speech from a specific dataset) and then tested it on completely different data (music and different voices). It worked just as well. It's like learning to drive on a quiet country road and then immediately driving perfectly on a busy city highway without extra practice.
It's Fast: Old methods took minutes or hours to clean a single song. HarmonicAttack does it in a fraction of a second (near real-time).
It's Effective: While other methods managed to remove watermarks only about 38% of the time (and often made the audio sound bad), HarmonicAttack removed them 92% to 100% of the time while keeping the audio sounding crystal clear.

The Bottom Line

The paper concludes that current "invisible stamps" on AI audio are not as safe as we thought. HarmonicAttack proves that if you can teach an AI to recognize the shape and location of a watermark, you can remove it easily, even if you don't know how the watermark was made in the first place.

Important Note: The paper strictly focuses on the technical ability to remove these watermarks to test their strength. It does not claim this tool should be used to commit fraud, nor does it suggest how to fix the problem other than saying future watermark designs need to be smarter to survive this kind of "learning-based" attack.

Technical Summary: HarmonicAttack

Problem Statement
The proliferation of high-quality AI-generated audio has introduced significant security risks, including misinformation and voice-cloning fraud. While watermarking is a primary defense mechanism to distinguish synthetic audio from genuine content, the robustness of these watermarks against removal is often evaluated under unrealistic assumptions. Previous watermark removal techniques typically require access to the target watermark detector during the attack process (e.g., via repeated queries for confidence scores). This "white-box" or "gray-box" assumption is often impractical in real-world scenarios where detectors are proprietary or inaccessible. Consequently, there is a lack of objective evaluation regarding the true robustness of audio watermarks against black-box adversaries who possess only paired samples of original and watermarked audio.

Methodology: HarmonicAttack
The authors introduce HarmonicAttack, the first learning-based, black-box audio watermark removal system that operates without access to the watermarking algorithm or the detector. The method relies solely on a dataset of paired original and watermarked audio samples to train a general removal model.

Architecture: HarmonicAttack employs a dual-path convolutional autoencoder operating in both temporal and frequency domains.
- Waveform Encoder: Processes 1D raw audio signals using 1D convolutions to capture multiscale temporal patterns.
- Spectrogram Encoder: Applies a Short-Time Fourier Transform (STFT) and processes the resulting 2D data using 2D convolutions to capture spectral features.
- Decoder: Utilizes an attention mechanism to selectively reconstruct the audio, focusing computational resources on time-frequency bins containing watermark artifacts while preserving authentic content.
Training Paradigm: The system uses GAN-style adversarial training. A discriminator is trained to distinguish between original audio and the generator's "unwatermarked" outputs. This forces the generator to produce audio that is statistically indistinguishable from natural audio, preventing the introduction of perceptual artifacts.
Loss Function: The generator is optimized using a novel multi-component loss function designed to address three specific objectives:
1. Precise Removal (Disentanglement Loss): Encourages the model to explicitly learn and model the structure of the watermark signal to actively remove it, rather than applying generic denoising.
2. Localization (Localization Loss): Guides the model to focus on time-frequency bins where watermark energy is concentrated, exploiting the psychoacoustic masking principles (embedding in less sensitive regions) used by modern watermarks.
3. Quality (Reconstruction Loss): Ensures the perceptual fidelity of the underlying audio is preserved using a hybrid $L_1$ and $L_2$ loss.

Key Contributions

Black-Box Capability: The paper presents the first learning-based removal system that does not require access to the watermark detector, relying instead on paired data to learn watermark characteristics independently of the underlying audio distribution.
Dual-Path Design & Multi-Objective Loss: The authors design an architecture and loss function that jointly address precise signal modeling, localization within masked time-frequency bins, and high-quality preservation.
Cross-Domain and Cross-Scheme Generalization: The study demonstrates that the model, trained on one dataset (LibriSpeech) against one scheme (AudioSeal), generalizes effectively to unseen datasets (FMA, VCTK) and unseen watermarking schemes (WavMark, SilentCipher, AudioMarkNet).

Experimental Results
The authors evaluated HarmonicAttack against four state-of-the-art watermarking schemes: AudioSeal, WavMark, SilentCipher, and AudioMarkNet.

Attack Success Rate (ASR): HarmonicAttack achieved significantly higher success rates than baselines (MP3/OGG compression, EnCodec, and the optimization-based AudioSquareAttack).
- On the FMA dataset (music), HarmonicAttack achieved 100% ASR against all evaluated watermarks, whereas the best baseline (AudioSquareAttack) achieved only 2% against AudioSeal and 44% against WavMark.
- On the VCTK dataset (speech) against AudioMarkNet, HarmonicAttack achieved a 92% ASR, substantially outperforming the best baseline (38%).
Perceptual Quality: The method maintained high audio quality, with ViSQOL scores around 4.33 on the FMA dataset and STOI scores above 0.93 on LibriSpeech, indicating that the removal process did not introduce audible artifacts or degrade intelligibility.
Efficiency and Scalability: Unlike optimization-based attacks whose cost scales with input length, HarmonicAttack operates in near real-time (approx. 0.03 seconds per sample) regardless of audio duration. In contrast, baseline attacks showed severe performance degradation as audio length increased (e.g., ASR dropping from 38% to 2% for 30-second clips).

Significance and Claims
The paper claims that HarmonicAttack reveals a critical vulnerability in modern audio watermarking schemes. The authors argue that because current watermarks are constrained by design principles to preserve perceptual quality (specifically by exploiting psychoacoustic masking), they inadvertently create consistent, learnable patterns in the time-frequency domain. By explicitly modeling these patterns and localizing them, an adversary can effectively remove watermarks without access to the detector.

The significance of this work lies in shifting the evaluation paradigm from "can a watermark survive compression?" to "can a watermark survive a learned, adaptive removal model?" The authors conclude that future watermarking designs must move beyond robustness to simple signal perturbations and must account for adaptive, learning-based attacks that can model, localize, and suppress watermark signals.