🤖 AI

The Perceived Fragility of Explanations in Audio Models: Manipulation of Attribution with Unchanged Predictions

This paper introduces a psychoacoustic framework demonstrating that post-hoc explanations for audio deepfake detection models are fragile, as adversaries can apply inaudible perturbations to systematically distort attribution heatmaps while preserving the model's original classification.

Original authors: Piotr Kitłowski, Dominik Wiącek, Mateusz Modrzejewski

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Piotr Kitłowski, Dominik Wiącek, Mateusz Modrzejewski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart security guard (the AI model) whose job is to listen to audio and decide if it's a real human voice or a fake, computer-generated "deepfake." To help you trust this guard, the system shows you a "heat map" (an explanation). This map lights up the specific parts of the sound wave the guard used to make its decision, like highlighting the suspicious cough or the robotic glitch that gave the fake away.

This paper asks a scary question: What if someone could trick the guard into pointing at the wrong thing, while still making the exact same decision?

Here is the breakdown of what the researchers found, using simple analogies:

1. The "Invisible Ink" Trick

In the past, researchers tried to trick AI image detectors by adding tiny, visible pixels to a picture. But in audio, if you add noise that the computer can "see," humans can usually "hear" it too. That ruins the trick.

The authors developed a new method using Psychoacoustics. Think of this as "invisible ink" for sound.

The Analogy: Imagine you are whispering a secret to a friend in a loud, busy rock concert. You can whisper something completely different than what you intended, and your friend won't even notice because the loud music "masks" your whisper.
The Result: The researchers found a way to add "whispers" (tiny perturbations) to the audio that are loud enough to confuse the AI's explanation map, but quiet enough that human ears hear nothing but the original song.

2. The "Decoupling" Act

The goal of the attack was to decouple the explanation from the prediction.

Before the attack: The AI says, "This is a fake," and the heat map points to the robotic glitch. (Truthful).
After the attack: The AI still says, "This is a fake," but the heat map now points to a completely innocent part of the song, like the background drums.
The Danger: If you only looked at the heat map, you would think the AI is looking at the drums to make its decision. You would be completely misled about why the AI made that call, even though the final verdict (Fake) didn't change.

3. Who Got Tricked the Most?

The researchers tested this on three different types of AI "guards" (architectures):

The "Token-Based" Guard (AST): This model was the most fragile. It's like a guard who focuses on specific, isolated words. The researchers could easily push its attention away from the real clues to fake ones.
The "Convolutional" Guard (VGGish): This model was in the middle. It looks at patterns like a human listening to a rhythm. It was harder to trick but still vulnerable.
The "Long-Range" Guard (SpecTTTra): This model was the toughest. It listens to the whole story from start to finish. Because it tracks long connections in the music, the "whispers" the attackers added got diluted and didn't confuse the explanation map as easily.

4. Why Some Songs Were Easier to Trick

The researchers noticed that the "easiest" songs to trick were busy, loud, and complex (like rock or electronic music).

The Analogy: It's easier to hide a secret note in a chaotic jazz solo than in a quiet, single-note piano melody. The "noise" of the busy music provided a perfect hiding spot (masking budget) for the invisible attack.
The Harder Targets: Quiet, sparse music (like classical or acoustic) left no room for the "whispers." The attack failed because there was no background noise to hide in.

5. The Big Conclusion

The paper concludes that we cannot blindly trust the "heat maps" or explanations that audio AI systems give us.

Just because the AI gives you a reason (a highlighted part of the sound) doesn't mean that's the real reason it made the decision.
An attacker could systematically rewrite the AI's "story" about why it made a decision, making the explanation look like it's focusing on safe, innocent parts of the audio, all while the AI still correctly (or incorrectly) identifies the deepfake.

In short: The AI might be right about what the audio is, but the "explanation" it gives you about why it thinks that can be completely fake, and you wouldn't even hear the difference.

Technical Summary: The Perceived Fragility of Explanations in Audio Models

Problem Statement
The proliferation of synthetic audio has necessitated the deployment of deepfake detection systems, which increasingly rely on Explainable AI (XAI) methods to foster trust by highlighting the acoustic artifacts driving model decisions. However, this reliance introduces a critical security vulnerability: the fragility of the explanations themselves. While the susceptibility of attribution maps to adversarial manipulation has been extensively documented in computer vision (Ghorbani et al., 2019; Dombrowski et al., 2019; Heo et al., 2019), its implications for audio models remain largely unaddressed. Existing vision-based attacks utilize $L_p$ norms to measure manipulation cost, a metric that fails to correlate with human auditory perception (Abdullah et al., 2021). Consequently, there is a lack of domain-specific frameworks to determine whether XAI techniques for audio provide robust interpretations or if they can be decoupled from the classifier's decision boundary through imperceptible perturbations.

Methodology
To investigate this vulnerability, the authors propose a novel optimization framework designed to manipulate post-hoc explanation heatmaps while strictly preserving the model's final prediction and maintaining imperceptible audio quality.

Target Models and XAI Methods: The study evaluates three distinct architectures representing different feature extraction paradigms: VGGish (classical convolutional), Audio Spectrogram Transformer (AST, self-attention), and SpecTTTra (attention-based, designed for the SONICS dataset). Two post-hoc attribution methods are analyzed: Grad-CAM (first-order gradients) and Layer-wise Relevance Propagation (LRP, conservation-based).
Attack Strategies: Three methodologies are applied to 100 sampled recordings from the SONICS deepfake dataset:
1. Standard PGD: An $L_\infty$ -bounded attack minimizing structural similarity between original and perturbed maps, serving as a baseline without perceptual constraints.
2. X-Shift (Adapted): A spatial displacement strategy forcing relevance to an irrelevant target region.
3. Psychoacoustic Noise Modeling (Proposed): A custom framework incorporating a dynamic psychoacoustic masking threshold into the loss function.
Optimization Framework: The proposed loss function $L(\delta)$ $L (δ)$ combines three terms:
- $L_{explain}(\delta)$ : Minimizes cosine similarity between original and perturbed attribution maps.
- $L_{audibility}(\delta)$ : Penalizes spectral energy exceeding a static psychoacoustic masking threshold $T(x)$ , ensuring perturbations remain inaudible.
- $L_{pred\_preserve}(\delta)$ : A margin-based hinge loss preventing changes to the original prediction.
  The optimization utilizes the Adam optimizer to handle the second-order derivatives required for attribution manipulation.
Evaluation Metrics: Perceptual transparency is assessed using a suite of domain-specific metrics (PEAQ, ViSQOL, Zimtohrli, CDPAM, PESQ, STOI). Explanation fragility is quantified via Cosine Similarity and Top-10 Overlap. The authors introduce the Audio Fragility Score (AFS $_{stable}$ ), a continuous metric that balances attribution displacement against prediction preservation and perceptual quality.

Key Contributions

Novel Optimization Framework: The paper adapts explanation-targeted adversarial attacks to the audio domain, successfully decoupling model attributions from final classifications.
Psychoacoustic Constraints: The integration of a dynamic psychoacoustic masking threshold into the loss function allows for large attribution map changes while penalizing any perturbation that exceeds human perception limits or alters the model's prediction.
Empirical Evaluation: A focused evaluation across diverse architectures (CNNs and Transformers) using the SONICS dataset, leveraging perceptual metrics to validate the imperceptibility of the attacks.

Results

Perceptual Quality: Unconstrained methods (PGD) severely degrade audio quality (e.g., PESQ $\approx$ 2.8). In contrast, the proposed Psychoacoustic framework maintains high objective perceptual quality (ViSQOL > 4.1, CDPAM $\ge$ 0.98) by bounding noise within human masking thresholds.
Architectural Vulnerability: The study reveals significant disparities in robustness. The token-based AST architecture proved the most fragile (Mean Rank: 3.00), allowing adversaries to easily manipulate attention maps. Conversely, SpecTTTra demonstrated the highest resistance (Mean Rank: 7.83), likely due to its mechanism of tracking long-range temporal dependencies which dilutes the impact of constrained noise.
Acoustic Dependencies: "Easy" samples to attack were characterized by dense, broadband audio (e.g., rock/electronic) with high spectral bandwidth and zero-crossing rates, providing a larger "masking budget" for noise. "Hard" samples featured extreme acoustic sparsity (e.g., classical music), where strict perceptual constraints limited allowable adversarial noise.
Geometric Analysis: PCA analysis indicates that psychoacoustic attacks induce smooth, directional attribution shifts in transformer-based models, whereas unconstrained attacks primarily induce variance reduction.

Significance and Claims
The paper claims to demonstrate that post-hoc explanations in audio deepfake detection are fragile under targeted perturbations. The authors argue that attribution maps (Grad-CAM, LRP) can be systematically manipulated without altering predictions or degrading perceptual fidelity, a vulnerability not captured by standard $L_p$ metrics.

The findings suggest that the structural limitations of post-hoc explanations (previously noted by Grinberg et al., 2025) extend beyond inherent inaccuracy to deliberate adversarial manipulation. Specifically, attention-based architectures appear more susceptible to explanation steering than convolutional networks, and acoustic topology (dense vs. sparse signals) dictates the robustness of the explanation.

The authors conclude that relying on visual attributions for audio model trustworthiness is premature. They posit that future research must focus on developing interpretability mechanisms that are mathematically tethered to the classifier's exact decision boundary. The work is framed as a diagnostic tool to improve the robustness of detection pipelines, with all code and artifacts released to support reproducibility and responsible use.