💻 computer science

Explainable-by-Design Audio Deepfake Detection via Wiener-Hopf Linear Prediction

This paper proposes an explainable-by-design audio deepfake detection framework that combines Wiener-Hopf linear prediction with a lightweight 2D CNN to achieve competitive performance, low computational complexity, and transparent interpretability regarding acoustic signal properties.

Original authors: Mattia Tamiazzo, Simone Milani, Massimo Iuliani, Marco Fontani

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mattia Tamiazzo, Simone Milani, Massimo Iuliani, Marco Fontani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to catch a voice actor who is faking their voice using a super-smart computer. In the past, other detectives used giant, mysterious "black box" computers to solve these cases. These boxes were incredibly smart and could catch the fakes, but they were also huge, slow, and no one knew how they made their decisions. It was like asking a magic 8-ball for the truth; it gave you an answer, but you couldn't see the gears turning inside.

This paper introduces a new kind of detective that is explainable-by-design. Instead of a magic black box, this detective uses a simple, transparent tool called a Wiener-Hopf linear predictor.

The Magic of the "Prediction Machine"

Think of the audio signal as a long line of dominoes. A real human voice has a natural rhythm and flow, like a skilled domino pusher who knows exactly how each piece will fall. A computer-generated fake voice, however, is built by a machine that sometimes stumbles on the rhythm, especially in the quiet parts or when switching from a loud sound to silence.

The authors' method works by trying to predict the next sound in the line based on the sounds that came before it.

Real Speech: The prediction machine guesses the next sound correctly most of the time because the physics of a real human throat and room are consistent.
Fake Speech: The machine gets confused. It stumbles because the fake voice doesn't have the natural "echo" or "reverberation" that happens in a real room. The computer-generated voice is often too clean, lacking the messy, natural background noise of a real recording.

The paper suggests that these "stumbles" in the prediction are the smoking gun. The detector doesn't just guess; it looks at the specific numbers (called filter coefficients) where the prediction went wrong.

The "Flashlight" (Grad-CAM)

To prove they aren't just guessing, the authors use a special flashlight called Grad-CAM. This flashlight shines on the parts of the audio that made the detector say, "Aha! This is fake!"

Here is the surprising twist the paper found: The flashlight doesn't just shine on the loud singing parts. It shines brightest on the silence and the transitions (the moments when the voice starts or stops).

The Theory: The authors suggest that real recordings have a natural "tail" of sound that lingers in the silence (like a sound fading out in a big hall). Fake voices often cut this tail off too cleanly. The detector catches this "too-clean silence" as a major clue.
The Proof: When the researchers tested this by removing the silence from the audio, the detector's performance dropped significantly (the error rate jumped by about 14.42% on average). This confirms that the silence is a critical part of the puzzle.

How Good is It? (The Scoreboard)

The paper tested this new detective on three different sets of fake voices (datasets named ASVspoof 2019, FakeOrReal, and DiffSSD).

The Score: The new method caught fakes with an average error rate of 3.16%. This is very competitive with the giant, complex black-box models, but the new model is 20 times smaller and much faster.
The Comparison: On the DiffSSD dataset, it got an error rate of 2.95%, beating other big models like Wav2Vec2 (which had 3.00%) and crushing the MFCC-ResNet (which had 11.00%). On the FakeOrReal dataset, it was incredibly sharp, with an error rate of just 0.56%.

What Happens When the Audio Gets Messy?

In the real world, audio gets noisy. People talk on phones, listen to MP3s, or deal with bad connections. The paper tested what happens when they add white, pink, or brown noise, compress the audio with MP3 (at bitrates like 32, 64, and 128 kbps), or filter it like a telephone (between 300–3400 Hz).

The Bad News: If you use the detective without retraining it, it gets confused. For example, with white noise at a low level (5 dB), the error rate on one dataset jumped to over 70%.
The Good News: If you give the detective a quick "refresher course" (fine-tuning) on the messy audio, it bounces back! The error rates drop back down to near-original levels. For instance, after fine-tuning on MP3 compression at 32 kbps, the error rate on the FakeOrReal dataset stayed incredibly low at 0.35%.

What the Paper Rules Out

The authors explicitly argue against relying solely on massive, complex neural networks that act as black boxes. They suggest that while those big models are accurate, they are too heavy for small devices and don't explain why they made a decision. They also rule out the idea that we only need to look at the loud, voiced parts of speech; their data shows that ignoring the silence makes the detector much worse.

The Bottom Line

This paper doesn't claim to have solved the problem of fake voices forever. Instead, it suggests that by using a simple, physics-based prediction tool and a tiny, fast computer brain, we can catch fakes just as well as the giants, but with the added superpower of knowing exactly why we caught them. It's a lighter, faster, and more honest way to protect the truth in our audio world.

Technical Summary: Explainable-by-Design Audio Deepfake Detection via Wiener-Hopf Linear Prediction

Problem Statement
The rapid advancement of synthetic speech generation has created a critical challenge in multimedia forensics: detecting audio deepfakes. While recent deep learning approaches achieve high accuracy, they typically rely on "black-box" architectures (e.g., large Self-Supervised Learning models or complex ensembles) that suffer from two primary limitations:

Lack of Interpretability: They offer limited insight into why a decision was made, failing to provide adequate justifications for forensic outcomes.
High Computational Complexity: Large model sizes hinder deployment in resource-constrained or embedded systems and often lead to overfitting on specific dataset conditions, reducing adaptability to new scenarios.

Methodology
The authors propose an "explainable-by-design" framework that bridges the gap between physical signal properties and classification outcomes. The methodology consists of three main stages:

Feature Extraction via Wiener-Hopf Linear Prediction:
Instead of processing raw waveforms or spectrograms directly, the input audio is segmented into consecutive overlapping windows (1024 samples, 50% overlap). A Wiener-Hopf linear predictor of order $M=30$ is applied to each window. The predictor coefficients ( $h$ ) are derived by solving the Wiener-Hopf equation ( $R \cdot h = p$ ), where $R$ is the Toeplitz autocorrelation matrix of the signal.
- Rationale: The authors hypothesize that real and synthetic speech exhibit different predictability characteristics. Specifically, synthetic generators often struggle to reproduce natural reverberation tails and statistical consistencies in silence and transition regions. The Wiener-Hopf coefficients capture these physical discontinuities and reverberation characteristics.
Lightweight Classification:
The extracted predictor coefficients from all frames are stacked to form a 2D matrix of shape $(F \times M)$ , where $F$ is the number of frames. This matrix is treated as a single-channel image and fed into a lightweight 2D Convolutional Neural Network (CNN).
- Architecture: The CNN comprises four convolutional layers (with Batch Normalization and ReLU), grouped into two blocks with Max Pooling, followed by Adaptive Max Pooling and fully connected layers. The total parameter count is approximately 422K.
- Preprocessing: To handle variable utterance lengths, matrices are padded to a fixed frame count ( $F=300$ ) using symmetric padding rather than zero-padding, which the authors found critical for preserving boundary continuity and feature extraction quality.
Explainability via Grad-CAM:
The framework utilizes Gradient-weighted Class Activation Mapping (Grad-CAM) on the final convolutional layer to generate saliency maps. This visualizes which specific Wiener-Hopf coefficients and temporal frames (e.g., silence, voiced, or transition regions) contributed most to the classification decision.

Key Contributions

Explainable-by-Design Architecture: The paper presents one of the first approaches to combine physically relevant audio features (Wiener-Hopf coefficients) with explainability algorithms (Grad-CAM) at a low computational cost, achieving performance comparable to complex schemes.
Justifiable Outcomes: The detector's decisions are explicitly linked to specific acoustic properties. Grad-CAM analysis reveals that the model focuses on low-order predictor coefficients and, crucially, on silence and transition regions (voice-to-silence and silence-to-voice).
Competitive Performance with Low Complexity: The method achieves competitive detection accuracy across heterogeneous datasets while requiring approximately 20 times less computational complexity than state-of-the-art solutions with similar performance.
Robustness: The approach demonstrates resilience to common post-processing degradations (additive noise, MP3 compression, telephone filtering) when fine-tuned, recovering performance close to the clean baseline.

Experimental Results
The method was evaluated on three benchmark datasets: ASVspoof 2019 (Logical Access), FakeOrReal (FoR), and Diffusion-Based Synthetic Speech Dataset (DiffSSD).

Performance: The proposed system achieved an average Equal Error Rate (EER) of 3.16% and an AUC of 0.987 across the datasets.
- On ASVspoof 2019, it achieved an EER of 5.97%, outperforming official baselines and remaining competitive with complex ensemble methods.
- On DiffSSD, it achieved an EER of 2.95%, outperforming large semi-supervised models like Wav2Vec2 (3.00%).
- On FoR, it reached an EER of 0.56%, matching high-complexity systems.
Ablation Study: Removing silence frames from the input caused a significant performance degradation (average EER increase of 14.42%), confirming that non-speech segments are vital for detection. Removing voiced frames had a lesser impact (2.89% increase).
Interpretability Findings: Grad-CAM heatmaps showed high activations in silence regions and transitions. The authors interpret this as the predictor capturing the absence of natural reverberation tails in synthetic speech, a feature often missing in deepfake generators.
Robustness: Under a "frozen model" (no retraining), performance degraded significantly under noise and filtering (EER > 50% in some cases). However, fine-tuning the model on perturbed data effectively recovered performance, with EERs returning close to baseline levels across all datasets and perturbation types (e.g., MP3 compression, telephone filtering).

Significance and Claims
The paper claims that its primary significance lies in offering a transparent alternative to black-box detectors. By grounding the detection in Wiener-Hopf linear prediction, the method allows forensic analysts to trace decisions back to the physical properties of the signal (specifically reverberation and statistical inconsistencies in silence). The authors emphasize that this approach does not sacrifice accuracy for interpretability; rather, it achieves state-of-the-art performance with significantly lower computational overhead, making it suitable for broader deployment. The work suggests that the most discriminative artifacts in synthetic speech are not always in the voiced speech itself, but in the statistical inconsistencies of silence and transition boundaries, which are effectively captured by linear prediction coefficients.