🤖 AI

Poisoned Acoustics

This paper demonstrates that training-data poisoning attacks on acoustic vehicle classification can achieve high targeted misclassification rates with negligible data corruption while remaining undetectable via aggregate accuracy monitoring due to structural bounds imposed by class imbalance, necessitating a shift toward cryptographically verifiable data provenance defenses.

Original authors: Harrison Dahme

Published 2026-02-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Harrison Dahme

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to recognize different vehicles on a busy city street just by listening to their engine sounds. You want the robot to know the difference between a tiny car, a loud truck, a bus, and a motorcycle.

This paper is a warning label for that robot. It shows how a sneaky hacker can trick the robot into making a dangerous mistake with almost no effort, and why the usual safety checks won't catch them.

Here is the story broken down into simple parts:

1. The Setup: A Skewed Classroom

The researchers used a dataset called MELAUDIS, which is like a giant audio library of city sounds.

The Problem: The library is very unbalanced. It has thousands of recordings of Cars (84% of the library) but very few Trucks (only about 3%).
The Analogy: Imagine a classroom with 100 students. 84 of them are named "Car," and only 3 are named "Truck." The teacher (the AI) spends most of its time studying the "Car" students.

2. The Attack: The "Whisper" Poison

The researchers asked: How many students do we need to bribe to make the teacher think "Truck" is actually "Car"?

They tried two methods:

Method A: The Label Flip (The "Name Tag" Swap)
The hacker takes just 48 of the "Truck" audio files (less than 0.5% of the total data) and changes their name tags from "Truck" to "Car."
- The Result: The robot learns that the sound of a truck is actually the sound of a car. When a real truck drives by later, the robot confidently says, "That's a car!"
- The Stealth: Because "Trucks" were so rare to begin with, the robot's overall test score (how well it knows everything) barely changed. It's like if you swapped the names of 3 students in a class of 100; the teacher's overall grade for the class stays the same, even though those 3 specific students are now misidentified.
Method B: The Backdoor (The "Secret Handshake")
Usually, hackers add a tiny, invisible "trigger" (like a specific pattern of static) to the audio. They hope that when the robot hears this trigger, it ignores the sound and just says "Car."
- The Surprise: The researchers found that because there were so few "Truck" examples to begin with, the robot didn't even need the secret trigger anymore. The "Label Flip" was so effective that the trigger became useless. The attack collapsed into the simpler "Name Tag Swap."

3. The Big Discovery: Why Safety Checks Fail

The most important part of this paper is a mathematical proof about why we can't trust standard safety checks.

The Myth: "If the AI's overall accuracy drops, we know it's been hacked."
The Reality: If you attack a rare, dangerous category (like Trucks, or in real life, a rare disease or a specific type of bomb), the overall accuracy will not drop enough to notice.
The Analogy: Imagine a security guard checking 1,000 bags. If 999 bags are fine, but 1 bag contains a bomb, the guard's "success rate" is still 99.9%. If a hacker swaps the label on that 1 bomb bag to say "It's a toy," the guard's success rate is still 99.9%. The system looks perfect, but it has a fatal blind spot.

The paper proves that as long as the target group is small (a "minority class"), a hacker can destroy the system's ability to recognize that group without the system's overall score ever blinking an eye.

4. The Solution: The "Unbreakable Ledger"

Since we can't rely on the AI's test scores to tell us if it's been hacked, we need a new way to protect the data. The authors propose a Trust-Minimized Pipeline:

The Idea: Instead of trusting the people who label the data, we use cryptography (super-secure math) to create a digital "fingerprint" for every single piece of data.
The Analogy: Imagine every audio file has a unique, unchangeable wax seal.
1. When the data is collected, it gets a seal.
2. When it's labeled, the label gets a seal.
3. These seals are linked together in a chain (called a Merkle Tree).
How it stops the hacker: If a hacker tries to change even one label (like changing "Truck" to "Car"), the wax seal breaks, and the whole chain of seals becomes invalid. The system immediately stops and says, "Something has been tampered with!"

They also suggest using Post-Quantum Signatures. This is like using a lock that even a super-computer from the future (a quantum computer) cannot pick. It ensures that even if hackers wait 10 years to break the code, the data will still be safe.

Summary

The Threat: Hackers can trick AI into ignoring dangerous things (like trucks or bombs) by changing just a tiny fraction of the training data.
The Trap: Standard safety checks (looking at overall accuracy) will not catch this because the mistake is hidden in the rare data.
The Fix: We need to stop trusting the data files themselves and start trusting the mathematical proof that the files haven't been touched. We need to sign every step of the process with unbreakable digital locks.

In short: You can't just check the final grade to see if a student was cheated; you have to check the integrity of every single homework assignment they ever turned in.

1. Problem Statement

The paper addresses the vulnerability of deep learning models used in safety-critical acoustic sensing (e.g., traffic monitoring, autonomous vehicles) to training-data poisoning attacks. Specifically, it investigates whether an adversary can induce targeted misclassification (e.g., classifying heavy trucks as passenger cars) by corrupting a negligible fraction of the training data, while remaining undetectable by standard aggregate accuracy metrics.

The core challenge lies in class imbalance. Real-world datasets often contain rare but safety-critical classes (e.g., trucks, anomalies) that make up a tiny fraction of the total data. The authors hypothesize that this structural imbalance allows attackers to manipulate the model's behavior on these specific classes without significantly impacting the model's overall performance, thereby evading detection.

2. Methodology

Dataset and Model

Dataset: The study uses MELAUDIS, a dataset of $\approx$ 9,600 single-vehicle audio clips from Melbourne urban intersections. It features six classes with severe imbalance: Car (84.4%), Tram (6.3%), and four minority classes (Truck, Bus, Motorcycle, Bicycle) each comprising $\approx$ 2.3–2.7% of the data.
Model: A compact 2-D Convolutional Neural Network (CNN) trained on 128-bin log-mel spectrograms. The clean baseline achieves 87.6% test accuracy.

Attack Variants

The authors evaluate two attack strategies against the "Truck" class (target) relabeled as "Car" (adversarial class):

Targeted Label-Flipping Attack: A fraction $p$ of "Truck" training records are relabeled as "Car." The audio and spectrograms remain unchanged.
Backdoor Trigger Attack: An extension of the label-flip where poisoned spectrograms are stamped with a 12 $\times$ 12-pixel white patch in the bottom-right corner. This acts as a feature-space trigger (inaudible in raw audio) intended to activate the misclassification only when the patch is present.

Threat Model

The attacker has write access to unsigned stages of the ML pipeline (e.g., annotation CSVs, feature manifests, or spectrogram generation code), simulating a supply-chain compromise or malicious insider.

3. Key Contributions

Empirical Upper Bound on Poisoning Rate: The study establishes that effective targeted attacks can be achieved with a poisoning rate ( $p^*$ ) of $\le$ 0.5% (corrupting only 48 out of 9,600 records).
Proof of Structural Stealth: The authors prove analytically that the maximum detectable accuracy degradation ( $\Delta Acc_{max}$ ) from a complete targeted attack on a minority class is bounded by the class fraction ( $\beta$ ):
$\Delta Acc_{max} \le \beta$
Since $\beta$ for trucks is $\approx$ 3%, even a 100% failure rate on trucks only reduces overall accuracy by $\approx$ 3%, which is often indistinguishable from normal training variance.
Trigger-Dominance Collapse: A novel discovery that for minority classes, backdoor triggers become functionally redundant. When the target class is small, the model cannot form a coherent representation of the "clean" class regardless of the trigger. Consequently, the "Clean ASR" (misclassification without trigger) equals the "Triggered ASR," effectively degenerating the backdoor attack into a pure label-flip attack.
Trust-Minimized Defense Architecture: Proposes a cryptographically verifiable pipeline using Merkle trees for dataset commitment and Post-Quantum Signatures (ML-DSA-65 / CRYSTALS-Dilithium3) to ensure data provenance and detect tampering.

4. Key Results

Attack Success Rate (ASR): At a poisoning rate of 0.5% (48 records), the label-flip attack achieved an ASR of 95.7% (95% CI: 88–100%). This means 95.7% of clean "Truck" test samples were misclassified as "Cars."
Stealth: The overall test accuracy remained statistically indistinguishable from the clean baseline (87.6% vs. 87.4–87.6%). Standard aggregate monitoring failed to detect the attack.
Backdoor Redundancy: In the backdoor experiment, the Clean ASR (94.87%) was identical to the Triggered ASR (94.87%). The patch provided no additional attack capability, confirming the "trigger-dominance collapse" phenomenon.
Generalizability: The authors argue these findings apply to other modalities (Thermal/FLIR, RF spectrum, Radar, LiDAR) where rare, safety-critical targets exist within imbalanced datasets.

5. Significance and Implications

Failure of Aggregate Metrics: The paper demonstrates that aggregate accuracy monitoring is provably insufficient for detecting targeted attacks on minority classes. Relying on overall model performance creates a false sense of security in safety-critical systems.
Supply Chain Vulnerability: The attack surface is not limited to the model weights but extends to the entire data pipeline (annotations, feature extraction). A single unsigned stage can compromise the system.
Cryptographic Necessity: The authors argue that statistical defenses are inadequate. The only reliable mitigation is cryptographic verification of data lineage. They propose a pipeline where every stage signs its output using Post-Quantum cryptography (to prevent "harvest-now, forge-later" attacks) and commits the dataset to a Merkle tree, allowing for immediate detection of any data manipulation.
Safety Impact: In real-world scenarios, such an attack could cause autonomous vehicles to ignore heavy trucks in weight-restricted zones or fail to detect rare anomalies in surveillance, leading to catastrophic failures that standard testing would never reveal.

Conclusion

"Poisoned Acoustics" reveals a fundamental structural weakness in machine learning security: class imbalance amplifies the stealth of targeted poisoning attacks. By corrupting less than 0.5% of data, an attacker can effectively blind a model to specific safety-critical objects without triggering any alarms in standard evaluation metrics. The paper concludes that securing the data supply chain via cryptographic provenance is the only viable defense against such invisible threats.