MultiAPI Spoof: A Multi-API Dataset and Local-Attention Network for Speech Anti-spoofing Detection

This paper introduces MultiAPI Spoof, a large-scale dataset featuring 230 hours of synthetic speech from 30 diverse APIs, and proposes Nes2Net-LA, a local-attention network that achieves state-of-the-art performance in speech anti-spoofing detection and API tracing under real-world conditions.

The Gist

Imagine you are a security guard at a high-tech club. Your job is to check IDs at the door to make sure everyone is who they say they are. In the world of voice technology, this is called Speech Anti-Spoofing. The "ID" is a person's voice, and the "fake IDs" are AI-generated voices (deepfakes) trying to sneak in.

For a long time, security guards (the AI models) have been trained using a very small, specific list of fake IDs. They know exactly what the "Standard Fake" looks like. But in the real world, criminals aren't just using one type of fake ID; they are using hundreds of different tools, apps, and websites to generate voices.

This paper introduces a solution to that problem, which we can break down into three main parts: The New Training Manual, The Smarter Guard, and The Detective Work.

1. The New Training Manual: "MultiAPI Spoof"

The Problem:
Imagine training a security guard to spot fake IDs, but you only show them fakes made by one specific printer. When a criminal shows up with a fake ID made by a different printer, the guard has no idea what to do. That's the current state of voice security. Most research uses a tiny, outdated list of fake voices, while real-world scammers use dozens of different commercial and open-source AI tools.

The Solution:
The authors created a massive new training dataset called MultiAPI Spoof.

• The Analogy: Instead of showing the guard one type of fake ID, they gave them a library containing 230 hours of audio generated by 30 different AI "printers" (APIs).
• What's in the library? It includes voices from big commercial companies, open-source projects, and random websites.
• The Result: By training on this diverse "library," the security system learns to recognize the concept of a fake voice, not just the specific look of one fake. It's like teaching a guard to spot "suspicious behavior" rather than just memorizing one specific face.

2. The Smarter Guard: "Nes2Net-LA"

The Problem:
Even with a better training manual, the guard needs a better way to look at the evidence. The previous best model (Nes2Net) was like a detective who only looked at a crime scene one room at a time, strictly in order. They missed the connection between Room 1 and Room 3, which might hold the key to solving the case.

The Solution:
The authors built a new model called Nes2Net-LA (Local-Attention Network).

• The Analogy: Imagine the detective now has a sliding window they can look through. Instead of just staring at the room right in front of them, they can glance at the rooms immediately to the left and right.
• How it works: This "Local Attention" allows the AI to look at small groups of audio features together. It helps the model understand the context of the sound. Is this voice smooth? Does it have a weird glitch right here?
• The Result: This new guard is much better at spotting subtle, high-tech fakes that the old guard missed. It achieved the best results (State-of-the-Art) in testing, proving that looking at the "neighborhood" of the data helps solve the crime.

3. The Detective Work: "API Tracing"

The Problem:
Usually, security just asks: "Is this voice real or fake?" But what if you need to know which AI tool the criminal used? Was it "VoiceBot A" or "DeepFake Pro"? This is crucial for tracking down the source of misinformation.

The Solution:
The paper introduces a new task called API Tracing.

• The Analogy: Instead of just saying "This is a fake ID," the guard now has to say, "This is a fake ID, and it was printed by Printer #14."
• The Challenge: The system is great at identifying printers it has seen before (the "Seen" APIs). However, when a criminal uses a brand-new, unknown printer (an "Unseen" API), the system sometimes struggles. It's like a detective who knows all the local forgers but gets confused when a new forger from another city shows up.
• The Future: The paper admits this is hard work. The AI needs to learn deeper patterns to identify any new tool, not just the ones it studied in school.

The Big Takeaway

This paper is a wake-up call to the security world: The old training methods are too narrow.

1. We need better data: We must train our systems on the messy, diverse reality of the internet (the MultiAPI Spoof dataset).
2. We need smarter models: We need AI that looks at the context of the sound, not just isolated parts (Nes2Net-LA).
3. We need to know the source: We need to move beyond "Is it fake?" to "Who made it?" (API Tracing).

By combining a massive, diverse dataset with a smarter, context-aware model, the authors have built a security system that is much harder to trick, helping us stay safe in an age where anyone can sound like anyone else.

Technical Summary

Here is a detailed technical summary of the paper "MultiAPI Spoof: A Multi-API Dataset and Local-Attention Network for Speech Anti-spoofing Detection."

1. Problem Statement

Current speech anti-spoofing research faces a critical domain gap between academic benchmarks and real-world deployment scenarios.

• Limitation of Existing Benchmarks: Most existing datasets rely on a narrow set of public, open-source Text-to-Speech (TTS) or Voice Conversion (VC) models.
• Real-World Complexity: In practice, commercial systems utilize diverse, often proprietary, closed-source APIs. Researchers cannot access the internal architectures or data pipelines of these systems.
• Consequence: Models trained on current open-source benchmarks fail to generalize to real-world synthetic speech generated by unseen commercial APIs. Furthermore, existing methods typically only distinguish between "bona fide" (real) and "spoofed" (fake) audio, lacking the ability to trace the specific source (API) of the spoof.

2. Methodology

A. The MultiAPI Spoof Dataset

To bridge the gap, the authors introduced MultiAPI Spoof, a large-scale dataset designed for both detection and source tracing.

• Scale: Approximately 230 hours of synthetic speech.
• Diversity: Generated by 30 distinct APIs, categorized into:
  1. Commercial TTS services (proprietary).
  2. Open-source neural TTS/VC models.
  3. Web-based TTS platforms.
• Balance: Maintains a 1:1 ratio between spoofed audio and bona fide speech (sourced from CommonVoice).
• Split Strategy:
  • Training/Dev: APIs A0–A20 (70%/10% split).
  • Dev (Unseen): APIs A21–A23.
  • Evaluation (Unseen): APIs A24–A29.
  • This allows for evaluation under both Seen (training APIs) and Unseen (completely new APIs) conditions.

B. Proposed Model: Nes2Net-LA

The authors propose Nes2Net-LA, an enhanced version of the Nested Res2Net (Nes2Net) architecture.

• Base Architecture (Nes2Net-X): Uses multi-scale feature extraction where audio segments are split into channel-wise subsets and processed hierarchically via Convolution, Weighted Summation (WS), and Squeeze-and-Excitation (SE) modules.
• The Innovation (Local Attention):
  • Problem: The original Nes2Net-X processes blocks strictly hierarchically, limiting long-range communication between nested blocks.
  • Solution: Nes2Net-LA integrates a Local Attention mechanism between nested blocks.
  • Mechanism: A sliding-window neighborhood $N(i, j)$ is defined around each block. A scaled dot-product self-attention operator aggregates features from neighboring blocks within a small window (radius $K=1$).
  • Benefit: This enhances local context modeling and fine-grained feature extraction without the computational cost of global attention on long sequences, leading to more robust representations.

C. API Tracing Task

A novel task is defined to identify the specific generation API of a spoofed audio sample.

• Formulation: Treated as a multi-class classification problem (21 seen classes + 1 "unseen" class).
• Inference: Samples with a maximum predicted probability below a threshold are classified as the "unseen" class, enabling open-set recognition.

3. Key Contributions

1. MultiAPI Spoof Dataset: A comprehensive resource comprising 230 hours of audio from 30 diverse APIs, addressing the lack of commercial/proprietary data in current research.
2. Nes2Net-LA Architecture: A novel local-attention enhanced network that outperforms existing baselines by improving cross-block feature interaction and robustness.
3. API Tracing Benchmark: The introduction of a fine-grained source attribution task, moving beyond binary detection to identifying the specific generation source.
4. Empirical Validation: Demonstrated that training on this diverse dataset significantly improves performance on both the new dataset and existing public benchmarks (ITW, AI4T).

4. Experimental Results

Anti-Spoofing Performance

• Impact of Dataset: Adding the MultiAPI Spoof training set to existing benchmarks (TIMIT, ODSS, etc.) drastically reduced Error Rates.
  • Example: XLSR+AASIST EER on the MultiAPI Spoof test set dropped from 7.30% (without MultiAPI data) to 0.70% (with MultiAPI data).
  • Generalization: Improvements were observed not only on seen APIs but also on unseen APIs, proving the model learned robust features rather than overfitting to specific sources.
• Model Performance: XLSR+Nes2Net-LA achieved State-of-the-Art (SOTA) results across multiple benchmarks (ITW, AI4T, MultiAPI Spoof) without using data augmentation or sample pruning.
  • On the MultiAPI Spoof test set (Unseen), Nes2Net-LA achieved an EER of 5.64%, outperforming Nes2Net-X (5.64% vs 5.64% overall, but superior on unseen splits) and other SOTA models like XLSR+LRC.

API Tracing Performance

• Seen APIs: High performance with F1 scores around 0.93–0.94 for both development and evaluation sets.
• Unseen APIs: High precision (0.97) but low recall (0.52), indicating the model is conservative; it correctly identifies known APIs but struggles to correctly classify truly unseen APIs, often misclassifying them as "unseen" rather than specific new classes.
• Analysis: t-SNE visualizations revealed that unseen API embeddings do not form separable clusters, suggesting a need for better invariant representation learning for zero-shot scenarios.

5. Significance

• Bridging the Gap: The paper provides the first major dataset specifically targeting the diversity of commercial and proprietary TTS APIs, making anti-spoofing research more relevant to real-world security threats.
• Robustness: The study proves that exposure to diverse synthesis mechanisms during training is crucial for generalizing to unseen attacks.
• Architectural Advancement: The Local Attention mechanism in Nes2Net-LA offers a computationally efficient way to improve feature discrimination in high-dimensional speech representations.
• Future Direction: The API tracing task highlights the difficulty of zero-shot attribution, setting a new benchmark for future research into source identification and forensic analysis of deepfakes.

Resources: The code and dataset have been released publicly to facilitate further research in this domain.