Speaker-Invariant Representation Learning for Spoofing Detection via Gradient Reversal and A Variational Information Bottleneck

Technical Summary: Speaker-Invariant Representation Learning for Spoofing Detection via Gradient Reversal and A Variational Information Bottleneck

Problem Statement
The paper addresses the critical issue of poor generalization in spoofing detection systems when transitioning from in-domain to out-of-domain settings. The authors identify speaker bias as a primary cause of this fragility. In datasets like ASVspoof 5, there is often a disjoint distribution of speaker identities between bona fide and spoofed partitions. Consequently, models may learn "shortcuts" by associating specific voice identities with the spoofing label rather than learning intrinsic acoustic artifacts of synthesis or manipulation. This leads to overfitting on training distributions and failure on unseen speakers or novel attack vectors. Furthermore, previous attempts to enforce speaker invariance via adversarial training relied on speaker labels within the spoofing dataset, which are often unavailable or insufficiently diverse, and risk discarding beneficial spoofing cues during aggressive adversarial suppression.

Methodology
The authors propose IVSpk-VIB, a teacher-student framework designed to learn speaker-invariant representations for spoofing detection without requiring speaker labels for the target dataset. The architecture consists of three core components:

1. Teacher-Student Framework with Gradient Reversal:

   • Teacher: A pre-trained speaker recognition model (trained on the large-scale VoxCeleb2 dataset) acts as a frozen teacher. It extracts speaker embeddings that encode rich speaker identity information.
   • Student: A spoofing detection model (based on XLSR and MHFA) is trained to predict the spoofing label.
   • Adversarial Learning: A Gradient Reversal Layer (GRL) is inserted between the student's feature extractor and a speaker classifier head. The student is guided to minimize the spoofing loss while maximizing the error in predicting the teacher's speaker embeddings. This encourages the student to learn representations that are invariant to speaker identity.

2. Variational Information Bottleneck (VIB):

   • To prevent the adversarial training from removing essential spoofing-related cues along with speaker information, a VIB is integrated into the speaker classification branch.
   • The VIB imposes an information-theoretic constraint on the latent representation. It models the latent variable as a Gaussian posterior and enforces a Kullback-Leibler (KL) divergence regularization against a standard normal prior.
   • This creates a controlled bottleneck, ensuring that the suppression focuses on principal speaker-related information while preserving the discriminative evidence necessary for spoofing detection.

3. Training Objective:
   The overall optimization minimizes a combined loss function: the cross-entropy loss for spoofing detection, the mean squared error (MSE) between student and teacher speaker embeddings (via GRL), and the VIB regularization term.

Key Contributions

• Speaker Bias Analysis: The authors empirically demonstrate that the ASVspoof 5 training set creates a spurious correlation between speaker identity and the spoofing label. Latent clustering analysis reveals that bona fide and spoofed utterances occupy distinct, non-overlapping regions in the speaker embedding space, confirming that models can exploit identity-based shortcuts.
• Label-Free Speaker Invariance: They introduce a teacher-student framework that suppresses voice identity influences without relying on speaker labels in the target spoofing dataset, overcoming the limitations of prior methods that required in-domain speaker annotations.
• Controlled Adversarial Training: By incorporating the VIB, the method regulates the adversarial suppression process, managing the trade-off between removing biased voice features and preserving spoofing-relevant information.
• Extensive Evaluation: The model is evaluated across nine diverse out-of-domain datasets, demonstrating consistent improvements over baselines.

Results
The proposed MHFA-IVSpk-VIB model was evaluated against several baselines (AASIST, Conformer, MHFA, and the prior SInMT method) across nine datasets, including ASVspoof 2019/2021, CodecFake, and LibriSeVox.

• Generalization: The model achieved a 25.7% relative reduction in Equal Error Rate (EER) compared to the MHFA baseline on the pooled dataset (combining all evaluation sets).
• Comparison with Prior Work: It outperformed the SInMT baseline (which used in-domain speaker labels) by 11.3% in relative pooled EER, highlighting the benefit of using a teacher pre-trained on diverse external data (VoxCeleb) rather than limited in-domain labels.
• Benchmarking: When compared to the top four submissions of the ASVspoof 5 Challenge, the proposed model showed superior out-of-domain robustness. While top challenge systems suffered significant performance drops on unseen datasets (e.g., rising from 3.30% EER on ASVspoof 5 to 17.33% on ASVspoof 2019), the proposed model maintained lower error rates across these shifts, achieving a 62% relative improvement in average EER compared to the best-performing challenge system (T23) across all benchmarks.

Significance
The paper claims that the integration of a Variational Information Bottleneck is essential for balancing speaker invariance and spoofing discriminability. Without VIB, adversarial training risks discarding critical spoofing cues; with VIB, the system achieves a controlled form of invariant learning. The results suggest that leveraging broad speaker diversity from external sources via a teacher model is more effective for generalization than relying on the limited speaker diversity found within typical spoofing detection training sets. This approach offers a principled solution to the speaker bias problem, enhancing the reliability of voice biometric systems against sophisticated generative attacks in real-world, out-of-domain scenarios.