Text-Dependent Speaker Verification (TdSV) Challenge 2024: Team Naive System Report

Technical Summary: Team Naïve System for the 2024 Text-Dependent Speaker Verification Challenge

Problem Statement
The paper addresses the Text-Dependent Speaker Verification (TdSV) task, a biometric security challenge requiring the simultaneous verification of a speaker's identity and the correctness of a specific spoken phrase. Unlike text-independent systems, TdSV mandates that the system recognize both "who" is speaking and "what" is being said. This dual requirement introduces complexity, necessitating precise modeling of voice characteristics alongside phrase-specific patterns to resist spoofing and replay attacks. The specific context is the 2024 TdSV Challenge, which utilized the DeepMine corpus containing Persian and English utterances recorded under real-world conditions with background noise. The authors faced significant constraints, including a nine-week timeline and limited access to dedicated GPU resources.

Methodology
The Team Naïve system employs a dual-module architecture designed to integrate speaker verification and phrase verification through an ensemble approach.

• Speaker Verification Module (Ensemble): The core of the system consists of three distinct Speaker Embedding Extractors (SEEs) whose outputs are fused:
  1. NeXt-TDNN: A state-of-the-art architecture fine-tuned on the challenge dataset using pre-trained weights from VoxCeleb 1 & 2. It utilizes a TS-ConvNeXt block design with multi-scale temporal convolutions and Global Response Normalization (GRN) to capture inter-frame and intra-frame contexts.
  2. ResNet-TDNN: Another model fine-tuned on the challenge dataset, leveraging residual connections to learn deep speaker representations. It was optimized with specific hyperparameters to adapt to the nuances of the DeepMine data.
  3. EfficientNet-A0 (EfficientNet-Absolute Zero): A lightweight model trained from scratch specifically on the challenge dataset. Designed for keyword spotting, this model was adapted to generate speaker embeddings, offering a resource-efficient component to the ensemble.
• Phrase Verification Module: To verify the spoken content, the system utilizes a wav2vec 2.0 model (specifically the lighteternal/wav2vec2-large-xlsr-53-greek variant). This model acts as a feature extractor for the spoken content, classifying the audio into predefined phrase categories.
• Scoring and Fusion:
  • Embedding Generation: The three SEEs process trial audio to generate high-dimensional embeddings.
  • Similarity Calculation: A Scorer module computes the cosine similarity between enrollment and trial embeddings.
  • Normalization: The system applies S-norm (Score Normalization) to adjust raw similarity scores based on the distribution of a cohort of embeddings from the training dataset, compensating for environmental variations.
  • Final Decision: The final score is derived by fusing the normalized speaker scores from the three models and multiplying this result by the probability output from the Phrase Classifier.

Key Contributions and System Design

• Resource-Constrained Adaptation: The paper highlights a strategy of adapting existing state-of-the-art models (NeXt-TDNN and ResNet-TDNN) pre-trained on VoxCeleb rather than training from scratch, a decision driven by the limited challenge duration and hardware resources.
• Hybrid Ensemble: The integration of a lightweight, scratch-trained EfficientNet-A0 model alongside fine-tuned heavyweights is presented as a method to strengthen ensemble performance in low-resource settings.
• Comprehensive Data Augmentation: The training pipeline incorporates extensive augmentation techniques, including noise injection (SNR 0-15 dB), reverberation via Room Impulse Responses (RIR), frequency band dropping, and temporal chunk dropping, to simulate real-world acoustic conditions.
• Preprocessing: The system employs Voice Activity Detection (VAD) to remove silence from audio clips before processing, preventing artifacts from non-speech segments from misleading the model.

Results
The system achieved the following performance metrics on the challenge evaluation set:

• Minimum Detection Cost Function (MinDCF): 0.0461
• Equal Error Rate (EER): 1.3%

The Detection Error Tradeoff (DET) curves indicate that the system performs robustly across different demographic and linguistic groups. The Male subset demonstrated the lowest False Negative Rates, while the Female subset showed slightly higher error rates. Performance across Persian and English language groups was consistent, though marginally lower than the overall male performance. The system maintained competitive performance in both text-constrained and text-independent task variations.

Significance and Claims
The authors claim that their system demonstrates the effectiveness of multi-model ensemble learning for both speaker and phrase verification. By combining advanced neural architectures with a robust scoring mechanism (S-norm) and score fusion, the system achieves strong generalization across diverse conditions.

The paper modestly frames its significance as a practical solution that leverages existing resources effectively under tight constraints. It notes that the use of the EfficientNet-A0 model for speaker verification represents one of the first instances of applying this specific architecture to the domain. The authors conclude that while the system performs well, future work could focus on refining architectures to minimize performance gaps across gender and language subsets and further improving phrase verification accuracy under variable conditions.