Sometin Beta Pass Notin (SBPN): Improving Multilingual ASR for Nigerian Languages via Knowledge Distillation

Technical Summary: Sometin Beta Pass Notin (SBPN)

Problem Statement
Despite Nigeria's linguistic diversity, with over 500 languages, modern multilingual Automatic Speech Recognition (ASR) systems consistently underperform on Nigerian languages compared to high-resource languages like English and French. The authors identify several critical hurdles: acute data scarcity, inconsistent orthography, the necessity of tonal diacritics (in Yorùbá and Igbo), diverse accents, frequent code-switching, and the presence of localized named entities. Existing datasets often consist of read speech based on predefined templates, failing to generalize to real-life conversational scenarios. Furthermore, the dominance of high-resource languages on the internet accelerates linguistic attrition, creating a digital divide that necessitates the development of specialized ASR resources to ensure linguistic continuity.

Methodology
The paper introduces Sometin Beta Pass Notin (SBPN), a multilingual foundational ASR model designed for five specific Nigerian languages: Yorùbá, Hausa, Igbo, Nigerian Pidgin, and Nigerian English. The framework employs a two-stage distillation process:

1. Knowledge Distillation from Monolingual Teachers:

   • Teacher Models: The authors utilized existing state-of-the-art monolingual models (based on architectures like W2v-bert-2.0 and w2v-large-xlsr-53) as "teachers."
   • Language Model Fusion: Recognizing that CTC-based teacher models struggle in low-resource regimes without language modeling, the team curated large-scale N-gram language models for each of the five languages using text corpora from sources like Common-Crawl-100 and specific repositories.
   • Pseudo-Labeling: The teacher models generated hard pseudo-labels for unlabelled audio data. Crucially, these labels were refined using a shallow fusion of the ASR predictions with the curated N-gram language models during inference (using the Flashlight decoder) to reduce Word Error Rates (WER).
   • Data Curation: The pipeline included extensive audio processing: denoising (MossFormer2), speaker diarization (Pyannote), voice activity detection, and language identification. A specific normalization strategy was applied to Nigerian Pidgin to handle non-standard orthography and homophones by clustering words and selecting the most probable token based on the N-gram LM.

2. Iterative Self-Improvement:

   • The multilingual student model (SBPN) was first trained on a combination of ground-truth data and the pseudo-labelled data generated in the first stage.
   • Subsequently, the model underwent a self-improvement loop where it generated new, more accurate pseudo-labels for the unlabelled data, which were then used to further fine-tune the model. This process continued until the average WER plateaued.

Model Architecture
The SBPN models are based on the Recurrent Neural Network-based Transducer (RNNT) architecture, featuring a Fast Conformer encoder, a stateful LSTM-based prediction network, and a feed-forward joint network. An auxiliary CTC head is attached to the encoder to regularize features, which is particularly beneficial when training with noisy pseudo-labelled data. The models are released in two sizes:

• SBPN-Base: 120 million parameters.
• SBPN-Large: 600 million parameters.
  Both are designed to run on CPUs during inference, making them accessible for researchers without large-scale GPU resources.

Key Contributions

• First Multilingual Foundational Model for Nigerian Languages: SBPN is the first model capable of accurately transcribing both read and fast conversational speech across the five target languages while performing spoken language identification (SLID).
• Curated N-gram Language Models: The authors created the first large-scale N-gram language models for these five languages and demonstrated their critical role in improving pseudo-label quality and overall ASR performance.
• Validation of Pseudo-Labeling in Low-Resource Settings: The study proves that large-scale pseudo-labelling, when combined with knowledge distillation and language model fusion, significantly improves recognition accuracy for low-resource languages, not just high-resource ones.
• Open Release: The models (SBPN-Base and SBPN-Large) and their checkpoints are released as open foundation models to facilitate further research into the region's phonetic and cultural landscape.

Results

• Performance Gain: SBPN significantly bridges the performance gap, achieving an average 29% relative reduction in Word Error Rate (WER) over monolingual baselines.
• Comparison with Teachers: The SBPN-Large model outperformed the monolingual teacher models (even those fused with N-gram LMs) by an average of 16% relative WER reduction.
• State-of-the-Art Comparison: SBPN models outperformed existing state-of-the-art multilingual models (such as AfriHuBERT, mHuBERT-147, and MMS-1B) on major benchmarks including Common Voice and Fleurs. Notably, the 600M parameter SBPN-Large outperformed billion-parameter models on average for Nigerian languages.
• Robustness: SBPN-Large demonstrated superior stability in transcribing fast conversational speech (up to 2x normal speed) compared to monolingual teachers, whose performance degraded sharply with increased speaking rates.
• Language Identification: The models achieved near-perfect micro-averaged F1 scores (100% for most languages) in identifying the spoken language, outperforming dedicated audio-based and text-based language identification models for Nigerian English and Pidgin.
• Diacritic Challenges: While performance improved, the paper notes that predicting diacritical marks remains a challenge, particularly for Yorùbá, where the WER gap between predictions with and without diacritics remains significant, though reduced compared to baselines.

Significance
The paper claims that SBPN represents a significant step toward counteracting the digital divide and ensuring linguistic continuity for Nigerian languages. By demonstrating that "average teachers" (monolingual models) can be distilled into a superior "student" (multilingual model) through rigorous pseudo-labelling and language model fusion, the work provides a scalable framework for low-resource language processing. The release of CPU-friendly models aims to democratize access to ASR technology for linguists and researchers in the region, fostering further exploration of the area's rich phonetic and cultural diversity. The authors modestly acknowledge that while SBPN is a foundational step, significant work remains to address diacritic prediction and to expand the framework to cover more of Nigeria's 500+ languages.