Building an ASR Solution for Training and Assessing Children's Reading

Technical Summary: Building an ASR Solution for Training and Assessing Children's Reading in Bambara

Problem Statement
Automatic Speech Recognition (ASR) systems for children's reading remain underdeveloped for most African languages, including Bambara. Existing corpora rarely represent the distinct acoustic profiles of children, such as irregular speech rates and high fundamental frequencies. In Mali, structural constraints in the school system hinder scalable, individual reading assessment, with estimates suggesting 90% of late-primary children lack reading proficiency. There is a critical need for an ASR-based assessment tool adapted specifically to the Bambara language, the domain of educational reading, and the child population, capable of functioning in low-resource, offline settings.

Methodology
The authors present An bɛ kalan, an open-source system developed through an end-to-end pipeline involving data collection, benchmark construction, model adaptation, and field validation.

• Data Collection: A mobile application was deployed in Bamako to record 60 children (aged 9–18) reading aloud from 22 Bambara textbooks. The protocol yielded 55 hours of raw speech, which was cleaned to retain 47 hours (85% retention rate). The dataset includes 1.6 hours of unique reading material and 45.6 hours of duplicate readings of the same texts by different voices.
• Benchmark Construction: A public benchmark (RobotsMali/an-be-kalan-bench) was constructed using texts from the GAIFE collection. The test set comprises unique readings from 11 children across 11 books, ensuring no overlap with training data to support fair generalization assessment.
• Model Adaptation and Architecture: The study compares two ASR architectures fine-tuned on pre-trained Bambara foundations (derived from jeli-asr and African Next Voices):
  • QuartzNet: A compact, fully convolutional architecture (~18M parameters) using character-based CTC decoding with a 63-character vocabulary.
  • Soloni: A Bambara-adapted Fast-Conformer model (~114M parameters) utilizing a Time-and-Duration Transducer (TDT) decoder paired with a Connectionist Temporal Classification (CTC) objective and a SentencePiece BPE tokenizer (512 vocab size).
• Experimental Protocol: Four configurations were tested for each architecture to evaluate the impact of vocabulary constraints, data augmentation (SpecAugment), and acoustic diversity (using unique vs. duplicate data). Training utilized early stopping based on validation Word Error Rate (WER).
• Deployment Constraints: Both models were selected for their ability to support offline, low-latency inference on low-specification mobile devices (specifically Tecno Camon 40), a necessity for the target deployment environment in Mali.

Key Results

• Model Performance: The best-performing configuration, Soloni (Exp3), achieved a WER of 0.22 and a Character Error Rate (CER) of 0.08. This represents a substantial improvement over the Soloni base checkpoint (WER 0.42) and significantly outperforms the best QuartzNet configuration (WER 0.40).
• Impact of Data Augmentation and Repetition:
  • SpecAugment: While SpecAugment helped stabilize training and reduce divergence, no augmented configuration outperformed the best unaugmented model.
  • Duplicate Data: Adding repeated readings of the same texts provided architecture-dependent benefits. It substantially improved QuartzNet (reducing WER from 0.93 to 0.40) but offered only marginal gains for Soloni (0.23 to 0.22). This suggests that for stronger architectures, raw acoustic volume from repeated texts is less critical than lexical and phonetic diversity.
• Demographic Analysis: Disaggregated analysis revealed that children under 10 remain the primary source of residual errors. Even with the best model, the WER for this cohort dropped from 0.56 to 0.29, which remains significantly higher than for older cohorts (10–15 years).
• Inference Speed: Both models demonstrated feasibility for on-device execution, with QuartzNet processing 8-second segments in <300ms and Soloni in 800–900ms.

Significance and Claims
The paper claims that An bɛ kalan provides a reproducible, open-source framework for assessing children's reading in Bambara, bridging the gap between field data collection and classroom application. The study establishes that:

1. Targeted Adaptation is Critical: Fine-tuning existing ASR models for the specific domain of child reading in a low-resource language yields significant performance gains.
2. Data Strategy Matters: For advanced architectures like Soloni, future improvements are more likely to stem from targeted data collection focusing on lexical and phonetic diversity (particularly from younger readers) rather than simply accumulating more hours of repeated readings or applying standard augmentation techniques.
3. Field Viability: The system is pedagogically acceptable and technically feasible for classroom use in Mali, with 9 out of 10 classroom trials supporting continued use.

The authors position the Soloni model (WER 0.22) as the new reference baseline for Bambara child-reading ASR, while acknowledging that smaller architectures like QuartzNet may remain relevant in contexts where memory and device cost are the dominant constraints. The work emphasizes the necessity of addressing the specific acoustic challenges of younger children to further reduce residual errors.