S-DiverSe: Spanish Diverse Speech

Technical Summary: S-DiverSe: Spanish Diverse Speech

Problem Statement
While Automatic Speech Recognition (ASR) has achieved remarkable success on standard speech benchmarks, it continues to struggle with speech affected by neurological conditions such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and stroke. These conditions often cause dysarthria, characterized by reduced articulation clarity, altered prosody, and variable intelligibility. A primary obstacle to advancing ASR for these populations is the scarcity of large, high-quality datasets, particularly for Spanish. Existing Spanish resources are limited in clinical coverage, domain diversity, or recording conditions, often relying on controlled clinical settings or biased demographics (e.g., elderly speakers). This lack of benchmarks hinders the development of robust ASR systems for pathological speech and limits progress in assistive communication tools.

Methodology
The authors introduce S-DiverSe (Spanish Diverse Speech), a corpus designed to bridge this gap. The dataset consists of 3.2 hours of "in-the-wild" Spanish speech recorded from 22 speakers diagnosed with ALS, PD, or post-stroke conditions.

• Data Collection: Audio was sourced from YouTube, filtered based on self-reported diagnoses and perceptual evidence of non-normative speech, with a preference for spontaneous contexts like interviews.
• Annotation: The corpus contains 444 manually transcribed audio segments. Three native Spanish-speaking linguists performed orthographic transcription, labeled speaker sex and condition, and assigned intelligibility scores on a 1–5 ordinal scale. Unintelligible segments were marked as <unk>, and filled pauses were retained.
• Dataset Composition: The corpus is male-dominant (87.4%) and ALS-dominant (78.1%), reflecting the availability of in-the-wild data rather than collection bias. It includes metadata on sex, condition, and intelligibility.
• Experimental Setup: S-DiverSe was used exclusively for evaluation due to its high variability and modest size, which would make training splits unreliable. The study evaluated four ASR systems: Whisper-large-v3, Voxtral-Mini, omniASR CTC 1B v2, and the commercial ElevenLabs Scribe v2.
• Adaptation Strategies: The authors investigated two adaptation approaches using data from other neuro-affected corpora (TORGO and NeuroVoz) and Spanish Common Voice:
  1. Fine-tuning: Including full fine-tuning (FFT) and Low-Rank Adaptation (LoRA) applied to various model components (encoder, attention layers, etc.).
  2. Text Post-processing (PP): A rule-based heuristic approach to mitigate hallucinations by analyzing character repetition, collapsing consecutive repeated words, and reducing repeated phrases.

Key Contributions

1. S-DiverSe Corpus: The first in-the-wild Spanish dataset covering multiple neurological conditions (ALS, PD, stroke) with human transcripts and metadata. The authors release annotations to foster open research.
2. Benchmarking: Evaluation of four state-of-the-art ASR systems on S-DiverSe, providing a baseline for Spanish pathological speech recognition.
3. Adaptation Analysis: A comparative study of fine-tuning versus text post-processing for adapting ASR models to out-of-domain neurological Spanish speech.

Results

• Baseline Performance: Among open-weight models, omniASR CTC 1B v2 achieved the lowest Word Error Rate (WER) on S-DiverSe, while the commercial Scribe v2 achieved the best overall performance. No single open-weight model dominated across all tested corpora (S-DiverSe, TORGO, NeuroVoz), suggesting architecture and training data interact differently with specific pathologies.
• Intelligibility vs. WER: An inverse relationship was observed between intelligibility and WER. At low intelligibility levels, open-weight models degraded significantly, whereas Scribe v2 maintained relative robustness.
• Adaptation Findings:
  • Post-processing: Heuristic text post-processing proved to be the most robust adaptation strategy for S-DiverSe. It consistently reduced WER without harming in-domain performance on TORGO, indicating that a substantial share of errors is correctable at the text level.
  • Fine-tuning: Fine-tuning consistently improved in-domain performance (on TORGO/NeuroVoz) but failed to generalize to the out-of-domain S-DiverSe data. In some cases (e.g., Whisper-large-v3 with FFT), fine-tuning caused catastrophic forgetting, leading to WERs exceeding 100% on S-DiverSe due to increased insertions and substitutions.
  • Data Composition: Adding Spanish Common Voice (read speech) to the training mix did not improve performance on S-DiverSe compared to using only pathological data, confirming that the bottleneck is domain mismatch (acoustic conditions) rather than language imbalance.

Significance and Claims
The paper claims that S-DiverSe establishes a necessary benchmark that exposes the limitations of current ASR systems in handling heterogeneous, in-the-wild pathological Spanish speech. The findings underscore a genuine domain gap: clean read-speech and controlled clinical corpora cannot substitute for diverse, real-world pathological data.

Crucially, the authors conclude that for out-of-domain neurological Spanish speech, heuristic text post-processing is more robust than fine-tuning. Parameter-updating methods fail to generalize beyond the training distribution, regardless of language composition. This highlights an urgent community need for dedicated data collection with broader coverage of stroke and female speakers to advance robust ASR for pathological Spanish speech. The dataset is intended for evaluation and development, not clinical inference.