🤖 machine learning

Huntington Disease Automatic Speech Recognition with Biomarker Supervision

This paper presents a systematic study on automatic speech recognition for Huntington's disease using a new high-fidelity clinical corpus, demonstrating that HD-specific adaptation and biomarker-based auxiliary supervision significantly reduce word error rates and reshape error patterns according to disease severity.

Original authors: Charles L. Wang, Cady Chen, Ziwei Gong, Julia Hirschberg

Published 2026-03-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Charles L. Wang, Cady Chen, Ziwei Gong, Julia Hirschberg

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to listen to a friend tell you a story, but they are speaking through a walkie-talkie that is constantly crackling, skipping, and changing volume. Now, imagine that friend has a condition called Huntington's Disease (HD). Their speech isn't just "muffled"; it's chaotic. Their voice might suddenly stop, speed up uncontrollably, or shake so much that words get distorted.

This paper is about teaching computers (specifically, Automatic Speech Recognition or ASR) to understand this chaotic speech, rather than just the clear, calm speech they are usually trained on.

Here is the story of their research, broken down into simple parts:

1. The Problem: The "One-Size-Fits-All" Failure

For a long time, scientists built speech-recognition systems (like Siri or Google Assistant) assuming all difficult speech was the same. They thought, "If we fix the system for a stutter, it will work for everyone with speech issues."

But Huntington's Disease is different. It's like trying to catch a butterfly with a net designed for a bumblebee. The "butterfly" (HD speech) has erratic, jerky movements that break the computer's expectations. The computer gets confused, deletes words, or makes up words that weren't there.

2. The New Tool: A Specialized Library

The researchers didn't just use old data. They got their hands on a brand-new, high-quality library of recordings from 94 people with Huntington's and 36 healthy people. This was the first time this specific "library" was used to train a computer to transcribe (write down) the speech, rather than just to diagnose the disease.

3. The Experiment: Testing Different "Ears"

The team tested three different types of computer "ears" (architectures) to see which one could handle the chaos best:

The Whisper Family: These are the famous, giant models everyone uses.
The CTC Model: An older, standard method.
The Parakeet-TDT: A newer, specialized model.

The Result: The giant "Whisper" models got very confused. They started hallucinating, inventing words that weren't spoken (like a student guessing answers on a test). The Parakeet model, however, was the champion. It didn't invent as many fake words and handled the chaos much better.

4. The Upgrade: Teaching the Computer "Clinical Clues"

Once they found the best model (Parakeet), they tried to teach it even better. They didn't just feed it audio; they gave it a "cheat sheet" based on biomarkers.

Think of biomarkers as medical vital signs for speech. The researchers taught the computer to look for three specific things while listening:

Prosody (The Rhythm): Is the person speaking too fast? Are there weird pauses?
Phonation (The Voice Shake): Is the voice trembling or unstable?
Articulation (The Mouth Shape): Are the vowels being stretched or distorted?

They added these clues as a secondary task. It's like asking a student to not only write down the story but also to keep a scorecard of the speaker's breathing and rhythm while they do it.

5. The Surprising Discovery: "Less is More" (But Only Sometimes)

Here is the twist:

For mild cases: When the disease wasn't too severe, giving the computer these "medical clues" helped it become more accurate. It reduced errors.
For severe cases: When the speech was very chaotic, the computer got too cautious. Because it was so focused on the medical clues, it started deleting words instead of guessing them. It was like a student who, afraid of getting a question wrong, just leaves the answer blank.

The Lesson: The "medical clues" didn't make the computer perfect everywhere. Instead, they changed how it made mistakes. It shifted from "making things up" (hallucinations) to "leaving things out" (deletions).

6. The Big Takeaway

This paper shows that:

Not all speech disorders are the same. You can't use the same tool for everyone.
The right model matters. Some computer brains are naturally better at handling chaos than others.
Medical knowledge helps, but with limits. Using doctor's knowledge (biomarkers) to train computers is a great idea, but it changes the computer's behavior. It makes it more careful, which is good for mild cases but can be risky for very severe cases.

In short: The researchers built a better translator for a very difficult language (Huntington's speech) and learned that sometimes, the best way to understand a chaotic speaker is to listen to their rhythm and voice stability, but you have to be careful not to make the computer too shy to speak up.

They have shared all their code and models online so other scientists can use this "translator" to help more people.

1. Problem Statement

Automatic Speech Recognition (ASR) for pathological speech remains a significant challenge, particularly for Huntington's Disease (HD). Unlike other motor-speech disorders (e.g., spastic or hypokinetic dysarthria), HD is characterized by hyperkinetic dysarthria, involving involuntary chorea of the vocal tract. This results in:

Irregular timing and unstable phonation.
Unpredictable phonatory arrests and respiratory bursts.
Loss of rhythmic regularity, which disrupts the temporal expectations of modern ASR models.

Existing benchmarks (e.g., UA-Speech, TORGO) and large-scale models (e.g., Whisper) often fail to generalize to HD because they are trained on different dysarthric patterns or lack the nuance for hyperkinetic speech. Furthermore, prior research has focused on diagnosing HD via speech rather than solving the transcription bottleneck (converting speech to text). There is also a lack of high-fidelity, open-source HD corpora for end-to-end ASR training.

2. Methodology

The authors propose a three-stage framework to systematically evaluate and adapt ASR for HD, utilizing a high-fidelity clinical corpus and biomarker-based auxiliary supervision.

A. Dataset and Biomarkers

Corpus: A dataset from Beth Israel Deaconess Medical Center (BIDMC) and Canary Speech containing 4.5 hours of audio from 130 individuals (94 HD-positive, 36 healthy controls). The HD cohort is stratified by severity (Pre-HD, Prodromal, Manifest) using UHDRS motor scores.
Biomarker Supervision: Instead of using high-dimensional feature sets, the authors distilled seven interpretable features across three motor-speech subsystems to serve as auxiliary supervision signals:
1. Prosody: Speaking rate proxy, pause-to-speech ratio, and fundamental frequency variance ( $\sigma(f_0)$ ).
2. Phonation: Local jitter, local shimmer, and Harmonics-to-Noise Ratio (HNR).
3. Articulation: Vowel Space Area (VSA) proxy derived from formant variance.
  These features are z-score normalized against healthy controls and discretized into low/medium/high bins for training.

B. Three-Stage Framework

Stage I: Cross-Architecture Evaluation:
- Evaluated multiple ASR families (Encoder-Decoder, Transducer/TDT, CTC) in a zero-shot setting to identify baseline performance and error regimes.
- Models tested: Whisper variants, Parakeet-TDT, and Meta Omnilingual CTC.
Stage II: HD-Specific Parameter-Efficient Adaptation:
- Selected the best zero-shot baseline (Parakeet-TDT 0.6B) and adapted it using encoder-side adapters (PEFT).
- The backbone remains frozen; only adapters are trained on the HD corpus.
Stage III: Biomarker-Informed Auxiliary Supervision:
- Augmented the Stage II model with a multi-task learning objective.
- The encoder predicts the standard transcription and the discretized biomarker family labels (Prosody, Phonation, or Articulation) via a linear head.
- Loss function: $L_{total} = L_{ASR} + \lambda L_{bio}$ (where $\lambda=0.1$ ).

3. Key Contributions

First Systematic HD-ASR Study: The first end-to-end ASR evaluation and adaptation study using a dedicated, high-fidelity HD clinical corpus.
Architecture-Specific Error Analysis: Demonstrated that HD speech induces distinct failure modes depending on the architecture, rather than a uniform increase in difficulty.
Parameter-Efficient Adaptation: Showed that adapter-based fine-tuning of Parakeet-TDT significantly improves performance for HD speech.
Biomarker-Guided Supervision: Proposed and evaluated a method to use clinically grounded biomarkers as auxiliary supervision, revealing that this reshapes error profiles rather than uniformly lowering Word Error Rate (WER).
Open Source: Released all code and models (ParakeetHD) to the community.

4. Key Results

A. Zero-Shot Baselines (Stage I)

Performance Gap: Parakeet-TDT (6.99% WER) vastly outperformed Whisper-large-v2 (18.44%) and the CTC baseline (30.46%).
Error Regimes:
- Whisper models were insertion-dominated (72–80% of errors were insertions/hallucinations).
- Parakeet showed a more balanced error distribution (41% substitution, 29% deletion, 28% insertion), indicating better preservation of lexical coverage.
Conclusion: Different architectures fail in fundamentally different ways under hyperkinetic disruption.

B. Adaptation and Biomarkers (Stage II & III)

Adaptation Gain: HD-specific adaptation of Parakeet reduced WER from 6.99% to 4.95%, improving all error types (substitutions, deletions, insertions) simultaneously.
Biomarker Impact:
- Adding biomarker supervision did not lower the overall WER below the plain adapted model (4.95%). The best biomarker variants hovered around 6.0–6.4% WER.
- Error Reshaping: Biomarkers shifted the error profile toward conservative decoding.
  - Phonation reduced substitution rates.
  - Articulation reduced insertion rates.
  - Cost: All biomarker variants incurred a deletion penalty (increased word omissions).
Severity Dependency:
- In mild/early stages (Control, Pre-HD, Prodromal), biomarker supervision offered slight improvements or neutral effects.
- In severe Manifest HD, biomarker supervision caused significant degradation (+3.06% to +3.59% WER increase). The models became overly conservative, prioritizing clinical structure over lexical coverage, leading to massive word omissions.

5. Significance and Implications

Architecture Selection: For hyperkinetic speech, Transducer/TDT architectures (like Parakeet) are superior to standard Encoder-Decoder models (like Whisper) because they are less prone to hallucination (insertions).
Clinical Nuance: The study proves that "better" ASR for pathology is not just about lower WER. Biomarker supervision can improve precision in mild cases but harms recall in severe cases by forcing the model to omit words rather than guess.
Future Directions: The work highlights the need for ASR systems that can dynamically adjust their "conservatism" based on disease severity. It also underscores the value of integrating clinical knowledge (biomarkers) into the training loop, even if it requires careful tuning to avoid over-regularization in severe cases.

Limitations: The corpus is relatively small (4.5 hours) and consists mostly of read speech; it lacks the most severe end of the HD spectrum and spontaneous conversational data.