Automated Measurement of Geniohyoid Muscle Thickness During Speech Using Deep Learning and Ultrasound

This paper introduces SMMA, a fully automated deep learning framework that accurately measures geniohyoid muscle thickness during speech, enabling scalable analysis of speech motor control and objective assessment of related disorders by eliminating the need for time-consuming manual annotation.

The Gist

Imagine trying to understand how a car engine works, but you can't open the hood. Instead, you have to shine a flashlight through the metal to see the pistons moving inside. That's essentially what researchers do when they study how we speak. They use ultrasound (sound waves that create images, like a sonogram for a baby) to peek inside our throats and watch our tongue muscles move while we talk.

However, there's a problem: looking at these moving pictures is like trying to count the number of grains of sand on a beach while a storm is blowing. It's slow, tiring, and different people might count differently.

This paper introduces a new "robot assistant" called SMMA that does this counting automatically, instantly, and with expert-level precision.

Here is the breakdown of how it works, using simple analogies:

1. The Problem: The "Human Bottleneck"

The researchers were interested in a specific muscle called the Geniohyoid (GH). Think of this muscle as the hydraulic lift under your tongue. When you lower your jaw to say a sound like "Ah" (as in "father"), this muscle pulls hard. When you say a high sound like "Ee" (as in "see"), it relaxes.

For years, scientists had to manually draw lines around this muscle on ultrasound videos, frame by frame. It was like trying to trace a moving shadow with a pencil. It took forever, and if two people did it, they might get slightly different results. This made it impossible to study hundreds of people or analyze speech disorders quickly.

2. The Solution: The "Smart Camera" (Deep Learning)

The team built a system called SMMA (Skeleton-based Morphometric Muscle Analysis). You can think of this system as having two superpowers:

• The Smart Eye (Deep Learning): They taught a computer program (a type of AI) to look at the blurry ultrasound images and instantly recognize the shape of the GH muscle. It's like teaching a toddler to recognize a cat in a messy photo. Once trained, the AI can spot the muscle in a split second, drawing a perfect outline every time, no matter how tired it is.
• The Skeleton Ruler: Once the AI finds the muscle, it doesn't just guess the size. It turns the muscle shape into a "spine" (a skeleton line running down the middle). Then, it measures the distance from that spine to the edges, like measuring the width of a river at every point along its length. This gives a precise thickness measurement.

3. The Test: Did the Robot Pass the Exam?

To see if the robot was good, they compared it to human experts.

• The Result: The AI was almost as good as the best human sonographers. In fact, it was so consistent that it was actually more reliable than two humans comparing notes with each other.
• The Accuracy: When the robot measured the muscle, it was off by less than half a millimeter (about the thickness of a pencil lead) compared to the human experts. That is incredibly precise.

4. What Did They Discover? (The "Aha!" Moment)

Once they had this fast, automatic tool, they tested it on 11 people speaking Cantonese. They asked them to say three vowels: /a/ (like "father"), /i/ (like "see"), and /u/ (like "moon").

Here is what the robot found:

• The "Ah" Effect: When people said /a/, the muscle got significantly thicker (about 7.3 mm).
• The "Ee" Effect: When people said /i/, the muscle got thinner (about 6.0 mm).

Why? Imagine the muscle is a rubber band. When you say "Ah," you drop your jaw. The muscle has to contract (squeeze) hard to pull the jaw down, making it bulge and get thicker. When you say "Ee," you lift your jaw, so the muscle relaxes and gets thinner. The robot confirmed this biological theory with hard data.

They also found that men's muscles were naturally 5-8% thicker than women's, which is just due to body size differences, not because men speak differently.

5. Why Does This Matter?

Before this paper, studying these muscles was like trying to count stars with your naked eye. Now, we have a telescope.

• Speed: We can now study thousands of people instead of just a dozen.
• Health: This could help doctors diagnose speech problems (like dysarthria) or swallowing issues in elderly patients by objectively measuring how well these muscles are working.
• Rehabilitation: It could track if a patient's speech therapy is actually making their muscles stronger over time.

In a nutshell: The researchers built a "smart camera" that automatically measures the thickness of a tiny throat muscle while people talk. It's fast, accurate, and proves that our muscles work like hydraulic lifts, bulging up when we drop our jaws to make deep sounds. This opens the door to a new era of understanding how we speak and how to fix it when it goes wrong.

Technical Summary

Here is a detailed technical summary of the paper "Automated Measurement of Geniohyoid Muscle Thickness During Speech Using Deep Learning and Ultrasound."

1. Problem Statement

• Context: Ultrasound imaging is a valuable, non-invasive tool for studying speech production, particularly tongue dynamics. However, current research predominantly focuses on the tongue contour, leaving deeper muscles like the geniohyoid (GH) largely unexplored.
• The Gap: The GH muscle is critical for controlling tongue body position, hyoid movement, and jaw mechanics (specifically mandibular depression), which directly influence vowel production.
• Challenges:
  • Visualization: GH muscles are historically difficult to visualize clearly in ultrasound due to anatomical depth and probe placement variability.
  • Bottleneck: Manual measurement of muscle morphology is time-consuming, subjective, and suffers from high inter-rater variability. This prevents large-scale studies of speech motor control and clinical assessments of swallowing or speech disorders.
• Goal: To develop a fully automated, standardized framework to measure GH muscle thickness from B-mode ultrasound images during speech, eliminating the need for manual annotation.

2. Methodology: The SMMA Framework

The authors propose SMMA (Skeleton-based Morphometric Muscle Analysis), a two-component computational framework:

Component 1: Deep Learning Segmentation

• Input: B-mode ultrasound mid-sagittal images (224×224 px).
• Model Selection: Five state-of-the-art segmentation architectures were evaluated: Attention UNet, UNet, UltraUNet, SwinUNet, and DeepLab v3 (with ResNet-50 backbone).
• Training Protocol:
  • Trained for 50 epochs with early stopping.
  • Loss Function: A weighted combination of Dice Loss (0.8) and Focal Loss (0.2).
  • Augmentation: Ultrasound-specific online augmentations were applied.
• Output: A binary mask delineating the GH muscle region.
• Selection: UltraUNet was selected as the backbone due to its superior balance of accuracy (Dice ~0.90) and stability (low variance across runs).

Component 2: Skeleton-Based Thickness Extraction

• Pre-processing: The segmentation mask undergoes morphological operations (closing, opening, hole filling) and Gaussian smoothing to remove artifacts and ensure connectivity.
• Skeletonization: A one-pixel-wide medial axis (spine) is extracted from the smoothed mask.
• Thickness Calculation:
  • For each point on the skeleton, the local thickness is calculated as twice the perpendicular distance to the muscle boundaries.
  • Robustness: To minimize edge effects, the mean thickness ($T_{mean}$) is computed using only the interquartile range (25th–75th percentile) of the skeleton points.
• Output: Quantitative thickness measurements in millimeters, aligned temporally with phonetic events.

3. Key Contributions

1. First Automated GH Framework: Introduces the first fully automated pipeline for quantifying geniohyoid muscle dynamics during speech, moving beyond manual tongue contour tracking.
2. Validated Deep Learning Model: Demonstrates that UltraUNet achieves near-human-level segmentation accuracy, significantly reducing the time and subjectivity associated with manual annotation.
3. Robust Morphometric Algorithm: Develops a skeleton-based method that provides reliable thickness measurements even in the presence of minor segmentation noise, validated against expert ground truth.
4. Phonetic Insights: Provides the first automated, large-scale evidence of systematic GH thickness variations across different vowels and sexes.

4. Results

Validation Performance

• Segmentation (Component 1):
  • Human Inter-annotator Agreement: Dice coefficient ranged from 0.9001 to 0.9179.
  • Model Performance: UltraUNet achieved a Dice of 0.9037 and IoU of 0.8263, closely matching human agreement levels. It outperformed other models (e.g., UNet, DeepLab v3) in both accuracy and run-to-run stability.
• Thickness Measurement (Component 2):
  • Random Images: Mean Absolute Error (MAE) = 0.88 mm; Correlation ($r$) = 0.707.
  • Clinically Selected (High Quality) Images: MAE = 0.53 mm; Correlation ($r$) = 0.901.
  • Conclusion: The system achieves expert-validated accuracy, with 95% of measurements falling within ±1.46 mm of manual annotations for high-quality images.

Application: Vowel Production Analysis (N=11)

• Vowel Differences: Significant differences in GH thickness were observed across vowels ($p=0.019$).
  • /a:/ (Low Vowel): Greatest thickness (7.29 mm).
  • /i:/ (High Front Vowel): Smallest thickness (5.95 mm).
  • Effect Size: The difference between /a:/ and /i:/ showed a large effect size (Cohen's $d > 1.3$).
• Physiological Interpretation: Thicker GH during /a:/ aligns with the muscle's role in mandibular depression (lowering the jaw) required for low vowels. Thinner GH during /i:/ reflects reduced activation as the jaw is raised.
• Sex Differences: Males showed 5–8% greater absolute thickness than females, likely due to anatomical scaling rather than functional differences.

5. Significance and Future Impact

• Scalability: By removing the manual annotation bottleneck, SMMA enables large-scale studies of speech motor control that were previously infeasible.
• Clinical Utility: The framework offers an objective tool for assessing speech and swallowing disorders (e.g., dysarthria, sarcopenia) and monitoring rehabilitation progress.
• Reproducibility: The automated nature of the tool ensures high repeatability and eliminates inter-rater variability, standardizing muscle morphology research.
• Limitations & Future Work: Current limitations include a modest sample size (N=11), reliance on a single language (Cantonese), and sensitivity to image quality. Future work aims to validate across diverse linguistic contexts, pathological populations, and continuous speech scenarios.

Conclusion: The paper successfully presents SMMA as a validated, high-accuracy tool that bridges the gap between deep learning and articulatory phonetics, providing a new standard for analyzing deep muscle dynamics in speech production.