⚡ electrical engineering

Voice Mapping of Text-to-Speech Systems: A Metric-Based Approach for Voice Quality Assessment

This study proposes a metric-based voice mapping framework to evaluate text-to-speech quality across six models, revealing that voice range is a primary capability indicator while specific metrics like cepstral peak prominence (CPPs) and spectrum balance effectively distinguish natural vocal effort from robotic artifacts.

Original authors: Huanchen Cai, Sten Ternström

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Huanchen Cai, Sten Ternström

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a music critic trying to judge how well different robots can sing. For years, the only way to do this was to ask a group of people to listen and give the robots a score from 1 to 5. But this is slow, expensive, and sometimes people just give high scores to be nice, or they can't agree on what "good" sounds like.

This paper proposes a new, scientific way to judge these singing robots (Text-to-Speech or TTS systems) without needing human ears. The authors call this method "Voice Mapping."

Here is a simple breakdown of what they did and what they found:

1. The Problem: The "Robot Voice" vs. The "Human Voice"

When a human speaks, they don't just get louder or quieter. When they shout, their voice changes texture; when they whisper, it changes again. It's a complex dance of effort and sound. Old computer voices often sounded like a flat, robotic drone because they couldn't handle these subtle changes. Newer AI models are getting better, but how do we measure exactly how human-like they are?

2. The Solution: The "Voice Map"

The authors created a visual tool called a Voice Map. Think of this like a weather map, but instead of temperature and rain, it maps Pitch (how high or low the voice is) and Loudness (how soft or loud it is).

The Grid: Imagine a grid where the horizontal axis is the note (pitch) and the vertical axis is the volume.
The Colors: Every time the computer speaks, it drops a dot on this map. The color of the dot tells us about the quality of the voice at that specific moment.
- Warm colors (Red/Orange): The voice sounds clear, rich, and natural.
- Cool colors (Blue): The voice sounds breathy, noisy, or robotic.

By looking at the whole map, you can see the "territory" the robot can cover. Can it sing high and loud? Can it whisper softly? Or is it stuck in a small, boring corner?

3. The Experiment: Testing the Robots

The researchers took 100 sentences from a famous recording (a woman reading a book) and asked six different AI models to read them. They then compared the AI's "Voice Map" to the original human's map.

They used three specific "rulers" to measure the voice quality:

Crest Factor (The "Peaks"): This measures how "spiky" the sound wave is. Too spiky, and it sounds harsh; too flat, and it sounds dull.
Spectrum Balance (The "Brightness"): This checks if the voice has enough high-frequency "sparkle" or if it sounds muffled like it's under a blanket.
CPPs (The "Harmony"): This measures how organized the sound waves are. A perfectly organized wave sounds clear; a messy wave sounds like static or a robot.

4. The Results: Who Sang Best?

The study compared models ranging from older ones (like Merlin from 2016) to the newest ones (like VITS from 2021).

The Old Guard (Merlin): Its map was small and scattered. It sounded very robotic. The "Harmony" score was too high (over 10 dB), which the authors say is a sign of a "perfectly stiff" robot voice rather than a human one.
The New Stars (VITS): This model created a map that looked almost identical to the human's. It covered the most territory (highs, lows, louds, and softs) and had the most natural sound quality. It was the closest to the real thing.
The Soft Whisperer (Glow-TTS): This model had a smaller map (it couldn't do as many loud or high notes), but when it whispered, it was the best. It captured the "soft" human voice better than anyone else, sounding very clear and natural in quiet moments.
The "Robotic" Warning: The study found a sweet spot for the "Harmony" score. If the score is between 7 and 8 dB, it sounds natural. If it goes above 10 dB, the voice starts to sound robotic and unnatural.

5. The "Engine" Check (Vocoders)

The researchers also tested the "engines" (called vocoders) that turn the AI's notes into actual sound. They found that one engine, UnivNet, produced a clearer, more natural sound than the other (Multiband-MelGAN), especially when the voice was loud.

The Bottom Line

This paper doesn't just say "this robot sounds good." It gives a visual blueprint of why it sounds good.

VITS is currently the champion for sounding like a real human across the board.
Glow-TTS is the champion for whispering softly.
Voice Mapping is the new tool that lets us see exactly where a robot voice fails (like being too robotic in the high notes) and where it succeeds, without needing a human to sit and listen for hours.

In short, they built a "voice fingerprint" scanner that tells us if a computer voice is truly alive or just a clever imitation.

1. Problem Statement

The evaluation of Text-to-Speech (TTS) systems faces significant challenges in balancing subjective and objective metrics:

Subjective Limitations: The "gold standard" Mean Opinion Score (MOS) is resource-intensive, time-consuming, and suffers from score instability and range-equalizing bias across different studies.
Objective Limitations: Traditional objective metrics (e.g., MCD, PESQ, ESTOI) operate at the frame or utterance level. They fail to capture within-speaker variability across different pitch ( $F_0$ ) and loudness (SPL) conditions. Consequently, they overlook temporal voice quality dynamics, such as how a model handles vocal effort, expressiveness, and physiological factors like glottal periodicity over long-form speech.
The Gap: Current TTS models often control pitch and loudness independently, failing to replicate the complex physiological coupling between vocal effort and voice quality found in natural human speech. There is a need for an evaluation framework that visualizes and quantifies a model's voice range and its ability to maintain natural voice quality across that range.

2. Methodology

The authors propose Voice Mapping, an objective, metric-based assessment framework adapted from clinical voice research (phonetograms) to evaluate synthetic voices.

Data & Setup:
- Dataset: 100 consecutive sentences from the LJSpeech dataset (ground truth) were used as the baseline.
- Models Evaluated: Six influential TTS architectures spanning a decade of development: Merlin (2016), Tacotron 2 (2018), Transformer TTS (2019), FastSpeech 2 (2020), Glow-TTS (2020), and VITS (2021).
- Vocoders: Specific comparisons were made between Multiband-MelGAN and UnivNet vocoders.
Voice Mapping Construction:
- The method maps acoustic features onto a 2D space defined by Fundamental Frequency ( $F_0$ ) (x-axis, in semitones) and Sound Pressure Level (SPL) (y-axis, in dB).
- The acoustic signal is processed cycle-by-cycle using FonaDyn software to extract metrics for each glottal cycle.
- The $(F_0, SPL)$ space is discretized into bins, and metrics are averaged within each cell to create a heat map.
Key Metrics:
1. Crest Factor (CF): Ratio of peak amplitude to RMS amplitude. Indicates "peakiness" and glottal flow interruptions.
2. Spectrum Balance (SB): Difference in acoustic power between frequencies >2 kHz and <1.5 kHz. Indicates spectral brightness/clarity.
3. Cepstral Peak Prominence (CPPs): Measures the degree of harmonic organization and periodicity. Higher values indicate clearer, more periodic voice; lower values indicate noise or instability.
Analysis: The study compares the voice maps of synthesized speech against the original speaker (LJSpeech) and analyzes difference maps to identify regions where quality degrades or improves.

3. Key Contributions

Novel Application of Voice Mapping: First application of clinical voice mapping techniques (originally used for voice disorders) to evaluate TTS systems, providing a visual and quantitative tool for comparing synthetic vs. natural voice dynamics.
Cycle-Synchronized Metric Extraction: Moves beyond frame-level averaging to analyze voice quality on a cycle-by-cycle basis, capturing the relationship between vocal effort (SPL) and quality.
Identification of Quality Thresholds: Established empirical thresholds for CPPs in natural female speech (7–8 dB) and identified that values exceeding 10 dB correlate with robotic-sounding output.
Comprehensive Model Comparison: Provided a systematic comparison of six major TTS architectures and two vocoders under controlled conditions, isolating the impact of architecture and vocoder on voice quality.

4. Key Results

Voice Range:
- VITS demonstrated the largest voice range (coverage of $F_0$ -SPL cells), closely matching the natural speaker.
- Glow-TTS exhibited a significantly limited voice range.
- Merlin (older statistical model) showed a scattered, limited range with poor tracking.
Metric Performance:
- CPPs (Periodicity):
  - Natural Range: 7–8 dB.
  - Merlin: ~12.24 dB (Excessively high, perceived as robotic).
  - Modern Models (Tacotron 2, Transformer, FastSpeech 2, VITS): Ranged between 6.68 and 7.96 dB, aligning well with natural speech.
  - FastSpeech 2: Lowest CPPs (6.68 dB), suggesting slightly less periodicity.
- Spectrum Balance (SB):
  - Glow-TTS: Showed superior SB (-14.89 dB) compared to the baseline, indicating better preservation of high-frequency energy in soft phonation (quiet speech), making it sound clearer in low-effort regions despite its limited range.
  - Merlin & Degraded Synthesis: Showed significant high-frequency energy loss (very negative SB), resulting in dull voice quality.
- Crest Factor: Glow-TTS had a slightly higher CF, indicating "peakier" signals, while most other models remained close to the baseline.
Vocoder Comparison:
- UnivNet outperformed Multiband-MelGAN in both Spectrum Balance and CPPs, particularly in louder speech regions, indicating better preservation of spectral characteristics and harmonic stability.

5. Significance and Conclusion

Evaluation Paradigm Shift: The study demonstrates that voice mapping is a powerful tool for visualizing the "expressiveness" and "vocal effort" capabilities of TTS models, which traditional metrics miss.
Model Insights:
- VITS is currently the strongest overall performer, balancing a wide voice range with natural periodicity.
- Glow-TTS excels in soft phonation quality but lacks range, suggesting a trade-off in its architecture.
- CPPs as a Diagnostic: The study validates CPPs as a critical indicator; values >10 dB serve as a warning sign for robotic artifacts, while 7–8 dB indicates natural quality.
Future Directions: The authors propose extending this framework to multi-speaker datasets, integrating additional metrics (jitter, shimmer, HNR), and potentially using voice maps for real-time model training feedback.

Conclusion: The paper successfully establishes voice mapping as a robust, objective framework for TTS evaluation, capable of capturing subtle nuances in voice quality and dynamic range that are essential for high-fidelity, expressive speech synthesis.