Technical Summary: ParsVoice

Problem Statement

Persian remains significantly underrepresented in open speech-text resources compared to high-resource languages like English. This scarcity hinders progress in multi-speaker text-to-speech (TTS) synthesis, speech-language modeling, and low-resource speech processing. Existing Persian TTS datasets are orders of magnitude smaller than their English counterparts, often proprietary, single-speaker, or reliant on explicit phonetic preprocessing. Furthermore, Persian presents unique linguistic challenges for TTS, including the systematic omission of short vowels and the Ezafe (a connective vowel) in written text, which complicates raw-text synthesis. While large-scale audiobook recordings exist, they lack the strict quality criteria required for TTS: clean audio free of background noise, precise sentence-level alignment, complete punctuation, and diverse speaker coverage.

Methodology

The authors introduce ParsVoice, a large-scale pipeline designed to transform raw, long-form audiobook recordings into a high-quality, multi-speaker speech-text corpus without relying on publicly available reference transcripts. The pipeline consists of nine interconnected stages:

1. Data Collection: Source material was selected from IranSeda, a platform hosting over 3,800 audiobooks recorded by professional narrators at 44.1 kHz.
2. Intelligent Audio Segmentation:
   • Acoustic Boundary Detection: WebRTC VAD identifies silence-based candidate boundaries.
   • Transcription: Segments are transcribed using the Google Web Speech backend.
   • Completeness Validation: A fine-tuned ParsBERT sentence-completion classifier distinguishes between complete and incomplete sentences. Incomplete segments undergo iterative boundary extension (in 0.1s increments) and re-transcription until they meet completeness criteria or reach a 5-second extension limit.
3. Boundary Optimization: To remove unwanted silence and artifacts at segment boundaries, a two-stage algorithm is employed:
   • Binary Search: Iteratively halves the trimming interval to find the maximum trim length that yields an identical transcription.
   • Linear Fine-Tuning: Applies 0.1-second increments to achieve precise alignment, reverting the final step if the transcription changes.
4. Quality Assessment: A multi-dimensional framework evaluates both audio and text:
   • Text Metrics: Character quality, length, repetition, linguistic complexity, and phonetic coverage (normalized to [0, 1]).
   • Audio Metrics: Signal-to-noise ratio, dynamic range, clipping, silence ratio, and background music detection. Segments scoring below 75/100 on audio or 0.5 on text are discarded.
5. Speaker Identification: A two-stage pipeline using ECAPA-TDNN embeddings clusters segments first within individual audiobooks and then globally across the corpus. Clustering consensus (Silhouette, gap statistic, DBSCAN) and ensemble methods (Agglomerative, Spectral, GMM) are used to assign global speaker IDs, filtering out low-confidence clusters.
6. Punctuation Restoration: Since ASR outputs lack punctuation, a ParsBERT-based model restores punctuation to all segments.
7. Final Filtering: The corpus is filtered to exclude segments with background music, low-confidence speaker assignments, or incomplete sentences, resulting in the final TTS-ready subset.

Key Contributions

• ParsVoice Corpus: The largest publicly available Persian speech-text corpus tailored for multi-speaker TTS. It contains 2,200 hours of speech, 1.36 million aligned segments, and 1,815 automatically identified speaker IDs. This is more than 25 times larger than the previously largest open Persian TTS dataset.
• Scalable Pipeline: A fully automated, reproducible pipeline for constructing high-quality TTS data from long-form audiobooks without ground-truth transcripts. Key innovations include the ParsBERT-based sentence-completion classifier for segmentation and the binary-search-based boundary optimization algorithm.
• Phoneme-Free TTS Validation: The authors demonstrate that high-quality multi-speaker TTS can be achieved using raw Persian text without explicit phoneme representations, a departure from traditional Persian TTS systems.

Results

To validate the corpus, the authors fine-tuned XTTS, a state-of-the-art zero-shot multilingual TTS model, on the ParsVoice dataset. The evaluation yielded the following results:

• Subjective Evaluation: The fine-tuned model achieved a Naturalness Mean Opinion Score (MOS) of 3.6/5 and a Speaker Similarity MOS (SMOS) of 4.0/5.
• Objective Evaluation: Intelligibility was assessed using three ASR systems (Google ASR, Persian-finetuned Whisper, and Qwen ASR). The Persian-finetuned Whisper model showed a Word Error Rate (WER) of 17.20% on synthesized speech versus 13.07% on real speech (FLEURS test set), indicating the synthesized speech is broadly intelligible.
• Speaker Similarity: ECAPA-TDNN cosine similarity between reference and synthesized speakers reached 80.0%, confirming effective zero-shot speaker adaptation.

Significance

The paper claims that ParsVoice addresses a critical resource gap in Persian language technology, enabling the development of multi-speaker TTS systems that were previously constrained by data scarcity. By providing a scalable pipeline and a massive, high-quality dataset, the work supports reproducible research in Persian speech synthesis and low-resource speech-language technologies. The successful validation of XTTS on raw Persian text suggests a pathway for future systems to bypass complex, language-specific phonetic preprocessing, lowering the barrier for deployment and further research. The authors note that while the corpus is formal and narrative in style (derived from audiobooks), it represents a foundational step toward more diverse Persian speech resources.