💬 NLP

Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis

The paper introduces Habibi, an open-source unified-dialectal Arabic text-to-speech framework that leverages repurposed ASR corpora and curriculum learning to synthesize 12+ dialects with performance rivaling commercial models, while also releasing the first standardized multi-dialect Arabic TTS benchmark and all associated code and data.

Original authors: Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu, Xie Chen

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu, Xie Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Arabic language as a massive, vibrant family reunion. There are over 400 million people speaking roughly 30 different "dialects" (like cousins who grew up in different neighborhoods). While they all share the same family name (Arabic), the way they speak, the words they use, and even their accents can be as different as a New Yorker talking to a Texan.

For a long time, computers (specifically AI) were great at understanding the "formal" version of Arabic (Modern Standard Arabic), which is used in news and books. But when it came to the casual, everyday slang used by regular people, the computers were often confused, sounding like a robot trying to speak with a broken accent.

Enter "Habibi."

"Habibi" is a new, free (open-source) AI tool created by researchers to fix this. Think of it as a universal translator and voice actor that can speak any Arabic dialect fluently, without needing to be trained separately for each one.

Here is how they built it, explained with some simple analogies:

1. The Problem: The "Noisy Library"

The researchers wanted to teach the AI to speak these dialects, but there was a huge problem: bad data.

The Analogy: Imagine trying to teach a student to speak French by only giving them a library of books that are half-written in English, half in French, and full of typos and background noise from a construction site.
The Reality: Most existing Arabic audio data was made for listening (transcription), not for speaking. It was messy, full of errors, and lacked the "clean" voice samples needed to teach an AI how to sound natural.

2. The Solution: The "Data Chef"

The team acted like master chefs. They took these messy, "noisy" ingredients (the old audio files) and put them through a rigorous cleaning pipeline.

They filtered out the silence and the typos.
They used a "noise-canceling" tool to clean up the static.
They organized the ingredients into a massive, high-quality recipe book covering 12+ different regional dialects (from Saudi Arabia to Morocco to Egypt).

3. The Secret Sauce: "Curriculum Learning"

This is the smartest part of their strategy. Instead of throwing the AI into the deep end with all the difficult dialects at once, they taught it in stages.

Stage 1 (The Foundation): They taught the AI Modern Standard Arabic first. Think of this as teaching a child to read a textbook before letting them watch a chaotic street market. It gave the AI a solid understanding of the language's structure.
Stage 2 (The Real World): Once the AI mastered the textbook version, they slowly introduced the messy, real-world dialects.
The Result: Because the AI had a strong foundation, it could learn the "tricks" of the dialects much faster and more accurately than if it had tried to learn them all from scratch.

4. The "Magic Token" (Regional Identifiers)

To help the AI know which dialect to speak, they gave it a special "name tag."

The Analogy: Imagine you are an actor. If I just say "Read this line," you might sound generic. But if I hand you a card that says "Act like a person from Cairo," you instantly adjust your accent and slang.
The Tech: They added special codes (like EGY for Egypt or SAU for Saudi Arabia) to the text. This told the AI exactly which "flavor" of Arabic to use, making the voice sound authentic.

5. The Showdown: Habibi vs. The Giant

The researchers tested their new AI against ElevenLabs, a famous commercial company that makes some of the best AI voices in the world.

The Result: Habibi didn't just hold its own; it beat the commercial giant in many categories.
Why it matters: ElevenLabs is a paid, closed system. Habibi is free and open-source. This means anyone can download it, use it, and build upon it. It proved that you don't need a billion-dollar company to build world-class Arabic voice technology.

Why Should You Care?

It's a Game Changer: Before this, there was no free, high-quality tool that could speak all these Arabic dialects.
It's Fair: It treats all dialects with respect, not just the formal ones.
It's Open: The code, the data, and the models are all free for the world to use. The researchers are essentially saying, "Here is the blueprint; let's build the future of Arabic speech together."

In short, Habibi is the first open-source "Swiss Army Knife" for Arabic voices, proving that with smart training and clean data, we can make AI speak the way real people actually talk.

1. Problem Statement

The paper addresses the critical lack of open-source, unified Text-to-Speech (TTS) systems for the Arabic language. Despite Arabic being spoken by over 400 million people in approximately 30 distinct varieties, current TTS technology faces three major barriers:

Linguistic Complexity: There is a massive divergence in vocabulary and phonology between Modern Standard Arabic (MSA) and the various spoken dialects. MSA is rarely used in daily life, while dialects are fluid and often lack standardized orthography.
Data Scarcity & Quality: High-quality, clean datasets specifically for TTS are virtually non-existent. Existing resources are primarily designed for Automatic Speech Recognition (ASR), contain significant noise, lack diacritics (crucial for pronunciation), and suffer from imbalanced speaker distributions.
Evaluation Gap: There is no standardized benchmark for evaluating multi-dialect Arabic TTS, making it difficult to compare models or measure progress objectively.
Commercial vs. Open Source: While commercial models (e.g., ElevenLabs) exist, they are closed-source and often lack deep dialectal coverage or transparency.

2. Methodology

The authors propose Habibi, a unified-dialectal TTS framework built on the open-source F5-TTS architecture (a non-autoregressive model operating on mel-spectrograms). The methodology consists of four core components:

A. Multi-Dialect Data Curation Pipeline

The team aggregated and curated a massive dataset from various open-source ASR corpora (e.g., MASC, MGB-2/3/5, SADA, FLEURS) and public datasets, totaling ~1,857 hours of audio across 12+ regional identifiers (covering MSA and dialects like Saudi, Egyptian, Moroccan, Algerian, etc.).

Filtering: They implemented a rigorous cleaning process using Speaking Rate (Characters Per Second) thresholds to remove silent segments or corrupted files.
Denoising: A source separation model was applied to low-SNR datasets, with a mixed-training strategy (using both raw and denoised audio) to balance quality and robustness.
Expansion: The dataset was expanded from a "Basic" set (D1) to an "Expanded" set (D2) by incorporating additional internal and public data.

B. Linguistically-Informed Curriculum Learning

To handle the complexity of Arabic without diacritics, the authors employed a two-stage training strategy:

Stage 1 (MSA Foundation): The model is first fine-tuned on Modern Standard Arabic (MSA) data only. Since MSA shares phonological and grammatical patterns across all dialects, it serves as a robust "bridge" to transfer capabilities from the base model (pre-trained on Chinese/English) to Arabic.
Stage 2 (Dialectal Adaptation): The model is then fine-tuned on the diverse dialectal data. This allows the model to learn specific dialectal features via in-context learning without overfitting to the noise found in raw dialect datasets.

C. Dialect-Aware Supervised Fine-Tuning

The framework introduces regional identifier tokens (e.g., SAU, EGY, MAR) into the text sequence during training.

Mechanism: The text sequence is wrapped as <ID, text>.
Benefit: This explicitly guides the model to generate the correct dialectal pronunciation. Crucially, the model remains robust even if the identifier is omitted during inference, relying on its learned in-context capabilities.

D. Standardized Benchmark

The authors released the first standardized benchmark for multi-dialect Arabic TTS, comprising 11,000+ utterances across 7 dialect subsets (MSA, SAU, UAE, ALG, IRQ, EGY, MAR). The benchmark includes manually verified transcripts and strict separation from training data.

3. Key Contributions

First Open-Source Unified Dialectal TTS: Habibi is the first open-source framework capable of zero-shot synthesis across 12+ Arabic dialects, covering over 20 ISO 639-3 languoids.
Curriculum Learning Strategy: Demonstrates that an MSA-first training curriculum is essential for stabilizing convergence and achieving high-quality dialectal synthesis without diacritics.
Comprehensive Benchmark: Established the first standardized evaluation suite for Arabic TTS, enabling reproducible research.
Full Open-Source Release: All model checkpoints, training/inference code, and benchmark data are released at https://SWivid.github.io/Habibi/.

4. Experimental Results

The model was evaluated against ElevenLabs' Eleven v3 (alpha) (a leading commercial model) and specialized per-dialect models.

vs. Commercial Baseline (ElevenLabs):
- Speaker Similarity (SIM): Habibi significantly outperformed ElevenLabs in speaker similarity across all dialects (e.g., +1.4x improvement in Algerian dialect). ElevenLabs appeared to prioritize naturalness over voice fidelity.
- Intelligibility (WER): Habibi matched or surpassed ElevenLabs in Word Error Rates (WER) on most dialects.
- Naturalness: While ElevenLabs had a slight edge in naturalness (UTMOS) due to heavy fine-tuning on specific voices, Habibi's zero-shot approach was highly competitive.
Unified vs. Specialized Models:
- The Unified Model (Habibi) achieved performance comparable to, and in some cases (MSA, MAR) better than, models trained specifically on single dialects.
- It demonstrated that a single model can generalize across the entire Arabic dialectal spectrum effectively.
Ablation Studies:
- Curriculum Learning: Models trained directly on dialects without the MSA stage failed to converge to competitive performance. The MSA-first approach was proven critical.
- Data Quality vs. Quantity: High-quality, filtered data yielded better results than simply increasing the volume of noisy data.
- Regional Identifiers: Including dialect IDs during training improved WER significantly with negligible impact on speaker similarity or naturalness.

5. Significance

Democratization of Arabic AI: By providing a high-quality, open-source unified model, Habibi lowers the barrier for researchers and developers to build Arabic speech applications, moving beyond the "MSA-only" limitation.
Scientific Validation: The paper provides empirical evidence that curriculum learning (MSA $\to$ Dialect) is a viable strategy for low-resource, high-variability languages.
Foundation for Future Research: The release of the benchmark and data addresses the "resource gap" in Arabic speech synthesis, enabling future advancements in dialect modeling, evaluation, and robust deployment.
Competitiveness: It proves that open-source solutions can rival or exceed proprietary commercial systems in specific domains (like speaker fidelity and dialect coverage) when backed by rigorous data curation and training strategies.