💬 NLP

Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs

The paper introduces Alexandria, a large-scale, community-driven dataset featuring 107K parallel English-Dialectal Arabic conversational turns across 13 countries and 11 domains with fine-grained city-level metadata and gender annotations, designed to improve machine translation and LLM performance for diverse Arabic dialects.

Original authors: Abdellah El Mekki, Samar M. Magdy, Houdaifa Atou, Ruwa AbuHweidi, Baraah Qawasmeh, Omer Nacar, Thikra Al-hibiri, Razan Saadie, Hamzah Alsayadi, Nadia Ghezaiel Hammouda, Alshima Alkhazimi, Aya Hamod, A

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abdellah El Mekki, Samar M. Magdy, Houdaifa Atou, Ruwa AbuHweidi, Baraah Qawasmeh, Omer Nacar, Thikra Al-hibiri, Razan Saadie, Hamzah Alsayadi, Nadia Ghezaiel Hammouda, Alshima Alkhazimi, Aya Hamod, Al-Yas Al-Ghafri, Wesam El-Sayed, Asila Al sharji, Mohamad Ballout, Anas Belfathi, Karim Ghaddar, Serry Sibaee, Alaa Aoun, Areej Asiri, Lina Abureesh, Ahlam Bashiti, Majdal Yousef, Abdulaziz Hafiz, Yehdih Mohamed, Emira Hamedtou, Brakehe Brahim, Rahaf Alhamouri, Youssef Nafea, Aya El Aatar, Walid Al-Dhabyani, Emhemed Hamed, Sara Shatnawi, Fakhraddin Alwajih, Khalid Elkhidir, Ashwag Alasmari, Abdurrahman Gerrio, Omar Alshahri, AbdelRahim A. Elmadany, Ismail Berrada, Amir Azad Adli Alkathiri, Fadi A Zaraket, Mustafa Jarrar, Yahya Mohamed El Hadj, Hassan Alhuzali, Muhammad Abdul-Mageed

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Arabic language as a massive, bustling city with 13 different neighborhoods (countries), each with its own unique slang, inside jokes, and local customs. For decades, the "official" language of the city—Modern Standard Arabic (MSA)—has been like the city's formal government broadcast. It's clear, polite, and used in news and schools.

But here's the problem: Nobody actually talks like that in their daily lives.

If you walk down the street in Cairo, Casablanca, or Riyadh, people are speaking their local dialects. They use slang, mix in French or English words, and talk differently depending on who they are talking to (a friend, a boss, or a neighbor).

The Problem: The "One-Size-Fits-None" Translator
Current AI translators are like tourists who only studied the government broadcast. When you ask them to translate a casual conversation from a local market, they often get it wrong. They might sound like a robot reading a dictionary, or they might miss the cultural nuance entirely. It's like trying to order a coffee in a local café using a phrasebook from 1950; you might be understood, but you won't fit in, and you might even offend someone.

The Solution: The "Alexandria" Project
The paper introduces Alexandria, a massive new project designed to teach AI how to speak real Arabic. Think of Alexandria not as a dictionary, but as a giant, community-run "Language Immersion Camp."

Here is how they built it, using simple analogies:

1. The Cast of Characters (The Community)

Instead of hiring a few experts in a lab, the researchers recruited 55 real people from 13 different Arab countries.

The Analogy: Imagine you want to learn the best local recipes. You don't ask a food critic; you ask the grandmothers and street chefs in every neighborhood. Alexandria did exactly this. They gathered locals from specific cities (not just countries) to ensure they captured the exact flavor of the dialect, down to the city level.

2. The Script (The Data)

They didn't just write random sentences. They created 34,000+ conversations covering 11 important areas of life, like farming, healthcare, business, and daily social chats.

The Analogy: Instead of giving the AI a list of vocabulary words ("apple," "car," "run"), they gave it scripts for a TV show. These scripts cover realistic scenarios: a farmer negotiating a water price, a doctor explaining a diagnosis, or friends planning a trip.
The Twist: Every conversation includes a "cast list" that specifies the gender of the speaker and the listener. In Arabic, the words you use change depending on whether you are talking to a man or a woman. Alexandria taught the AI this social dance.

3. The Quality Control (The Peer Review)

After the locals translated the scripts, another group of locals from the same country reviewed them.

The Analogy: Think of it like a local film festival. If a translator from a specific village in Morocco wrote a line, another person from that same village checked it. They asked: "Does this sound like something we would actually say? Or does it sound like a textbook?" If it sounded fake, they fixed it.

4. The Stress Test (The Evaluation)

Once the dataset was ready, they threw it at the world's smartest AI models (like Gemini, Qwen, and others) to see how they performed.

The Result: The AI models were like students who had only studied the textbook.
- Good news: They were okay at translating formal stuff.
- Bad news: When asked to speak in a local dialect, they often sounded stiff, used the wrong gender forms, or slipped back into formal Arabic.
- The Hardest Challenge: The models struggled the most with Maghrebi dialects (Morocco, Algeria, Tunisia, Mauritania), which are very different from the formal language, almost like a different language entirely compared to the "official" broadcast.

Why This Matters

Alexandria is a bridge.
Right now, if you are a farmer in rural Yemen or a doctor in a small clinic in Sudan, the technology available to you is built for the "official" language, not your reality. This project provides the training data needed to build AI that understands you.

It's about cultural inclusion. It ensures that when you talk to an AI, it doesn't just understand your words; it understands your context, your culture, and your voice.

In a nutshell:
Alexandria is a massive, community-built library of real-life conversations that teaches AI to stop sounding like a robot reading a manual and start sounding like a real person chatting with a neighbor. It's a crucial step toward making technology that truly works for everyone in the Arab world, not just the elite or the formal speakers.

1. Problem Statement

The paper addresses a critical gap in Natural Language Processing (NLP) regarding Arabic diglossia. While Modern Standard Arabic (MSA) is the formal written standard, daily communication across the Arab world occurs in diverse regional dialects.

The Limitation: Existing Machine Translation (MT) systems and Large Language Models (LLMs) are predominantly trained on MSA or English-centric resources. They generalize poorly to dialectal inputs, often failing to capture vernacular forms, cultural nuances, and specific lexical variations.
Data Scarcity & Quality: Previous dialectal datasets (e.g., PADIC, MADAR, FLORES+) suffer from limitations such as:
- Coarse Granularity: Grouping dialects by broad regions (e.g., "Levantine") rather than specific cities.
- Domain Narrowness: Focusing heavily on travel/tourism or controlled sentences rather than complex, multi-turn conversations.
- Lack of Context: Short, isolated sentences that fail to capture conversational flow, register shifts, or gender-conditioned language use.
- MSA Leakage: Benchmarks often contain entries that are too close to MSA, failing to test true dialectal capabilities.

2. Methodology

The authors introduce Alexandria, a large-scale, community-driven dataset created through a rigorous six-month process involving 55 participants from 13 Arab countries.

A. Dataset Creation Workflow

The creation process followed a three-phase pipeline (illustrated in Figure 2):

English Source Generation:
- Used Gemini-2.5 Pro to generate multi-turn English conversational scenarios.
- Scope: 13 countries $\times$ 11 high-impact domains (e.g., Healthcare, Agriculture, Legal, Logistics).
- Structure: 2–4 turn dialogues conditioned on specific personas, including speaker/addressee gender configurations.
- Quality Control: Automated checks for PII and length, followed by human review to ensure cultural plausibility and remove "translationese" or culturally mismatched scenarios.
Human Dialectal Translation:
- Native speakers from specific cities translated English turns into their local dialects.
- Guidelines: Translators were instructed to use colloquial phrasing, maintain semantic faithfulness, and adhere to specific gender/social registers. Code-switching (using Latin script for loanwords) was permitted where natural.
- AI Assistance: AI tools were allowed only as auxiliary aids for technical terminology, with rigorous human post-editing to ensure dialectal authenticity.
Peer Revision & Validation:
- A second native speaker from the same country reviewed each translation.
- Criteria: Dialectal authenticity, gender alignment, register appropriateness, and faithfulness.
- Outcome: 68.4% of turns remained unchanged; 30.6% required minor edits; 1% required major revisions.

B. Dataset Statistics

Scale: 107,000 total turns across 34,488 multi-turn conversations.
Coverage: 13 Arab countries (Egypt, Jordan, Lebanon, Libya, Mauritania, Morocco, Oman, Palestine, Saudi Arabia, Sudan, Syria, Tunisia, Yemen) and 11 domains.
Granularity: City-level metadata (e.g., distinguishing between Ramallah and Kobar dialects in Palestine).
Annotations: Includes speaker-addressee gender configurations (F $\to$ M, M $\to$ F, M $\to$ M, F $\to$ F) and code-switching rates.

3. Key Contributions

Unprecedented Granularity: Unlike prior resources, Alexandria associates data with city-of-origin metadata, enabling analysis of fine-grained sub-dialects rather than broad regional labels.
Multi-Turn Conversational Context: The dataset focuses on realistic, multi-turn dialogues rather than isolated sentences, allowing for the study of context-dependent translation.
Gender-Conditioned Variation: Explicit annotation of speaker-addressee gender pairs allows researchers to study how dialectal forms change based on social dynamics (e.g., male-to-female vs. female-to-female interactions).
Community-Driven Curation: The dataset is the result of a collaborative effort involving 55 researchers and translators, ensuring authentic representation of local varieties.
Comprehensive Benchmark: It serves as both a training corpus and a rigorous evaluation benchmark for 24 state-of-the-art LLMs (including Gemini, Qwen, Command, and Gemma).

4. Results and Evaluation

The authors evaluated 24 Arabic-capable LLMs (both open and closed-source) using the Alexandria public test set.

A. Automatic Evaluation (spBLEU & chrF++)

Directional Asymmetry: Models perform significantly better in Dialect $\to$ English than English $\to$ Dialect.
Dialect Performance:
- High Performance: Egyptian and Levantine dialects (Syria, Lebanon, Jordan) generally yield higher scores, likely due to better representation in pre-training data.
- Low Performance: Maghrebi dialects (Morocco, Mauritania, Tunisia) remain the most challenging, with Mauritanian yielding the lowest scores.
Model Rankings: Gemini-3-flash and Command-A variants consistently outperform other models across both directions.
Sub-Dialect Consistency: Within a country, the relative difficulty ranking of sub-dialects (e.g., urban vs. rural Palestinian varieties) is consistent across different model families.

B. Impact of Metadata and Reasoning

Metadata: Providing full context (country, domain, gender) generally helps, but the effect is model-dependent. For some models (e.g., aya-expanse), full metadata actually degraded performance compared to partial or no metadata.
Reasoning: Enabling "thinking" processes (Chain-of-Thought) generally did not help and often hurt translation performance, with the exception of gemini-3-flash, which saw a slight boost.

C. Human Evaluation

Semantic Adequacy vs. Dialectness: Models generally preserve meaning well (Semantic Adequacy > 3/5) but struggle with dialect authenticity (Dialectness $\approx$ 2/5).
The "MSA Leak": A major finding is that models often revert to MSA or a hybrid register when struggling with specific dialectal forms, resulting in grammatically correct but culturally unnatural output.
Gender Accuracy: Models adhere well to explicit gender constraints (>90% pass rate) when prompted.

D. Linguistic Factors

Lexical Overlap: There is a positive correlation between a dialect's lexical overlap with MSA and translation quality. Dialects structurally distant from MSA (e.g., Maghrebi) are harder to translate.
Code-Switching: The presence of code-switching (Latin script loanwords) generally degrades translation performance for most models, suggesting they struggle to handle mixed-script inputs effectively.

5. Significance

Cultural Inclusivity: Alexandria moves beyond "one-size-fits-all" Arabic NLP, acknowledging that "Arabic" is not a monolith but a collection of diverse, city-specific varieties.
Real-World Utility: By covering high-impact domains like healthcare, agriculture, and logistics, the dataset addresses the practical needs of millions of Arabic speakers who rely on dialects for daily life and professional services.
Future Research: The dataset provides a robust framework for developing culturally sensitive and dialect-aware LLMs. It highlights that current models, despite their scale, still lack the nuance to handle the full spectrum of Arabic diglossia, particularly in low-resource, high-variability regions like the Maghreb.
Open Resource: The dataset, creation prompts, guidelines, and evaluation code are publicly available, fostering reproducibility and further research in low-resource dialectal MT.

In conclusion, Alexandria represents a paradigm shift in Arabic NLP, moving from coarse, MSA-centric benchmarks to a fine-grained, community-validated resource that captures the true linguistic and cultural diversity of the Arab world.