💻 computer science

A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition

This paper presents a systematic empirical study demonstrating that fine-tuning the Wav2Vec2-XLSR-53 pretrained Transformer model on a 870-hour Quranic dataset with Arabic text without diacritics significantly reduces Word Error Rates and improves training efficiency compared to baseline systems.

Original authors: Nabil Mosharraf Hossain, Riasat Islam, Unaizah Obaidellah

Published 2026-09-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nabil Mosharraf Hossain, Riasat Islam, Unaizah Obaidellah

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

For centuries, the recitation of the Quran has been a deeply personal and communal act, where the precise sound of every letter carries spiritual weight. In the Islamic tradition, the rules of pronunciation, known as Tajweed, are not merely stylistic choices but essential requirements for the correct delivery of the text. For millions of people who do not speak Arabic as their first language, mastering these nuances without a human teacher present is a significant challenge. While computers have become remarkably good at understanding human speech in general, teaching them to listen to Quranic recitation with the same care and accuracy has proven difficult. Standard speech software often struggles with the unique rhythms, the specific sounds of Classical Arabic, and the vast differences between a professional reciter in a studio and a learner practicing in a living room.

This is where the work of a team of researchers comes in. They set out to build a computer system that could listen to Quranic verses, whether spoken by a master or a student, and convert that audio into accurate text. Their goal was not just to create a transcription tool, but to develop a system that could help learners check their own pronunciation, find specific verses by voice, and memorize the text more effectively. To do this, they turned to a type of artificial intelligence known as a Transformer. Think of a Transformer as a machine that learns to understand language by reading vast amounts of text and listening to hours of audio, allowing it to grasp the context and relationships between sounds without being explicitly programmed with every rule. The researchers took these powerful, pre-trained models—which had already learned the general patterns of human speech—and carefully taught them to focus specifically on the sounds of the Quran.

The team began by gathering a massive collection of audio recordings, totaling more than 870 hours of content. This dataset was a mix of two very different types of voices. The first part came from professional studios, featuring high-quality recordings of renowned reciters reading every chapter of the Quran. The second part was far more chaotic, consisting of recordings submitted by everyday users through a mobile app. These user recordings contained background noise, varying accents, and occasional mistakes, mirroring the real-world environment where most learners would actually use such a tool. The researchers split this data, using the majority to train their models and a smaller portion to test how well the systems performed on unseen verses.

To find the best way to teach the computer, the researchers experimented with several different approaches. They tested various methods for converting sound waves into digital features, including systems that had been trained on multiple languages and others trained only on English. They also tried different ways of writing down the output. Some systems were asked to transcribe the text with all the small marks that indicate pronunciation (diacritics), while others were asked to write only the basic letters without those marks. They even tested writing the Arabic sounds using English letters. By running thousands of experiments, they discovered that the most effective approach was surprisingly simple. The system performed best when it used a specific multilingual model called Wav2Vec2, which had been pre-trained on 53 different languages, and when it was asked to output the Arabic text without the extra pronunciation marks.

The results showed a clear path forward. The best-performing model, trained on the professional recordings and then fine-tuned with the user data, made far fewer mistakes than previous systems. It reduced the error rate by a significant margin compared to a strong baseline system that had been used in earlier studies. Perhaps just as important was the efficiency of the process. While the older baseline system required 140 hours of computing time to reach its final level of accuracy, the new approach achieved superior results in just 30 hours. This speed and accuracy suggest that the system could be practical for real-world use, such as helping a student practice a verse and immediately receiving feedback on whether they pronounced the words correctly.

However, the researchers were careful to note where the system still faces challenges. While it excels at recognizing words, it sometimes struggles with the subtle differences between letters that sound very similar, a critical issue in Quranic recitation where a single sound change can alter the meaning. The system also found it harder to transcribe very short verses or to distinguish specific nasal sounds and vowel lengths that are central to the rules of Tajweed. These limitations highlight that while the computer can now hear the words with great clarity, it does not yet fully understand the intricate rules of pronunciation in the way a human teacher does.

Despite these hurdles, the study marks a significant step forward in the field of Quranic speech recognition. By proving that pre-trained models can be adapted to this specific domain with high accuracy and low computational cost, the researchers have opened the door for new tools that could support millions of learners. The work demonstrates that with the right combination of data and technology, machines can be taught to listen to the Quran with a level of care that respects both the complexity of the language and the devotion of the speaker. This foundation allows for future developments, such as systems that can not only transcribe text but also offer detailed guidance on pronunciation, helping to preserve and share the tradition of Quranic recitation in an increasingly digital world.

Technical Summary: A Comparative Study of Pretrained Transformer Models for Quranic ASR

Problem Statement

Quranic Automatic Speech Recognition (ASR) aims to convert recitations of the Quran into text to facilitate applications such as aided memorization and search engines. However, existing systems face significant hurdles, including high Word Error Rates (WER) on user-recited verses, limited coverage of the full Quranic corpus, and inadequate handling of the unique acoustic characteristics of Classical Arabic and Tajweed rules. Previous approaches often rely on narrow subsets of verses, specific reciters, or traditional architectures (e.g., GMM-HMM, MFCC-based DNNs) that struggle with the phonetic variations inherent in both professional and layman recitations. Furthermore, current literature lacks systematic comparisons of advanced self-supervised speech representations and optimal label formats specifically for this domain.

Methodology

The study employs a systematic empirical approach to fine-tune pretrained Transformer-based models for Quranic ASR. The methodology is structured into three phases:

1. Data Collection and Preprocessing

The research utilizes a comprehensive dataset comprising over 870 hours of audio, split into two distinct sources:

EveryAyah Dataset: 1,310 hours of professional recitations from 44 renowned male reciters covering all 114 Surahs, recorded in studio conditions.
Tarteel Dataset: 62 hours of user-generated recordings from a mobile application, representing diverse demographics, accents, and proficiency levels.
Preprocessing: Audio was standardized to 16kHz WAV format. Clips were filtered to a 1–30 second duration to manage GPU memory, with silence segments intentionally retained to preserve contextual cues. The data was split 80/20 for training and testing, ensuring speaker independence.

2. Feature Extraction and Model Architecture

The study compares several speech feature extraction methods and model architectures:

Feature Extractors: The research evaluates Wav2Vec2.0, HuBERT, and XLS-R (Cross-Lingual Speech Representations), which utilize self-supervised learning to capture context-aware speech features. These are compared against a baseline using MFCC features.
Architecture: The primary models are end-to-end Transformers (specifically facebook/wav2vec2-large-xlsr-53) featuring a frozen CNN encoder and a fine-tuned Transformer decoder. A Citrinet model (Nvidia) serves as the strong baseline.
Training Strategy: Models are trained using Connectionist Temporal Classification (CTC) loss with greedy decoding. Hyperparameters include a learning rate of $3 \times 10^{-5}$ , dropout of 0.1, and SpecAugment regularization.

3. Label Format Analysis

A critical component of the study is the comparative analysis of four distinct output label formats to determine their impact on transcription accuracy:

Arabic without Tashkeel (Diacritics removed).
Arabic with Tashkeel (Full diacritics).
Buckwalter Transliteration.
English Transliteration.

Key Contributions

The paper makes four primary contributions to the field of Quranic ASR:

Domain-Specific Adaptation: A thorough empirical study of fine-tuning pretrained Transformer architectures (Wav2Vec2, HuBERT, XLS-R) specifically for Quranic recitation, rather than proposing a novel architecture from scratch.
Feature Extraction Comparison: A systematic evaluation demonstrating that self-supervised representations (specifically Wav2Vec2 and XLS-R) outperform traditional MFCC features in this domain.
Label Format Optimization: Identification of the most effective output representation, finding that Arabic text without diacritics yields superior results compared to diacritized text or transliterations.
Training Strategy and Dataset Composition: Insights into the trade-offs between professional and user data, the impact of clip duration, and the efficiency gains of transfer learning over training from scratch.

Results

The experiments yielded the following key findings:

Model Performance: The Wav2Vec2-large-XLSR-53 model trained on the EveryAyah dataset with Arabic text (without Tashkeel) achieved the best performance, reducing the Word Error Rate (WER) to 0.08 (8%) and Character Error Rate (CER) to 0.015 (1.5%).
Baseline Comparison: The hybrid model, which combined the EveryAyah dataset with fine-tuning on the Tarteel dataset, achieved a WER of 0.11. This represents a 5.2% absolute improvement over the Citrinet baseline (WER 0.163).
Feature Extractors: Wav2Vec2 and XLS-R significantly outperformed HuBERT (which struggled with English-only pretraining) and MFCC-based baselines.
Label Formats: Arabic without Tashkeel was the optimal label format (WER 0.08 on EveryAyah; 0.12 in specific ablation tests), whereas transliterations and Buckwalter formats resulted in significantly higher error rates (WER ~0.38–0.40).
Dataset Impact: Models trained solely on professional data (EveryAyah) achieved WER 0.08. Fine-tuning on the noisier user data (Tarteel) resulted in WER 0.23. A hybrid approach (fine-tuning on Tarteel after EveryAyah) achieved a WER of 0.11, demonstrating generalizability to layman recitations.
Training Efficiency: The proposed transfer learning approach required approximately 30 hours of training time to reach optimal performance, compared to 140 hours for the baseline Citrinet model.
Clip Duration: Longer clip durations improved performance, with 30-second clips yielding the optimal WER (0.075) compared to 10-second clips (WER 0.70).

Significance and Claims

The paper claims that its findings establish a new benchmark for Quranic ASR by demonstrating that pretrained Transformer models, particularly Wav2Vec2, offer a transformative advantage over traditional approaches. The significance of this work lies in:

Scalability and Generalization: The model successfully handles diverse recitation styles, from professional Imams to layman users, addressing a gap in previous systems that were often limited to studio-quality data.
Efficiency: The study proves that self-supervised pretraining drastically reduces the computational resources and time required to achieve high accuracy, making robust Quranic ASR more accessible for development.
Practical Application: The system enables real-world applications such as voice-enabled Quranic search, automated memorization feedback, and accessibility tools for the hearing-impaired.
Limitations and Future Directions: The authors modestly acknowledge that the current model operates at the word level and lacks explicit phoneme-level resolution required for strict Tajweed rule enforcement (e.g., distinguishing Ghunnah or Qalqalah). They note the model's large size (1.2 GB) poses challenges for mobile deployment, suggesting future work should focus on model compression (distillation, quantization) and the development of phoneme-aware models for deeper Tajweed-sensitive analysis.