AtlasOCR: Building the First Open-Source Darija OCR Model with Vision Language Models

1. Problem Statement

Darija (Moroccan Arabic) is a linguistically rich dialect widely used in social media, informal documents, and handwritten materials. However, it suffers from a critical lack of specialized Optical Character Recognition (OCR) tools.

• The Gap: Existing OCR systems focus primarily on Modern Standard Arabic (MSA) or English, failing to handle Darija's unique linguistic nuances, diverse fonts, and informal writing styles.
• Consequences: This absence hinders digital preservation of historical manuscripts, limits large-scale social media analysis, impedes accessibility for screen readers, and restricts linguistic research.
• Challenge: Developing high-performance OCR for Darija is difficult due to data scarcity and the high cost of annotating large datasets for low-resource languages.

2. Methodology

The authors propose AtlasOCR, a 3-billion-parameter Vision Language Model (VLM) specifically fine-tuned for Darija. The methodology consists of four core pillars:

A. Data Curation (Hybrid Approach)

To overcome data scarcity, the team created a large-scale dataset (30,092 samples, ~10.7M words) using a hybrid strategy:

• Synthetic Data (86%): Generated using OCRSmith, an open-source toolkit developed by the authors. This tool simulates realistic conditions (various fonts, layouts, backgrounds, and distortions) to produce thousands of labeled images with bounding boxes.
• Real-World Data (14%): Curated from diverse sources to ensure authenticity:
  • Scanned Literature: Historical texts (e.g., works by Mohammed El-Madlaoui El-Mounabhi).
  • Social Media: Educational posters from LinkedIn.
  • Educational Materials: Driving license exam preparations.
  • Recipe Collections: Moroccan cookbooks for domain-specific vocabulary.
  • Annotation: Real-world data was pseudo-labeled using Gemini 2.0 Flash and manually verified.

B. Base Model Selection

The team benchmarked several open-source VLMs on 55 manually curated Darija images.

• Winner: Qwen2.5-VL 3B was selected as the base model due to its superior performance across diverse Darija text domains compared to Qwen2-VL 2B, Qari OCR 2B, and ArabicNougat.

C. Parameter-Efficient Fine-Tuning (PEFT)

To train a 3B model on consumer-grade GPUs, the authors utilized a combination of:

• QLoRA (Quantized Low-Rank Adaptation): Quantized the model to 4-bit precision, reducing memory requirements by up to 80% while maintaining performance.
• Unsloth: Optimized GPU kernels and memory management to accelerate fine-tuning by 5× and reduce memory usage by 60%.

D. Ablation Studies & Optimization

The team conducted extensive ablation studies to optimize hyperparameters:

• LoRA Configuration: The optimal configuration was found to be Rank (r) = 128, Scaling Factor (α) = 128, and Dropout = 0.05.
• Quantization: 4-bit quantization was validated as sufficient, showing negligible performance loss compared to 16-bit.
• Vision Layers: Fine-tuning the vision encoder layers (rather than freezing them) significantly improved performance (Eval Loss dropped from 0.3173 to 0.2155), confirming the need to adapt visual features to Darija's specific characteristics.
• RSLoRA: Rank-Stabilized LoRA was tested but found to degrade performance in this specific setting.

3. Evaluation Framework

• New Benchmark: The authors created AtlasOCRBench, a publicly available benchmark containing 251 samples (55 from scanned literature + 196 synthetic/controlled samples).
• Existing Benchmark: Evaluation was also performed on KITAB-Bench (8,800+ samples) for standard Arabic.
• Metrics:
  • Character Error Rate (CER): The primary metric, chosen because Darija lacks standardized orthography and spelling variations are common.
  • Word Error Rate (WER): Used as a secondary metric but noted as potentially misleading for Darija.
• Preprocessing: Included text normalization (removing diacritics/harakat) and standardizing whitespace to ensure fair comparison.

4. Key Results

• State-of-the-Art (SOTA) on Darija: AtlasOCR achieved the lowest CER on AtlasOCRBench, significantly outperforming all other open-source alternatives.
• Competitive on Standard Arabic: On KITAB-Bench, AtlasOCR (3B parameters) demonstrated competitive performance against much larger models, including Gemma3 (12B) and Qwen2.5-VL (7B).
• Cross-Lingual Generalization: Despite being trained primarily on Darija, the model showed strong generalization capabilities for standard Arabic tasks.
• Efficiency: The model proves that high-performance OCR can be achieved with a 3B parameter model using efficient fine-tuning, challenging the notion that larger models are strictly necessary for complex OCR tasks.

5. Key Contributions

1. AtlasOCR: The first open-source OCR model specifically designed for Darija.
2. OCRSmith: An open-source toolkit for synthetic data generation tailored for low-resource languages.
3. AtlasOCRBench: A new, comprehensive benchmark for evaluating Darija OCR.
4. Methodological Blueprint: A detailed guide on using QLoRA and Unsloth to fine-tune VLMs for low-resource dialects, including specific hyperparameter optimizations.
5. Data Strategy: Validation of a hybrid synthetic-real data approach as a viable solution for low-resource language development.

6. Significance and Future Directions

Significance:
This work bridges a critical gap in North African digital infrastructure. By providing an open-source, high-performance tool, it enables digital preservation of Moroccan heritage, facilitates large-scale social media analysis, and improves accessibility for Darija speakers. It demonstrates that resource-efficient techniques can rival massive models for specific dialectal tasks.

Limitations:

• Struggles with Arabic diacritics (harakat) when present.
• Performance degrades on highly complex or artistic document layouts.
• Potential bias toward specific training domains (e.g., recipes, driving exams).

Future Work:

• Expanding the dataset to include more handwritten text and diacritized content.
• Developing sub-3B parameter variants for mobile/edge deployment.
• Enhancing layout understanding for complex documents.
• Adapting the methodology for other North African Arabic dialects.