Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning

1. Problem Statement

The paper addresses the significant gap in Optical Character Recognition (OCR) capabilities for the Tigrinya language, which uses the Ge'ez script (an abugida).

• The Challenge: While Transformer-based OCR models like TrOCR have achieved state-of-the-art results on Latin and CJK scripts, they fail completely when applied to Ge'ez.
• Specific Failure Modes:
  • Vocabulary Mismatch: Standard Byte-Pair Encoding (BPE) tokenizers (trained on English) do not recognize Ge'ez characters, mapping them to <unk> or fragmenting them into meaningless byte sequences.
  • Boundary Failures: Even after extending the vocabulary, the models exhibit a systematic failure where they consistently drop the first character of a word following a whitespace. This is caused by a conflict between the pre-trained tokenizer's space-marker conventions (e.g., _Word) and the new script tokens, which lack learned merge rules for space prefixes.
• Resource Scarcity: Tigrinya is a low-resource language in the digital domain, with no dedicated commercial OCR support and limited high-quality datasets compared to major languages.

2. Methodology

The authors propose a three-stage adaptation pipeline to transfer the TrOCR architecture to Tigrinya:

A. Dataset Utilization

• Source: The GLOCR dataset (news text-lines corpus), specifically a subset of 20,000 synthetic images generated from Haddas Ertra newspaper text.
• Split: 10,000 for training, 5,000 for validation, and 5,000 for testing.
• Characteristics: The dataset features uniform grayscale images with an average line length of ~15.8 characters.

B. Model Architecture & Tokenizer Extension

• Base Model: Microsoft's TrOCR (both base-handwritten and base-printed variants), which pairs a BEiT Vision Transformer encoder with a RoBERTa-based autoregressive decoder.
• Tokenizer Adaptation:
  • The original RoBERTa vocabulary (50,265 tokens) was extended by appending 230 unique Ge'ez characters (including base consonants, vowel variants, labialized forms, numerals, and punctuation).
  • New embeddings were initialized from a normal distribution $N(0, 0.02)$.
  • This treats each Ge'ez character as an atomic token, respecting the script's linguistic structure.

C. Word-Aware Loss Weighting (The Core Innovation)

To solve the "first-character drop" issue at word boundaries without retraining the tokenizer:

• Mechanism: The authors identified that standard cross-entropy loss treats all tokens equally. However, the pre-trained BPE tokenizer expects space-prefixed tokens for word starts, while the new Ge'ez tokens do not have these variants.
• Solution: A position-dependent weight is applied during training.
  • Tokens identified as "boundary tokens" (those containing the BPE space delimiter) are assigned a weight of 2.0.
  • All other tokens retain a weight of 1.0.
• Formula: $L = \sum_{i=1}^{N} w_i \cdot CE(y_i, \hat{y}_i)$, where $w_i \in \{1.0, 2.0\}$.
• Effect: This doubles the penalty for errors occurring at word onsets, forcing the optimizer to learn the transition from space tokens to Ge'ez characters more reliably.

D. Training Configuration

• Hardware: Single NVIDIA GeForce RTX 5060 Laptop GPU (8 GB VRAM).
• Duration: ~2 hours 40 minutes per variant.
• Hyperparameters: AdamW optimizer, learning rate $4 \times 10^{-5}$, 10 epochs, batch size 8 (via gradient accumulation), and mixed-precision (FP16).

3. Key Contributions

1. First TrOCR Adaptation for Ge'ez: The first successful application of a Transformer-based OCR model to the Tigrinya script, demonstrating that pre-trained visual and linguistic representations can transfer to a completely different writing system.
2. Word-Aware Loss Weighting: A novel, lightweight technique to resolve systematic boundary failures in cross-script BPE adaptation. The authors prove this is the critical bottleneck, not just vocabulary size.
3. Efficient Low-Resource Benchmark: Demonstrated that high-performance OCR can be achieved on consumer hardware (8 GB GPU) in under 3 hours using only 10,000 training samples, with all code and weights publicly released.

4. Results

The models were evaluated on a held-out test set of 5,000 synthetic images.

• Ablation Study: Without Word-Aware Loss Weighting, the model achieved a 20.06% CER (near-total failure). With the weighting, CER dropped by two orders of magnitude to 0.38%.
• Performance: The TrOCR-Printed variant achieved 0.22% CER and 97.20% accuracy, outperforming the handwritten variant and approaching the performance of a CRNN-CTC model trained from scratch (0.12% CER).
• Error Analysis: The remaining errors were primarily due to digit recognition (1.08%) and diacritic confusions (0.50%), confirming that the boundary issue was successfully resolved.

5. Significance and Impact

• Methodological Breakthrough: The paper proves that the "hidden bottleneck" in cross-script OCR is often the tokenizer-decoder interface, not just the visual encoder. The proposed Word-Aware Loss Weighting is a generalizable solution for any scenario where BPE conventions clash with new scripts.
• Accessibility: By training on a single consumer GPU in under 3 hours, the work democratizes high-quality OCR development for low-resource languages, removing the need for massive computational clusters.
• Open Science: The authors released the full pipeline, including the 230-character tokenizer, model weights (hosted on Hugging Face), and evaluation scripts, setting a new standard for reproducibility in African language NLP.
• Future Direction: This work establishes a template for adapting Transformer OCR to other non-Latin scripts (e.g., other African or Asian abugidas) where standard tokenizers fail to handle word boundaries correctly.