Improving Indigenous Language Machine Translation with Synthetic Data and Language-Specific Preprocessing

Technical Summary: Improving Indigenous Language Machine Translation with Synthetic Data and Language-Specific Preprocessing

Problem Statement
Indigenous languages of the Americas, such as Aymara, Guarani, and Quechua, face significant risks of endangerment due to limited digital presence and a scarcity of linguistic resources. A primary barrier to developing robust Neural Machine Translation (NMT) systems for these languages is the lack of large-scale parallel corpora. Conventional supervised MT approaches, which rely on abundant training data, often perform poorly in these low-resource settings. Furthermore, these languages frequently exhibit complex morphological structures (particularly agglutination) and orthographic variations that standard preprocessing pipelines fail to address effectively.

Methodology
The authors propose a data-centric approach to mitigate data scarcity by augmenting curated parallel datasets with synthetic sentence pairs. The methodology involves three core components:

1. Synthetic Data Generation:

   • The study utilizes the MultiScript30k dataset pipeline. Instead of relying solely on back-translation from monolingual corpora, the authors employ forward translation.
   • Spanish captions from the Multi30k dataset are translated into Aymara and Guarani using the high-capacity NLLB-200 (3.3B) multilingual translation model.
   • This approach treats Spanish as a high-resource pivot language, generating synthetic parallel data that preserves semantic content while increasing target-side linguistic diversity.

2. Language-Specific Preprocessing:

   • Guarani: The pipeline addresses orthographic variability by normalizing text via Unicode NFKC, lowercasing, and standardizing frequent digraphs (e.g., merging space-separated "c h" into "ch"). Crucially, it preserves specific diacritics and nasal vowels (e.g., ã, ˜e) and applies a length-ratio filter to remove misaligned pairs.
   • Quechua: The authors identify systematic intra-word spacing artifacts (e.g., "ch aypiqa" instead of "chaypiqa") that fragment tokens and skew evaluation metrics. A deterministic orthographic normalization procedure is applied to merge these split patterns without external linguistic resources.
   • Aymara: A conservative pipeline is used, focusing on Unicode normalization and removing spurious whitespace around apostrophes used for ejective consonants. However, the authors note that generic normalization may fragment the language's strongly agglutinative morphology.

3. Model Fine-tuning:

   • The multilingual mBART model (facebook/mbart-large-50) is fine-tuned on both curated-only and synthetically augmented datasets.
   • For Guarani, the tokenizer vocabulary is extended to include specific characters, nasal vowels, and common digraphs to better handle orthography.
   • Experiments are conducted on Aymara–Spanish, Guarani–Spanish, and Quechua–Spanish translation tasks.

Key Results
The study evaluates performance primarily using chrF++, the metric favored in recent AmericasNLP shared tasks for agglutinative languages, due to the known limitations of BLEU for these language families.

• Guarani–Spanish: Incorporating synthetic data improved the chrF++ score from 42.00 (curated-only) to 44.00 on the development set, a gain of +2.00 points.
• Quechua–Spanish: The application of deterministic orthographic normalization yielded substantial improvements, achieving a chrF++ score of 36.6. This outperformed the shared-task baseline and matched the best reported systems from the University of Sheffield.
• Aymara–Spanish: Synthetic data augmentation resulted in a modest increase in chrF++ (from 26.85 to 30.82). However, overall performance remained low, suggesting that generic preprocessing and data augmentation are insufficient for highly agglutinative languages without morphology-aware strategies.
• Comparison: The authors' systems, utilizing mBART and forward-translated synthetic data, achieved competitive results against the AmericasNLP 2023 baseline and top-performing submissions (e.g., University of Sheffield), which often utilized larger NLLB-200 models with back-translation.

Significance and Claims
The paper claims that forward-translated synthetic data serves as an effective and scalable strategy for improving NMT quality in data-scarce scenarios for indigenous languages. The study demonstrates that:

1. Synthetic data augmentation consistently improves translation quality (measured by chrF++) across different language pairs when paired with robust multilingual models.
2. Language-specific preprocessing, particularly orthographic normalization for Quechua and Guarani, is critical for reducing corpus artifacts and improving token-level alignment.
3. While data augmentation helps, it has limitations for highly agglutinative languages like Aymara, where generic subword tokenization may fragment morphemes, indicating a need for future morphology-aware approaches.

The authors position their work within the broader AmericasNLP initiative, contributing to efforts in language preservation and accessibility. They emphasize that while automatic metrics show improvement, the reliance on synthetic data and the lack of human evaluation for cultural nuance remain limitations. Future work is proposed to include human evaluation, explore multimodal translation using visual context (leveraging resources like MultiScript30k), and develop more targeted preprocessing for morphologically complex languages.