No One-Size-Fits-All: Building Systems For Translation to Bashkir, Kazakh, Kyrgyz, Tatar and Chuvash Using Synthetic And Original Data

Technical Summary: No One-Size-Fits-All: Building Systems For Translation to Bashkir, Kazakh, Kyrgyz, Tatar and Chuvash

Problem Statement
The paper addresses the challenge of machine translation (MT) for low-resource Turkic languages, specifically focusing on five language pairs: Russian-Bashkir, Russian-Kazakh, Russian-Kyrgyz, English-Tatar, and English-Chuvash. The authors note that high-quality parallel data is scarce for these languages, necessitating the investigation of multiple approaches to improve translation quality under data-scarce conditions.

Methodology
The authors employed a hybrid strategy combining data augmentation, parameter-efficient fine-tuning, and retrieval-augmented prompting.

• Data Construction and Augmentation:

  • Original Data: The study aggregated existing parallel corpora from various sources (e.g., NLLB Team, Plotnikov and Antonov, Zhang et al.) for the five language pairs.
  • Synthetic Data: To address data scarcity, the authors augmented the training sets using Yandex.Translate. They translated English phrases to Russian (where necessary) and then to the target Turkic languages. This process utilized "document translation" features on large chunks (50k–200k samples).
  • Filtering: Synthetic data was meticulously filtered to remove any phrases appearing in the test sets.
  • Pseudolabeling: The authors utilized data pseudolabeling, finding that multilingual training without it yielded inferior results in preliminary experiments.
  • Final Dataset: After augmentation, the training data size reached approximately 2.46 million samples per language pair. A specific subset, YaTURK-7lang, was released, containing data translated into six languages.

• Approach 1: Fine-tuning with LoRA (Kazakh and Bashkir)

  • Base Model: The authors selected facebook/nllb-200-distilled-600M as the base model.
  • Tokenization: Custom language pair prefixes (e.g., <prefix_rus_bash>) were added to the tokenizer to guide the model.
  • Training Strategy: Two modes were explored:
    1. Multi-task fine-tuning on all languages for one epoch, followed by LoRA adapter training for specific languages.
    2. Single-task fine-tuning.
  • LoRA Configuration: The authors used DORA (Weight-Decomposed Low-Rank Adaptation), an extension of LoRA. Key hyperparameters included $r=64$, $\alpha=64$, LoRA dropout of 0.2, and PiSSA weight initialization. The target modules included projection layers ($q, v, k, out$) and feed-forward layers ($fc1, fc2$).
  • Optimization: Training utilized the paged AdamW-8bit optimizer with a linear learning rate scheduler (start rate $5e^{-4}$) for LoRA and cosine scheduling for full fine-tuning.

• Approach 2: Retrieval-Augmented Prompting (Chuvash, Tatar, Kyrgyz)

  • Rationale: Fine-tuning NLLB on Chuvash performed poorly, likely because the base model was not pretrained on Chuvash. Similarly, zero-shot performance for English-Chuvash was weak.
  • Retrieval System: An ANNOY index was built using source-language phrases from the dataset. For new queries, the system retrieved the most similar source phrases and their translations to serve as in-context examples.
  • Embeddings: thenlper/gte-small was used for English-Chuvash, while sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 was used for other pairs.
  • LLM Prompting: Several models were tested via API (DeepSeek-R1, DeepSeek-N1, MiMoV2, Gemma3, DeepSeek-V3.2). The prompt instructed the model to translate a phrase based on retrieved similar examples, with strict constraints to return only the translation.
  • Configuration: For English-Chuvash, a large context window (TOP_N=7000) was used. For English-Tatar, zero-shot prompting with DeepSeek-V3.2 proved superior to retrieval-based methods.

• Ensembling (Stacking):

  • The authors attempted to select the best translation from multiple submissions using semantic similarity (LaBSE encoder) and perplexity-based filtration. However, these methods generally led to minor deteriorations in validation scores compared to the best single submission.

Key Results
The authors report chrF++ scores from the competition leaderboard:

• Kazakh: The LoRA-based approach achieved the highest score of 49.71.
• Bashkir: The LoRA-based approach achieved 46.94.
• Chuvash: Prompting DeepSeek-V3.2 with retrieved examples achieved 39.47, significantly outperforming the fine-tuned NLLB model (11.70) and zero-shot baselines.
• Tatar: A zero-shot or retrieval-based approach achieved 41.6. Notably, the zero-shot result from DeepSeek-V3.2 (43.66) was unbeatable by retrieval methods.
• Kyrgyz: The zero-shot approach reached 45.6, outperforming retrieval-based prompting which caused performance drops when context windows were expanded.

Contributions and Significance
The paper's primary contribution is the demonstration that a "one-size-fits-all" strategy is ineffective for low-resource Turkic languages; instead, the optimal approach depends on the specific resource availability and characteristics of the target language.

• Resource-Dependent Strategies: For relatively well-resourced languages (Bashkir, Kazakh), fine-tuning pretrained models (NLLB) on synthetic data using LoRA/DORA proved most effective.
• Scarcity Adaptation: For extremely data-scarce languages like Chuvash, where the base model lacked pretraining, retrieval-augmented prompting with large language models yielded significant quality improvements.
• Zero-Shot Limits: For languages with decent zero-shot performance (Kyrgyz, Tatar), complex retrieval mechanisms did not outperform simple zero-shot prompting with advanced models like DeepSeek-V3.2.
• Open Release: The authors released the YaTURK-7lang dataset and the model weights for Kazakh and Bashkir (including the multi-task fine-tuned model and LoRA adapters) to facilitate further research.

Conclusion
The study concludes that while fine-tuning on synthetic data is the most promising path for languages with some existing resources, prompting with retrieved examples is a viable alternative for languages with extreme data scarcity. The authors acknowledge that future research could explore fine-tuning models pretrained specifically on low-resource languages, but their current work establishes a baseline for handling these specific Turkic language pairs using the available tools and data.