Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast India

1. Problem Statement

Kokborok (ISO 639-3: trp) is an indigenous Tibeto-Burman language spoken by approximately 1.5 million people in Tripura, India. Despite having official language status, it is severely under-resourced in the Natural Language Processing (NLP) community.

• Data Scarcity: Prior Machine Translation (MT) efforts were limited to small, domain-restricted corpora (primarily Bible translations), resulting in poor performance.
• Baseline Performance: The best previous system (WMT 2025) achieved BLEU scores of only 6.99 (English→Kokborok) and 2.99 (Kokborok→English).
• Gap: There is a critical need for a high-quality Neural Machine Translation (NMT) system that can handle diverse domains and overcome the extreme data scarcity typical of low-resource languages.

2. Methodology

The authors developed KokborokMT, a fine-tuned NMT system based on the NLLB-200-distilled-600M model. The methodology involves three core components:

A. Model Architecture & Tokenization

• Base Model: Facebook's NLLB-200 (600M parameters), a multilingual transformer covering 200 languages.
• Token Adaptation: Since Kokborok was not natively supported, the authors introduced a new special token, trp_Latn, into the tokenizer vocabulary. This allows the model to condition generation on Kokborok while leveraging representations from typologically related Tibeto-Burman languages (e.g., Burmese, Tibetan) already present in NLLB.
• Training Strategy: The model was trained on concatenated datasets for both translation directions (en↔trp) to encourage shared representations.

B. Data Construction (36,052 Sentence Pairs)

The training corpus is a hybrid of three sources:

1. Professional Translations (SMOL Dataset): 9,284 sentences covering diverse domains (health, education, culture, tech). This is the highest-quality component.
2. Domain-Specific Data (WMT Bible): 1,769 sentences derived from Bible translations, used to maintain compatibility with previous benchmarks.
3. Synthetic Back-Translation: 24,999 pairs generated using Google Gemini Flash.
   • Process: English sentences from the Tatoeba project were translated into Kokborok using a specific system prompt.
   • Cost: Approximately INR 600 (USD $7).
   • Quality Check: The authors investigated LaBSE-based quality filtering but found it unreliable for Kokborok because the language was absent from LaBSE's training data, leading to artificially low similarity scores even for high-quality translations. Consequently, all synthetic data was retained.

C. Experimental Setup

Three systems were compared:

• Zero-Shot NLLB: Base model with the new token but no fine-tuning.
• System 1 (No BT): Fine-tuned on Professional + Bible data only (~11k pairs).
• System 2 (Full): Fine-tuned on Professional + Bible + Synthetic data (~36k pairs).

3. Key Contributions

1. KokborokMT System: The first high-quality NMT system for Kokborok, significantly outperforming all prior baselines.
2. Novel Token Integration: Successful integration of the trp_Latn token into the NLLB framework for a previously unsupported language.
3. Synthetic Data Augmentation: Demonstration that LLM-generated back-translation (via Gemini Flash) provides consistent, significant improvements for extremely low-resource languages at minimal cost.
4. Negative Finding on Quality Filtering: The paper reports that LaBSE-based quality filtering is ineffective for languages absent from the embedding model's training data, serving as a cautionary note for the community.
5. Resource Release: Public release of the model, data, and code under CC-BY-4.0.

4. Results

Automatic Evaluation

The System 2 (Full) achieved state-of-the-art results, representing a massive leap over previous work:

• English → Kokborok (en→trp): BLEU 17.30 (vs. prior 6.99).
• Kokborok → English (trp→en): BLEU 38.56 (vs. prior 2.99).
• Other Metrics: Significant improvements were also observed in chrF, ROUGE-L, METEOR, and COMET scores across both the general-domain (SMOL) and Bible-domain (WMT) test sets.

Human Evaluation

Three annotators (a linguistic expert, a native speaker, and a native researcher) evaluated 50 translations:

• Adequacy: 3.74/5
• Fluency: 3.70/5
• Agreement: Substantial agreement (Cohen's κ = 0.67) between trained evaluators.
• Conclusion: The system preserves meaning effectively and produces largely natural output, though fluency still has room for improvement.

Asymmetry

The system performed significantly better in the Kokborok→English direction (BLEU 38.56) compared to English→Kokborok (BLEU 15.25). This is attributed to NLLB's strong native English generation capabilities versus the difficulty of generating a low-resource language with limited representation in the base model.

5. Significance

• Advancing Low-Resource NLP: This work proves that combining small amounts of high-quality professional data with cost-effective LLM-generated synthetic data can "bootstrap" effective MT for languages with almost no existing parallel corpora.
• Community Impact: By releasing the model and data, the authors provide a foundational resource for further research on Kokborok and other Tibeto-Burman languages in Northeast India.
• Methodological Insight: The findings regarding the failure of LaBSE filtering for unseen languages offer a critical lesson for researchers attempting to apply standard quality control pipelines to truly low-resource languages.
• Practical Utility: With human evaluation scores nearing 4/5, the system is deemed practically useful for general communication, bridging a significant digital divide for 1.5 million speakers.