AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation

Technical Summary: AfriScience-MT

Problem Statement
The dominance of colonial languages (English, French, Portuguese) in African education and scientific communication excludes hundreds of millions of indigenous language speakers from accessing and producing scientific knowledge. A primary technical barrier to bridging this gap is the lack of established, standardized scientific terminology in African languages. Existing machine translation (MT) resources for African languages largely focus on general-purpose news, religious texts, or web-mined data, which often suffer from noisy alignments and lack the specific register and vocabulary required for STEM disciplines. Consequently, scientific concepts remain inaccessible, and accurate translation is impeded by the absence of domain-specific parallel corpora.

Methodology
The authors introduce AfriScience-MT, a high-quality parallel corpus designed to address these gaps through a decolonial approach to data creation.

• Data Construction: The dataset covers six African languages representing major regions and families: Hausa and Yorùbá (West Africa), Amharic and Luganda (East Africa), and Northern Sotho and isiZulu (Southern Africa). It spans 11 scientific domains: Agriculture, Biochemistry, Biology, Chemistry, Computer Science, Engineering, Geography, Health, Indigenous Knowledge, Sociology, and Statistics.
• Source Selection: 230 open-access English papers were selected from AfricArXiv, Zenodo, and Lens.org. Selection criteria prioritized papers introducing new terminology, supporting undergraduate education, and featuring African first authors to ensure diverse representation.
• Translation Pipeline: Recognizing that full-text translation of technical papers is prohibitively time-consuming for professional translators, the authors adopted a two-stage process:
  1. Expert Summarization: Domain-expert science communicators produced layperson summaries (250–350 words) for each paper, simplifying technical content while preserving key contributions.
  2. Professional Translation & Terminology Creation: Professional translators translated these summaries, abstracts, and associated glossaries into the target languages. Crucially, where established terms did not exist, translators and experts co-developed new scientific terminology.
• Dataset Statistics: The final corpus comprises 230 papers and 7,605 aligned English sentences, translated into the six target languages. It includes co-developed bilingual scientific glossaries ranging from ~100 to ~600 entries per language.
• Experimental Setup: The authors benchmarked translation quality across English↔Target directions using:
  • Models: Sequence-to-sequence models (M2M100, NLLB) and decoder-only LLMs (open-weight: Llama3, Gemma2, AfriqueLlama, AfriqueQwen, Tiny-Aya, TranslateGemma; closed-source: GPT-4o, GPT-5.4, Gemini-1.5-Flash, Gemini-3.1-Flash-Lite).
  • Settings: Zero-shot, in-context learning (ICL/few-shot), and supervised fine-tuning (including LoRA).
  • Metrics: SSA-COMET (an African-language-tuned semantic metric) and chrF (surface-level metric), supplemented by LLM judges for document-level evaluation.

Key Results

• Sentence-Level Performance:
  • Fine-tuning is decisive: A fine-tuned NLLB-1.3B (an encoder-decoder model) achieved an average sentence-level COMET score of 67.3, outperforming all fine-tuned open-weight LLMs.
  • Closed vs. Open: Closed-source models (GPT-5.4 and Gemini-3.1-Flash-Lite) led with average scores of 68.3 and 68.0, respectively. However, the fine-tuned NLLB-1.3B remained within 1 COMET point of these proprietary leaders.
  • Degeneration in Open LLMs: Several open LLMs (e.g., AfriqueLlama-8B, AfriqueQwen-8B, TranslateGemma-12B) showed high COMET scores in zero-shot/ICL settings but suffered from "fluent but off-target" generation, evidenced by a large gap between COMET and chrF scores.
• Document-Level Performance:
  • Closed models (GPT-5.4, Gemini-3.1-Flash-Lite) led with an average COMET of 48.3.
  • The best open-weight model, TranslateGemma-12B (using 1-shot ICL), reached 44.0, surpassing older proprietary models like GPT-4o and Gemini-1.5-Flash (which scored ~32–35).
• Domain Specificity:
  • Training on domain-specific data (AfriScience-MT) significantly outperformed training on general-domain news data (MAFAND-MT) by 5–10 COMET points.
  • Artificially balancing domain distributions (uniform sampling) hurt performance more than it helped, particularly for data-rich domains like Health. Natural distribution proved superior.
• Directional Asymmetry: Across all models and settings, Target→English translation consistently outperformed English→Target, reflecting the difficulty of generating morphologically complex African languages compared to English.

Significance and Claims
The paper claims that in-domain data is more decisive than model scale for African scientific translation. A modest, fine-tuned 1.3B parameter model can compete with much larger proprietary systems when trained on high-quality, domain-specific parallel data.

The authors position AfriScience-MT not merely as a dataset but as a step toward decolonizing science in Africa. By releasing the corpus, the co-developed bilingual glossaries, and trained models, they aim to:

1. Provide a benchmark for document-level scientific MT in African languages.
2. Demonstrate that high-quality scientific translation is achievable with modest, openly available resources, making the technology accessible to African institutions.
3. Address the critical gap of scientific terminology, enabling speakers of indigenous languages to access STEM knowledge without relying solely on colonial languages.

The work emphasizes that while translation is a pragmatic bridge, it must be accompanied by community engagement and ethical considerations regarding terminology standardization and the equitable ownership of scientific knowledge.