💬 NLP

English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training

This paper systematically demonstrates that incorporating multilingual data during LLM post-training significantly enhances performance across all languages, including English, and that even minimal non-English inclusion is superior to English-only approaches, particularly for low-resource languages.

Original authors: Mehak Dhaliwal, Shashwat Chaurasia, Yao Qin, Dezhi Hong, Thomas Butler

Published 2026-04-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mehak Dhaliwal, Shashwat Chaurasia, Yao Qin, Dezhi Hong, Thomas Butler

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant student who is already very smart because they read millions of books in English. This student is a "Large Language Model" (LLM). Now, you want to teach them specific new skills, like solving complex math problems or learning how to use a new set of tools (APIs).

Traditionally, teachers (developers) have only taught these students using English-only materials, even though the students will eventually need to help people who speak Spanish, Japanese, Swahili, and many other languages. The result? The student gets really good at English tasks but struggles when asked to do the same tasks in other languages.

This paper asks a simple question: What happens if we stop teaching in just English and start mixing in other languages during this "special training" phase?

Here is the breakdown of their findings, using some everyday analogies:

1. The "English-Only" Trap

Think of the student as a chef who only knows how to cook using English recipes. If you ask them to cook a dish for a French guest, they might try to translate the recipe in their head, but the result is often messy.

The Old Way: Teachers thought, "If we just teach the chef English really well, they can figure out the rest later."
The Reality: The paper shows this is a mistake. Teaching only in English is actually the worst way to prepare the student for a multilingual world. It leaves them weaker than they could be, even at their English tasks.

2. The "Gym" Analogy: Why Adding Languages Helps

The researchers put the student through a rigorous training camp (Post-Training) with 220 different scenarios. They tested what happens when they add more languages to the mix.

The Finding: Adding even one extra language (like Spanish) to the training diet made the student better at everything, including English.
The Analogy: Imagine the student is lifting weights. If they only lift with their right hand (English), they get strong on that side. But if they start lifting with their left hand (Spanish) too, their whole body (brain) becomes more balanced and stable. The "left hand" training actually makes the "right hand" stronger, too.
The Result: The student didn't get confused or slower; they got smarter. Even languages they didn't study directly (like Swahili) got better because the student learned how to "think" in a more flexible way.

3. The "Swiss Army Knife" vs. The "Specialized Tool"

The paper tested two types of tasks:

Math Reasoning: Like solving a logic puzzle.
API Calling: Like following a strict instruction manual to plug in a USB drive.

They found that for the "puzzle" tasks, adding more languages was like giving the student a Swiss Army Knife. The more languages they knew, the more tools they had in their mental toolkit to solve the problem, even if the problem was in a language they hadn't seen before.

The Surprise: For the "strict instruction" tasks (APIs), the student got slightly confused if they were too small (like a tiny 0.6B parameter model) and tried to learn too many languages at once. But for bigger, smarter models, adding languages was always a win.

4. The "Magic Transfer" Effect

One of the coolest findings is about Zero-Shot Transfer.

The Scenario: Imagine you train the student on English, Spanish, and French. Then, you ask them to do a task in German (a language they never saw during training).
The Result: If the student had a diverse training mix (many different languages), they could often guess the German answer almost as well as if you had explicitly taught them German.
The Caveat: This "magic" works great for languages that are related (like Spanish and French) or for languages with lots of data (High-Resource). But for very distant, rare languages (like Swahili or Telugu), the student still needs a little bit of direct help. You can't just guess the rules of a totally alien language without seeing it at least once.

5. The Bottom Line: "English is Not All You Need"

The title of the paper is a play on the famous phrase "English is all you need." The authors are saying: No, it's not.

For the Big Picture: If you want an AI that works well for everyone, you cannot just train it in English.
The Sweet Spot: You don't need to train it in every language in the world. But adding a diverse mix of languages (even just a few) acts like a "supercharger." It makes the AI smarter in English, better at math, and much more capable of helping people in other languages without needing to be explicitly taught every single one.

In short: Don't just teach your AI English. Give it a global education. It will surprise you by becoming better at everything, even the things you thought it already knew.

1. Problem Statement

Despite the global deployment of Large Language Models (LLMs), post-training pipelines (Supervised Fine-Tuning or SFT) remain predominantly English-centric. This creates significant performance disparities across languages. While prior work suggests that replacing English data with multilingual data can improve multilingual performance, it often leads to diminishing returns or degradation in English performance (the "curse of multilinguality").

Existing studies are fragmented, often focusing on narrow sets of languages, tasks, or model scales. Consequently, there is a lack of systematic understanding regarding:

The trade-offs between English and non-English performance.
How increasing language coverage affects different task types (e.g., reasoning vs. structured generation).
How these effects scale with model capacity.
Whether data augmentation (adding multilingual data to English) is superior to data substitution (replacing English data), and how language diversity specifically drives cross-lingual generalization.

2. Methodology

The authors conducted a large-scale, controlled empirical study involving 220 supervised fine-tuning runs.

Experimental Design

Models: Two open-weight model families were used:
- Qwen-3: 0.6B, 1.7B, and 8B parameters.
- Gemma-3: 1B and 4B parameters.
Tasks: Two distinct task types were evaluated:
1. Mathematical Reasoning: Using the mCoT-MATH dataset (Chain-of-Thought reasoning).
2. API Calling: Using the newly introduced mAPICall-Bank (structured generation/tool use).
Languages:
- 11 Typologically Diverse Languages: English (En), Spanish (Es), Japanese (Ja), Bengali (Bn), Swahili (Sw), French (Fr), Thai (Th), German (De), Russian (Ru), Chinese (Zh), and Telugu (Te).
- Roles: 5 "Core" languages (En, Es, Ja, Bn, Sw) were used for evaluation and training. Telugu (Te) was an unseen evaluation language. The remaining 5 languages were used only to expand training mixtures to test diversity effects.
Data Mixtures:
- The study constructed 22 distinct data mixtures ranging from 2 to 10 languages.
- Controlled Volume: Crucially, the number of examples per language was kept constant across mixtures. This isolates the effect of language diversity/coverage from the confounding factor of simply increasing total dataset size.
- Parallel Data: All training data was parallel translated to ensure semantic equivalence across languages.

Training & Evaluation

Models were trained for 6 epochs using AdamW.
Evaluation was performed on held-out test sets for the core languages and the unseen Telugu language.
Metrics included accuracy for Math Reasoning and exact match for API Calling arguments.

3. Key Contributions

Systematic Scaling Study: The first large-scale, controlled study varying language coverage (2 to 10 languages) across multiple model scales and task types using parallel data.
Introduction of mAPICall-Bank: A new multilingual dataset for API calling across 11 languages, addressing a gap in structured generation benchmarks.
Refutation of English-Only Suboptimality: Demonstrating that English-only post-training is consistently suboptimal, even for English performance.
Quantification of Diversity vs. Direct Exposure: Showing that high linguistic diversity can enable zero-shot cross-lingual transfer that matches or exceeds the performance of direct language inclusion for high-resource languages.

4. Key Results

A. Multilingual Scaling is Largely Beneficial

General Trend: Increasing the number of training languages generally improves or maintains performance across all tasks and model scales (≥1B parameters).
Low-Resource vs. High-Resource: Low-resource languages (e.g., Bengali, Swahili) benefit the most from added multilinguality. High-resource languages (e.g., English, Spanish) plateau rather than degrade, contradicting the "curse of multilinguality" in post-training contexts.
Exception: The smallest model (Qwen-3 0.6B) on the structured API calling task showed slight degradation after 4-5 languages, suggesting capacity constraints in sub-1B models for highly structured tasks.

B. Minimal Multilinguality Helps (The "Bilingual" Effect)

English is Not All You Need: Adding just one non-English language to English-only training yields consistent gains:
- Direct Exposure: Large gains for the added language (e.g., +9.27% for API calling).
- Cross-Lingual Transfer: Gains extend to languages not in the training set (+3.38% for API calling).
- English Performance: Surprisingly, English performance also improves (+0.88% to +3.4%), proving that English-only post-training is suboptimal.

C. High Diversity Enables Zero-Shot Transfer

Compensation Effect: At sufficient diversity (6–9 languages), zero-shot cross-lingual transfer (testing on a language not seen in training) can match or exceed the performance of direct exposure (training on that specific language).
High-Resource Languages: For high-resource languages, zero-shot performance in high-diversity settings is statistically indistinguishable from direct exposure.
Low-Resource Languages: Typologically distant, low-resource languages still benefit more from direct inclusion than from diversity alone, though diversity still provides significant gains over English-only baselines.

D. Regression Analysis

A pooled regression analysis confirmed that training-language coverage is positively associated with performance ( $\beta = 0.053$ ), even after controlling for whether the specific evaluation language was included in the training set. This confirms that diversity itself drives generalization, not just direct exposure.

5. Significance and Implications

Paradigm Shift: The paper challenges the industry standard of English-centric post-training. It argues that multilingual post-training is a necessity, not a luxury, even for improving English capabilities.
Efficiency: Practitioners can achieve strong cross-lingual generalization without needing to curate massive, specific datasets for every target language, provided they maintain high linguistic diversity in the training mix.
Model Scaling: The findings suggest that as model capacity increases (≥1B), the "negative interference" of adding languages diminishes, making multilingual scaling a safe and effective strategy.
Task Specificity: While structured tasks (API calling) show sensitivity to model size, reasoning tasks show robust benefits from multilinguality across scales.

6. Limitations

Language Scope: The study covers 11 languages, which is a fraction of global linguistic diversity.
Model Scale: The largest model tested is 8B; effects at trillion-parameter scales are unknown.
Data Origin: Datasets were created via translation of English data, which may introduce "translationese" artifacts not present in naturally occurring multilingual data.
Task Scope: Findings are specific to reasoning and tool-use; other domains (e.g., creative writing) may behave differently.