💬 NLP

When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

This paper empirically demonstrates that conventional fine-tuning-based heuristics for selecting source languages do not reliably apply to cross-lingual In-Context Learning, revealing the need for alternative strategies to mitigate language confusion and optimize transfer across diverse tasks and models.

Original authors: Fred Philippy, Siwen Guo, Jacques Klein, Tegawendé F. Bissyandé

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fred Philippy, Siwen Guo, Jacques Klein, Tegawendé F. Bissyandé

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: A New Way to Teach AI

Imagine you have a brilliant student (a Large Language Model) who has read almost everything in the library, but mostly books written in English. You want this student to solve a math problem or write a story in a language they haven't practiced much, like Swahili or Bengali.

In the past, to teach the student a new language, you would give them a textbook and make them memorize it (this is called Fine-Tuning). In that old method, the best way to teach them was to use a language very similar to the one they were learning (e.g., using Spanish to teach Italian).

This paper asks a different question: What happens if we don't give the student a textbook? What if we just show them a few examples of how to do the task right in front of them, and ask them to figure it out on the spot? This is called In-Context Learning (ICL).

The researchers wanted to know: If we use this "show, don't tell" method, does English still make the best teacher? And does a language similar to the target language still make the best teacher?

The Surprise: English is Actually a Bad Teacher Here

The team tested this with six different AI models and seven different tasks (like answering questions, doing math, or classifying topics) across 18 different languages.

Here is what they found, which breaks all the old rules:

1. The "Native Speaker" Myth is False

The Old Belief: If you want to teach a student French, the best examples should come from a French speaker.
The Reality: In this new "show, don't tell" method, the target language (French) is actually the best teacher for itself only about 24% of the time. Sometimes, using a completely different language works much better.

2. English is Often the Worst Teacher

The Old Belief: Since the AI was trained mostly on English, English should be the strongest teacher for everything.
The Reality: English was the worst source language in about 16% of the experiments. It's like trying to teach someone how to drive a stick-shift car by having them watch a video of someone driving an automatic car; the rules are too different, and the student gets confused.

3. Similarity Doesn't Matter

The Old Belief: Languages that are "cousins" (like Spanish and Italian) should transfer knowledge easily.
The Reality: In this specific learning style, being linguistically similar doesn't help much. The AI doesn't seem to care if the languages sound alike.

The Real Secret: The "Quiet Classroom" Effect

So, who is the best teacher? The paper found a surprising pattern:

Low-resource, non-Latin script languages (like Thai, Telugu, or Bengali) are the best teachers.

The Analogy:
Imagine a classroom.

High-resource languages (like English or Spanish) are like a noisy, crowded party. When the AI sees examples in these languages, it gets distracted by all the patterns it already knows from its massive training data. It starts guessing based on what it usually hears, rather than looking closely at the specific examples you gave it.
Low-resource languages are like a quiet, empty room. Because the AI hasn't seen these languages as much, it can't rely on its "muscle memory" or old habits. It is forced to pay close attention to the specific examples you provided to figure out the pattern. It acts like a "regularizer"—it forces the AI to focus on the task rather than the language.

The "Confusion" Problem

The researchers also looked at what happens when the AI gets mixed up. Sometimes, you ask the AI to answer in Swahili, but it accidentally replies in English. This is called Language Confusion.

They found that:

If you use a "noisy" language (like English) as the teacher, the AI is more likely to get confused and reply in the wrong language.
If you use a "quiet" language (like a low-resource non-Latin script) as the teacher, the AI is less likely to get confused. It stays focused on the instructions.

The "Donor vs. Recipient" Twist

The paper discovered a strange relationship between languages:

Languages that are great at receiving help (performing well when they are the target) are often terrible at giving help (performing poorly when they are the source).
Conversely, languages that struggle to perform well on their own are often the best at helping other languages.

It's like a sports team: The team that wins the most championships (English) might actually be the worst at coaching a rookie team because they play too differently. A team that struggles a bit might have a coaching style that fits the rookie perfectly.

Summary

The paper concludes that when we use AI to learn from examples (In-Context Learning) instead of memorizing textbooks (Fine-Tuning), the rules change completely:

Don't assume English is the best teacher.
Don't assume similar languages are the best teachers.
Sometimes, the languages the AI knows the least about are the best at teaching it new tasks.

The authors warn that while this is a useful trick for now, we shouldn't stop teaching the AI low-resource languages. We still need to make sure the AI understands them well, not just use them as a tool to teach other languages.

Technical Summary: When English Isn't the Best Teacher

Problem Statement
While cross-lingual transfer in multilingual Natural Language Processing (NLP) has been extensively studied in supervised fine-tuning contexts, its dynamics in the emerging paradigm of In-Context Learning (ICL) remain under-explored. In fine-tuning, factors such as linguistic similarity, lexical overlap, and writing systems are known to dictate transfer quality. However, as the field shifts toward instruction-tuned Large Language Models (LLMs) that rely on few-shot ICL rather than parameter updates, it is unclear whether these established heuristics hold. Specifically, there is a lack of rigorous evaluation regarding how to select effective source languages for cross-lingual ICL, particularly for low-resource target languages. A prevailing assumption is that high-resource languages (like English) or linguistically similar languages serve as the best sources, but this has not been validated in the ICL regime.

Methodology
The authors conducted a broad empirical study to investigate source–target language relationships in cross-lingual ICL.

Scope: The study spanned seven multilingual benchmarks (covering tasks like paraphrase identification, commonsense reasoning, knowledge comprehension, and mathematical reasoning) and six LLMs (ranging from 1B to 4B parameters, including Llama 3.2, Gemma 3, Qwen3, and Phi-4-Mini).
Languages: Eighteen typologically diverse languages were selected to maximize benchmark overlap.
Experimental Design: The researchers evaluated all possible source-target language combinations. To ensure content consistency and isolate transfer dynamics, they utilized fully parallel datasets with aligned test and in-context examples. A 4-shot configuration was adopted based on prior findings that additional demonstrations yield diminishing returns in multilingual scenarios.
Metrics and Analysis:
- Transfer Performance: Measured via mean z-scores across tasks and models.
- Donor/Recipient Analysis: Defined donor scores (average performance a language provides to others) and recipient scores (average performance a language receives).
- Correlation Studies: Investigated correlations between transfer performance and (1) linguistic similarity (using URIEL and lang2vec for syntactic, genetic, phonological, and featural features), (2) cross-lingual alignment within model representations (measured via Centered Kernel Alignment on FLORES-200 data), and (3) script type and resource availability.
- Language Confusion: Extended the analysis to generative tasks using the Language Confusion Benchmark to measure the frequency of models failing to generate output in the requested language.

Key Contributions and Results
The study challenges several assumptions inherited from the fine-tuning era:

The Target Language is Not Always the Best Source: Contrary to common belief, the target language itself serves as the most effective source in only approximately 24% of cases.
English is Often a Poor Source: Despite its dominance in pretraining corpora, English emerged as the worst source language in about 16% of experiments.
Linguistic Similarity is Not a Predictor: Unlike in supervised fine-tuning, linguistic similarity (genetic, syntactic, etc.) shows no strong correlation with transfer effectiveness in ICL.
Internal Alignment Drives Transfer: Cross-lingual alignment within the model's internal representation space (learned during pre-training) is a significantly stronger indicator of transfer success than surface-level linguistic features.
The "Low-Resource, Non-Latin" Paradox: The most effective source languages are often low-resource, non-Latin script languages (e.g., Thai, Telugu, Bengali). Conversely, high-resource Latin-script languages (like English, Spanish, German) are among the least effective sources.
Donor-Recipient Asymmetry: There is a strong negative correlation (Pearson $r = -0.932$ ) between a language's capability as a donor and its capability as a recipient. Languages that perform well as targets often perform poorly as sources, and vice versa.
Language Confusion: In generative tasks, language confusion (outputting in the wrong language) is a distinct challenge not directly linked to task transfer performance. Non-Latin script languages tend to suffer more from confusion when used as targets but create less confusion when used as sources compared to Latin-script languages.

Significance and Claims
The paper argues that cross-lingual ICL relies on mechanisms distinct from those in fine-tuning. While fine-tuning leverages structural overlap and parameter updates guided by linguistic similarity, ICL appears to rely on the representational properties learned during pre-training. The authors suggest that high-resource languages may be "highly entropic," activating overlapping pretraining patterns that encourage reliance on language-specific heuristics rather than task structure. In contrast, low-resource or typologically atypical languages may act as implicit regularizers, forcing the model to focus on abstract input-output mappings.

The significance of these findings lies in rethinking source language selection strategies. The authors posit that the least supported languages in current LLMs—particularly low-resource non-Latin script languages—may serve as unexpectedly strong sources for cross-lingual ICL. This challenges the standard practice of defaulting to English or linguistically similar languages for prompting. The work underscores that strong performance as a target language does not imply effectiveness as a source language, suggesting that current evaluation practices focused solely on target-side performance may overlook critical aspects of source-language selection.

Limitations and Ethical Considerations
The authors note that their analysis is limited to 18 languages and models with $\le$ 4B parameters for computational tractability, though high stability across runs suggests these trends are robust. Ethically, the authors explicitly state that their findings should not be interpreted as a recommendation to reduce the presence of certain languages in pretraining; rather, the results reflect the current properties of models and training distributions. They caution that as data availability for low-resource languages improves, their role as effective sources may change.