💬 NLP

Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?

This paper identifies that multilingual reasoning gaps in reasoning language models primarily stem from failures in translating inputs into the model's dominant reasoning language, and proposes a "Selective Translation" strategy that detects these failures to translate only a small fraction of inputs, thereby substantially bridging the performance gap while maintaining efficiency.

Original authors: Deokhyung Kang, Seonjeong Hwang, Daehui Kim, Hyounghun Kim, Gary Geunbae Lee

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Deokhyung Kang, Seonjeong Hwang, Daehui Kim, Hyounghun Kim, Gary Geunbae Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Language Barrier" in Smart AI

Imagine you have a super-smart genius named Reasoning Robot. This robot is incredible at solving complex math puzzles and logic problems, but there's a catch: it only really "thinks" in English.

If you ask it a question in English, it solves it perfectly. But if you ask the same question in Swahili, Telugu, or Thai, it often gets the answer wrong. This is called the Multilingual Reasoning Gap.

The researchers wanted to know: Why does this happen? Is the robot bad at math when speaking other languages, or is it just bad at understanding the question?

The Investigation: Breaking Down the Process

The team realized that when the robot gets a question in a foreign language, it goes through three steps:

Understanding: It tries to translate the question into its "thinking language" (English).
Reasoning: It solves the problem using English thoughts.
Generating: It writes the final answer back in the original language (e.g., Swahili).

They wanted to find out which step was breaking.

The Analogy: Think of the robot as a translator who is also a mathematician.

If the translator messes up the translation, the mathematician gets the wrong problem.
If the translator is perfect, but the mathematician is bad at math, the answer is still wrong.
If the math is perfect, but the translator can't write the final sentence in the local language, the answer is still wrong.

The Discovery: It's All About the Translation

The researchers tested the robot by "helping" it at each stage.

Helping with Math: They forced the robot to use English for the math part. (No change in performance).
Helping with the Final Answer: They forced the robot to write the answer in the local language. (No change in performance).
Helping with Understanding: They gave the robot a perfect English translation of the question before it started thinking.

The Result: When they fixed the Understanding step, the robot's performance skyrocketed, especially for low-resource languages like Swahili.

The Metaphor: It turns out the robot isn't bad at math; it's just bad at reading the menu. When the menu is in a language it doesn't fully grasp, it guesses the ingredients, gets confused, and orders the wrong dish. Once you give it a clear translation of the menu, it cooks the perfect meal.

Conclusion: The gap exists because the robot fails to understand the input, not because it fails to reason.

The Solution: The "Selective Translator"

Since the problem is understanding, the researchers proposed a fix called Selective Translation.

The Analogy: Imagine a bouncer at a club.

Normally, the robot tries to solve every problem on its own.
The new system adds a bouncer who listens to the robot's "thinking process" (its internal monologue).
If the robot starts sounding confused (e.g., "Wait, is this word 'apple' or 'car'? This is confusing..."), the bouncer steps in.
The bouncer says, "Stop! You don't understand this. Let me translate it to English for you first."
The robot then solves the problem using the English translation and gives the answer back in the original language.

Why is this cool?

Efficiency: They don't need to translate every question. They only translate the ones the robot is struggling with (about 20% of the time).
Performance: This method closed the gap almost as well as translating everything, but it was much faster and cheaper.

The "Early Warning System"

The researchers also found that the robot shows signs of confusion very early in its thinking process.

Analogy: It's like a student raising their hand and saying, "I'm lost," after reading just the first sentence, rather than waiting until the end of the exam to realize they didn't understand the question.
Because the robot signals confusion early, the "bouncer" can intervene quickly without waiting for the robot to waste time on a long, confused thought process.

Summary

The Problem: AI reasoning models are great at English but struggle with other languages.
The Cause: They aren't bad at logic; they are bad at understanding the foreign input. They get confused trying to translate the question in their head.
The Fix: We can detect when the AI is confused (by listening to its internal thoughts) and only then provide a translation.
The Result: This "Selective Translation" makes the AI nearly perfect in all languages without needing to translate every single question, making AI more fair and accessible to everyone, regardless of what language they speak.

Technical Summary: Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?

1. Problem Statement

Reasoning Language Models (RLMs), such as OpenAI's o1 and DeepSeek-R1, have demonstrated superior performance on complex reasoning tasks by generating long chains of thought (reasoning traces). However, a significant multilingual reasoning gap persists: these models perform substantially better on high-resource languages (e.g., English) compared to low-resource languages (e.g., Swahili, Telugu).

While previous studies have attempted to bridge this gap through prompt engineering or representation editing, the underlying causes of this disparity in RLMs have remained unexplored. The central research question is: Why do reasoning failures occur more frequently in low-resource languages, and can these failures be detected and mitigated?

2. Methodology

2.1 Theoretical Framework: Three-Stage Process

The authors conceptualize multilingual reasoning in RLMs as a three-stage process:

Language Understanding: The model internally translates the input from the source language into the dominant language of its reasoning trace (typically English).
Reasoning: The model performs logical deduction and calculation within the dominant language (English).
Generation: The model generates the final response in the original input language.

The authors hypothesize that the performance gap stems primarily from failures in the Understanding stage (misinterpreting the input) rather than the reasoning or generation stages.

2.2 Stage-wise Attribution Analysis

To quantify the contribution of each stage to the reasoning gap, the authors employ a Shapley decomposition with targeted interventions:

Understanding Intervention (U): The model is provided with a gold-standard English translation of the input as a prefix before reasoning. If performance improves significantly, the gap was caused by understanding failures.
Answer Extraction (T): The final answer is extracted directly from the reasoning trace rather than the final generation. This isolates errors caused by the generation stage.
Residual Analysis: The remaining gap after accounting for Understanding and Generation is attributed to the Reasoning stage.

2.3 Understanding Failure Detection

The paper investigates whether models leave detectable signals when they fail to understand an input. Several detection methods are evaluated:

LLM-based Detectors: Using an external LLM (GPT-4.1) to judge understanding based on the reasoning trace.
Self-Reflection: Prompting the model to explicitly state if it understood the problem.
Token-Probability Signals: Analyzing token-level confidence (average/minimum) and Input Negative Log-Likelihood (NLL) to detect uncertainty.
Supervised Approaches: Training classifiers (fine-tuned mmBERT and a hidden-state Prober) on binary labels derived from the difference between Base performance and Understanding Intervention performance.

2.4 Selective Translation Strategy

Based on the detection capability, the authors propose Selective Translation (ST). Instead of translating all inputs (which is costly), the system:

Runs the model in the Base setting.
Uses a detector to identify "understanding failures."
If a failure is detected, re-runs the model with an English translation of the input injected into the reasoning trace.
Otherwise, proceeds with the original input.

3. Key Contributions

Systematic Attribution: The first systematic analysis identifying understanding failures (the inability to translate/interpret low-resource inputs into the reasoning language) as the dominant source of the multilingual reasoning gap, rather than reasoning capability or generation errors.
Detectability of Failures: Demonstration that understanding failures produce recognizable behavioral signals (e.g., confusion, low confidence, specific phrasing in traces) that can be automatically detected with high accuracy using supervised methods.
Efficient Mitigation (Selective Translation): A novel strategy that bridges the gap by translating only the inputs where the model is likely to fail. This achieves near-full-translation performance while translating only a fraction of inputs.

4. Experimental Results

4.1 Attribution Analysis

Dominance of Understanding: Across multiple models (Qwen3 family, gpt-oss-20b) and datasets (Polymath, MMLU-ProX-Lite), Understanding failures account for 60–90% of the multilingual reasoning gap, particularly in low-resource languages.
Reasoning Gap Collapse: Once the Understanding Intervention is applied, the performance ratio between low-resource and high-resource languages approaches 1.0, indicating that the reasoning capability itself is language-agnostic once the input is understood.
Correlation: There is a strong correlation ( $r=0.951$ ) between a model's xx→en translation quality and its reasoning performance ratio.

4.2 Detection Performance

Supervised Superiority: Supervised detectors (specifically the Prober and mmBERT) significantly outperform other methods (LLM-based, self-reflection, token probabilities) in detecting understanding failures.
Generalization: These supervised detectors generalize well to unseen languages (e.g., French, Marathi, Wolof), maintaining robust performance.
Early Detection: Reliable detection is possible using only the first ~4,096 tokens of the reasoning trace, allowing for early intervention without waiting for the full trace.

4.3 Selective Translation Effectiveness

Performance Gains: Using Qwen3-4B on the Polymath-Low benchmark, Selective Translation improved average accuracy from 81.1% to 88.0%.
Efficiency: This performance is nearly identical to Full Translation (89.4%) but requires translating only ~20% of inputs (specifically those detected as failures).
Low-Resource Impact: The strategy is most effective for low-resource languages (e.g., Swahili accuracy jumped from 29.3% to 81.3% with ST), where understanding failures are most frequent.

5. Significance and Implications

Paradigm Shift: The findings challenge the assumption that multilingual reasoning gaps are due to a lack of reasoning capability in low-resource languages. Instead, the bottleneck is language understanding.
Cost-Effective Solution: Selective Translation offers a highly efficient path to equitable multilingual reasoning. It avoids the computational cost of translating every query while effectively neutralizing the performance gap for low-resource languages.
Future Directions: The work suggests that future RLM training should focus on improving the initial "understanding" phase or integrating detection mechanisms directly into the model architecture to trigger translation or abstention dynamically. It also highlights the potential for "early detection" to reduce latency in reasoning pipelines.

In summary, the paper provides a clear diagnosis of the multilingual reasoning gap (understanding failure), a reliable tool for detecting it, and an efficient, scalable strategy (Selective Translation) to mitigate it, paving the way for more equitable AI reasoning across diverse languages.