ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance

This paper introduces ReasonXL, a large-scale multilingual reasoning corpus, and demonstrates that a two-stage fine-tuning and reinforcement learning pipeline can successfully adapt LLMs to reason in non-English languages without sacrificing performance, while revealing that language identity is determined by early model layers and adaptation is concentrated in upper layers.

The Gist

Imagine you have a brilliant, multilingual genius named Reasoning-Rob. He can solve complex math problems, write code, and explain science, but he has a strange quirk: no matter what language you speak to him, he always thinks in English inside his head before speaking your language.

If you ask him a question in German, he translates your question to English, does all the hard thinking in English, and then translates the answer back to German.

The Problem:
This creates a few issues:

1. Trust: If you don't speak English, you can't see how he solved the problem. You just see the answer, which feels like magic rather than logic.
2. Errors: Sometimes, translating a tricky math problem into English and back loses the "flavor" or specific meaning of the original question.
3. Bias: He's just better at English. His English answers are usually more accurate than his German or French ones.

The paper introduces a project called ReasonXL to fix this. Here's how they did it, explained with some everyday analogies:

1. The Massive Library (The Dataset)

To teach Reasoning-Rob to think in other languages, the researchers couldn't just tell him "Please think in German." They needed to show him examples of thinking in German.

They built ReasonXL, a massive digital library containing over 2 million examples for five languages (English, German, French, Italian, and Spanish).

• The Analogy: Imagine they took a huge pile of English textbooks where the author wrote out their step-by-step thinking process, and they hired a team of expert translators to rewrite every single step of the thinking process into the other languages.
• The Result: Now, instead of just having "Question -> Answer," they have "Question -> Thinking Process in German -> Answer."

2. The Two-Step Training Camp (The Method)

They didn't just dump this library into the robot's brain. They used a two-step training camp to rewire his brain without breaking his intelligence.

Step A: The "Language Swap" (Supervised Fine-Tuning)

• What happened: They showed the robot the new library and said, "From now on, when you see a German question, you must write your thoughts in German."
• The Side Effect: It worked! The robot started thinking in German. But, like a student who is so focused on speaking the new language that they forget the math, his answers got a bit worse. He was thinking in German, but his logic was shaky.

Step B: The "Coach's Whistle" (Reinforcement Learning)

• What happened: They gave the robot a second round of training. This time, they acted like a strict coach. The robot would try to solve a math problem in German. If the answer was right and the thinking was in German, the coach gave a high-five (a reward). If the answer was wrong or he slipped back into English, the coach gave a gentle tap on the wrist (a penalty).
• The Result: This "coach" helped the robot regain his sharp logic while keeping his new habit of thinking in German. Suddenly, he was thinking in German and getting the answers right, often even better than before!

3. The "Brain Scan" (The Discovery)

The researchers didn't just stop at the results; they looked inside the robot's "brain" (the neural network) to see how the change happened. They found two fascinating things:

• The "Switch" Layer: They discovered that the robot's brain has a specific "switch" located in the middle layers (around layers 6–8). This is where the decision to speak German or English is actually made. It's like a traffic light that decides which language highway the thoughts travel on.
• The "Refinement" Layers: The top layers of the brain (the later ones) are where the actual heavy lifting happens—solving the math, writing the code, and refining the logic.
• The Efficiency Surprise: They found that the second training step (the "Coach's Whistle") was incredibly efficient. It managed to completely change the robot's behavior by tweaking very few parts of the brain, whereas the first step (the "Language Swap") required changing a lot of weights. It's like the first step built a new road, but the second step just installed a few smart traffic signs that made the whole system run perfectly.

Why Does This Matter?

Before this, the AI world assumed that to be smart, you had to think in English. This paper proves that you don't.

• For Users: You can now get AI assistants that explain their reasoning in your native language, making them easier to trust and understand.
• For AI: It shows that we can teach AI to be "native" in any language without losing its smarts.

In a nutshell: The researchers built a giant library of "thinking in other languages," taught a robot to use it, and discovered that the robot's brain has a specific "switch" for language and a "workshop" for logic. They proved that an AI can be just as smart in German, French, or Spanish as it is in English, as long as we give it the right tools to learn.

Technical Summary

1. Problem Statement

Despite advances in multilingual Large Language Models (LLMs), a fundamental mismatch exists in reasoning tasks:

• English-Centric Reasoning: Even when prompted in non-English languages, LLMs predominantly generate intermediate reasoning traces (Chain-of-Thought) in English.
• Barriers to Trust and Understanding: This creates a barrier for non-English users who cannot verify the model's logic, reducing trust and transparency.
• Performance Disparity: Models consistently achieve higher accuracy in English than in other languages.
• Data Scarcity: There is a lack of large-scale, parallel datasets containing reasoning traces in multiple languages, hindering direct supervised training for non-English reasoning.
• Optimization Gap: Existing attempts to force non-English reasoning often degrade performance (the "accuracy vs. compliance" trade-off).

2. Methodology

The authors propose a comprehensive solution involving a new dataset and a two-stage training pipeline.

A. Dataset: ReasonXL

• Scale & Scope: The first large-scale parallel corpus of cross-domain reasoning traces spanning five European languages: English, German, French, Italian, and Spanish.
• Volume: Over 2 million aligned samples per language (approx. 9 billion tokens in the current release, expanding to 20–30B).
• Composition: Derived from 10 existing sources (e.g., MATH, GSM8K, scientific QA, code).
• Construction Pipeline:
  1. Annotation: English samples are annotated using Propella-1 (a small multilingual LLM) to filter for safety, integrity, and domain diversity (math, code, science, etc.).
  2. Translation: Samples (input, reasoning trace, output) are translated independently using Qwen3-32B. The translation prompt is engineered to preserve technical terminology, mathematical notation (LaTeX), and code while adapting tone and cultural references.
  3. Quality Control: Rigorous filtering ensures high information density and educational value.

B. Training Pipeline: Two-Stage Adaptation

The authors adapt a base model (SmolLM3-3B) to reason in a target language using a two-stage process:

1. Stage 1: Supervised Fine-Tuning (SFT)

   • Goal: Shift the model's reasoning language from English to the target language.
   • Method: Train on the parallel corpus using completion-only loss (masking user/system tokens).
   • Outcome: Successfully forces the model to output reasoning in the target language but often causes a significant drop in reasoning accuracy (5–10 points).

2. Stage 2: Reinforcement Learning with Verifiable Rewards (RLVR)

   • Goal: Recover the reasoning quality lost during SFT while maintaining target-language compliance.
   • Algorithm: Uses Dr. GRPO (a variant of Group Relative Policy Optimization).
   • Reward Function: A composite reward consisting of:
     • Accuracy (w=1.0): Binary signal based on symbolic equivalence between the predicted answer and ground truth.
     • Language (w=0.1): FastText-based score ensuring the reasoning trace and output are in the target language.
     • Format (w=0.2): Rewards correct use of <thought> tags and \boxed{} formatting.
     • Repetition Penalty (w=0.3): Prevents length-hacking and token flooding.
     • Naturalness (Spanish only, w=0.5): Penalizes unnatural punctuation patterns (e.g., inverted question marks in declarative sentences).

3. Key Contributions

1. ReasonXL Corpus: Release of the first large-scale, five-language parallel reasoning dataset with rich metadata, enabling direct supervision of language-specific reasoning.
2. Feasibility of Non-English Reasoning: Demonstration that LLMs can be adapted to reason entirely in a target language without sacrificing (and sometimes improving) task performance, challenging the assumption that English is a necessary intermediary for high-quality reasoning.
3. Mechanistic Insights: A representational analysis revealing how language adaptation reshapes model internals:
   • Weight Changes: Adaptation concentrates the largest weight updates in the upper layers of the transformer.
   • Activation Bottleneck: A causal "bottleneck" exists in layers 6–8 (early-to-mid layers). Patching these layers with target-language activations in the base model shifts the output language, suggesting language identity is determined early and propagated.
   • Efficiency: RLVR achieves greater behavioral divergence from the base model with significantly smaller parameter updates compared to SFT, suggesting a more efficient representational rerouting.

4. Results

Performance on Target-Language Tasks

• SFT Impact: Successfully shifts reasoning to the target language (100% compliance) but degrades performance on science benchmarks (MATH100, GPQA, MGSM) by ~5–10 points.
• RLVR Recovery: The RL stage recovers performance, with final models matching or exceeding the base model's performance in the target language.
  • Example: For Italian, the base model scored 64.9 (avg); SFT dropped to 65.9 (but 100% target language); RLVR improved to 71.0, outperforming the English-only base.
• Generalization: Improvements extend beyond science tasks to out-of-domain benchmarks (Belebele, Global MMLU), indicating strengthened general in-language capabilities.

General Knowledge & Cross-Lingual Transfer

• Knowledge Retention: English general knowledge benchmarks (MMLU, PIQA, etc.) show minimal degradation (0.6–3.2 points), indicating the model's factual knowledge base remains intact despite the reasoning shift.
• Cross-Lingual Effects:
  • Adaptation to Italian and German yields positive transfer to other languages.
  • Adaptation to French causes some degradation in other languages.
  • English performance declines slightly, representing the primary trade-off.
  • Crucially, the shift in reasoning language does not disrupt underlying language-agnostic representations significantly.

5. Significance and Implications

• Paradigm Shift: The paper challenges the prevailing "English-as-pivot" hypothesis, proving that high-quality reasoning can be achieved directly in non-English languages.
• User Trust: By enabling transparent, followable reasoning in a user's native language, the approach enhances trust and usability for non-English speakers.
• Efficient Adaptation: The finding that RLVR induces large behavioral changes with minimal weight updates suggests that future adaptation strategies could be more parameter-efficient.
• Mechanistic Understanding: The identification of a specific "language-routing bottleneck" (layers 6–8) provides actionable guidance for future model architecture design and parameter-efficient fine-tuning (e.g., targeting specific layers).
• Resource Availability: The release of ReasonXL provides a critical foundation for future research in multilingual reasoning, curriculum design, and cross-lingual transfer.

In conclusion, ReasonXL demonstrates that with sufficient data and a structured two-stage training pipeline, the reasoning language of an LLM can be durably redirected to a target language without compromising performance, offering a path toward truly multilingual and transparent AI.