Technical Summary: AdaMame – A Training Recipe for Adaptive Multilingual Reasoning

Problem Statement

Large Reasoning Models (LRMs) exhibit strong performance in English but frequently fail to reason in the language of the user's query, a phenomenon termed language collapse. Instead of generating reasoning traces (Chain-of-Thoughts) in the query language, models default to English or engage in mid-trace code-switching.

Existing attempts to rectify this via Reinforcement Learning (RL) typically append a binary language fidelity reward to the accuracy objective using a manually tuned weighting ratio. However, the paper identifies three persistent limitations in these approaches:

1. Accuracy Trade-offs: Improving language fidelity often comes at the cost of reasoning accuracy.
2. Code-Switching: Models frequently alternate languages within a single reasoning trace.
3. Overthinking: Models consume excessive tokens without proportional gains in correctness.
   Furthermore, many prior methods require English reference reasoning traces for supervision and rely on fixed weighting regimes, limiting scalability and robustness.

Methodology: AdaMame

The authors propose AdaMame, a two-stage training recipe designed to adaptively align reasoning language with the query language while jointly optimizing for accuracy and token efficiency.

Stage 1: Supervised Fine-Tuning (SFT)

• Goal: Establish foundational multilingual reasoning capability and sensitivity to language-specific patterns.
• Data: A dataset of 30K naturally occurring reasoning traces across five languages (French, Portuguese, Japanese, Korean, Thai). Crucially, these traces are not machine-translated from English but are generated and verified to be fluent in the target language.
• Technique: The model is fine-tuned using Low-Rank Adaptation (LoRA) to prevent catastrophic forgetting and improve efficiency. Prompts explicitly encourage reasoning in the query language.

Stage 2: Reinforcement Learning (AdaMame-GRPO)

• Goal: Generalize reasoning capabilities to out-of-domain languages and enforce language alignment without sacrificing accuracy.
• Core Innovation: An adaptation of Group Relative Policy Optimization (GRPO) called AdaMame-GRPO.
• Mechanism: Instead of a fixed reward weight, the method introduces a query-conditioned alignment factor ($\alpha_i(t)$) that grows progressively during training.
  • Exploration Phase: Early in training, the alignment factor is weak, allowing the model to explore diverse reasoning languages to maximize accuracy.
  • Exploitation Phase: As training progresses, the factor increases, amplifying rewards for rollouts where the reasoning language matches the query language.
  • Mathematical Formulation: The reward is scaled as $r'(q, o_i, g) = \alpha_i(t) \cdot r(q, o_i, g)$, where $\alpha_i(t)$ follows a cosine growth schedule from 0.1 to 1.0.
• Advantage: This approach requires no English reference traces and avoids manual tuning of reward weights, instead using a curriculum-based "explore-then-exploit" strategy.

Key Contributions

1. AdaMame Recipe: A two-stage (SFT + RL) post-training framework that adaptively aligns reasoning language to the query, resolving the trade-off between accuracy and language fidelity.
2. Dataset Release: A new dataset of 30K naturally occurring, high-quality reasoning traces across five languages, supporting multilingual reasoning research without reliance on machine-translated English data.
3. AdaMame-GRPO Algorithm: A novel RL variant that achieves Pareto-optimal performance across reasoning accuracy, language fidelity, and token efficiency.

Experimental Results

The method was evaluated on two benchmarks (MGSM-Rev2 and MSVAMP) using two backbone models (DISTILL-QWEN 1.5B and QWEN3 4B) across 12 languages (5 in-domain, 7 out-of-domain).

• Pareto Optimality: AdaMame-GRPO achieved the best balance of accuracy, language fidelity (measured by Language Confusion Pass Rate, LCPR), and token efficiency (Test-Time Compute, TTC) compared to all baselines, including Vanilla, Prompt, SFT, standard GRPO, and M-Thinker.
• Out-of-Domain Generalization: While SFT improved in-domain performance, it generalized poorly to out-of-domain languages. The RL stage significantly improved generalization, particularly for low-resource languages (e.g., Bengali, Swahili, Telugu), where AdaMame-GRPO showed the largest gains over baselines.
• Token Efficiency: Unlike methods that induce "overthinking," AdaMame-GRPO resulted in the lowest token consumption among all tested variants.
• Curriculum Effect: Analysis confirmed the "explore-then-exploit" dynamic: models initially generated diverse languages before progressively converging on the query language as the alignment factor grew.

Significance and Claims

The paper claims that AdaMame addresses the core limitations of current multilingual reasoning approaches by eliminating the need for English reference traces and fixed reward weights. By implementing an adaptive curriculum, the method successfully mitigates language collapse and code-switching while maintaining or improving reasoning accuracy.

The authors position this work as a step toward more equitable multilingual reasoning, demonstrating that high performance is achievable not just for high-resource languages but also for low-resource and out-of-domain settings. They argue that moving beyond English-centric reward designs is essential for training models that serve the full diversity of user languages.

Limitations Acknowledged:

• Experiments are constrained by computational budget, limiting the scope to specific model sizes and mathematical reasoning tasks.
• The method relies on the LINGUA language detector, which may have reliability issues with heavily code-switched traces or very low-resource languages.
• The training corpora depend on traces generated by GPT-5 NANO, potentially inheriting its specific language biases.