CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning

1. Problem Statement

Large Language Models (LLMs) have demonstrated strong capabilities in monolingual mathematical and commonsense reasoning but remain unreliable for multilingual medical reasoning. Current models face two critical failure modes in clinical settings:

1. Logical Inaccuracy: Models often generate fluent, plausible-sounding explanations that lack clinical validity or contain hallucinations.
2. Language Instability (Drift): In multilingual settings, models frequently fail to maintain the target language, reverting to English or mixing languages (code-switching) inappropriately, which erodes interpretability and trust for clinicians and patients.

Existing benchmarks are predominantly monolingual, closed-form (multiple-choice), or lack rigorous verification of reasoning traces. There is a significant gap in evaluating and training models for open-ended medical reasoning across diverse, low-resource languages (e.g., Amharic, Yoruba, Swahili).

2. Methodology

The authors propose CURE-MED, a two-stage training framework designed to jointly optimize logical correctness and language fidelity. The methodology consists of three core components:

A. Dataset Construction: CUREMED-BENCH

To address the lack of high-quality evaluation data, the authors introduced CUREMED-BENCH, a dataset of 15,774 open-ended medical QA instances across 13 languages (spanning Africa, Asia, and Europe).

• Source: Grounded in MedlinePlus (clinically validated US federal health data).
• Pipeline:
  1. Generation: GPT-4o generates closed-ended MCQs based on MedlinePlus content in each target language.
  2. Filtering: Trivial questions are removed by testing against compact LLMs (Qwen2.5-3B/7B, LLaMA-3.1-8B). Ambiguous items are filtered to ensure a single verifiable answer.
  3. Conversion: Remaining items are converted to open-ended prompts with explicit reasoning chains ($r$) and free-form ground-truth answers ($y^*$).
  4. Verification: Native speakers and medical experts (physicians, PhD candidates) verify clinical correctness, linguistic fidelity, and cultural appropriateness.

B. Training Framework: CURE-MED

The framework employs a Curriculum-Informed Reinforcement Learning approach with two distinct stages:

1. Stage 1: Cold-Start Supervised Fine-Tuning (SFT)

   • Objective: Stabilize multi-step reasoning before enforcing strict language constraints.
   • Technique: The model is fine-tuned on code-switched reasoning trajectories. Intermediate reasoning steps ($r$) are allowed to mix English (for precise clinical terminology) with the target language, while the final answer ($y^*$) must be strictly in the target language.
   • Benefit: This initializes the model with strong logical reasoning capabilities without forcing premature language rigidity.

2. Stage 2: GRPO-Guided Curriculum Reinforcement Learning

   • Algorithm: Uses Group Relative Policy Optimization (GRPO).
   • Curriculum Design: Training progresses based on language resource availability rather than problem complexity.
     • Tier 1 (High-Resource): French, Japanese, Spanish, Vietnamese.
     • Tier 2 (Medium-Resource): Korean, Thai, Turkish, Bengali.
     • Tier 3 (Low-Resource): Amharic, Yoruba, Hausa, Hindi, Swahili.
   • Strategy: The model trains on high-resource languages first to establish stable reward signals, then progressively expands to medium and low-resource tiers. To prevent catastrophic forgetting, data from previous tiers is retained (retention ratio $\alpha=0.85$).
   • Reward Function: A composite reward $R$ combines:
     • Correctness ($R_{acc}$): Semantic and clinical equivalence to the ground truth (verified by GPT-4.1).
     • Language Consistency ($R_{lang}$): Binary reward for outputting the answer entirely in the query language.
     • Format Reward ($R_{fmt}$): Adherence to structured output tags (<thinking>, <step>, <answer>).

3. Key Contributions

1. CUREMED-BENCH: The first large-scale, open-ended multilingual medical reasoning benchmark covering 13 languages, including underrepresented low-resource languages, with human-verified clinical validity.
2. CURE-MED Framework: A novel training pipeline combining code-switching-aware SFT (to leverage English medical knowledge) with curriculum-informed GRPO (to enforce language fidelity and logical consistency).
3. Systematic Evaluation: A comprehensive analysis of the trade-off between language consistency and logical accuracy, demonstrating that these metrics are often decoupled in baseline models but jointly optimized by CURE-MED.
4. State-of-the-Art Performance: The model achieves superior results across varying parameter scales (1.5B to 32B) and demonstrates robust out-of-distribution (OOD) generalization.

4. Experimental Results

The model was evaluated against 28 baselines (general-purpose and medical-specific LLMs) across 13 languages.

• Performance Metrics:
  • 7B Parameters: Achieved 85.21% language consistency and 54.35% logical correctness.
  • 32B Parameters: Achieved 94.96% language consistency and 70.04% logical correctness.
• Comparative Analysis:
  • CURE-MED significantly outperformed all baselines, including proprietary models (e.g., GPT-5-nano, Gemini 2.5 Pro) and medical-specialized models (e.g., MedAlpaca, Meditron).
  • Trade-off: While baselines often sacrificed language consistency for reasoning (or vice versa), CURE-MED shifted performance to the "upper-right" of the consistency-accuracy plot, improving both simultaneously.
• Scaling Laws: The framework showed smooth scaling; as model size increased from 1.5B to 32B, both consistency and accuracy improved monotonically, whereas baselines showed diminishing returns or instability.
• Ablation Studies:
  • Code-Switching SFT: Crucial for initializing reasoning; naive SFT without code-switching led to lower accuracy.
  • Curriculum Order: Training from High $\to$ Low resource languages yielded significantly better results than the reverse order, confirming the importance of curriculum design for low-resource transfer.
• Out-of-Distribution (OOD): CURE-MED generalized effectively to unseen benchmarks (MMedBench, MedExpQA, MedQA), showing particularly strong gains in smaller models (e.g., 1.5B model accuracy jumped from ~6% to ~24-44% on OOD tasks).

5. Significance

• Equitable Healthcare AI: By successfully addressing low-resource languages (Amharic, Yoruba, Swahili), CURE-MED moves beyond the English-centric bias of current medical AI, enabling more equitable deployment in diverse global healthcare settings.
• Reliability: The framework solves the "plausible but wrong" problem by enforcing strict logical verification and language constraints, making the models safer for clinical decision support.
• Methodological Insight: The paper demonstrates that curriculum learning based on language resources is a superior strategy for multilingual reasoning compared to uniform training, and that code-switching during intermediate reasoning is a viable strategy to bridge the gap between high-resource knowledge and low-resource output.

In conclusion, CURE-MED establishes a new standard for multilingual medical reasoning, proving that with the right curriculum and reward structures, LLMs can achieve high-fidelity, clinically accurate reasoning across a wide spectrum of global languages.