mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR

The paper introduces mAceReason-Math, a high-quality multilingual dataset comprising over 10,000 challenging math problems per language across 14 languages, specifically curated and cleaned to advance Reinforcement Learning with Verifiable Rewards (RLVR) research beyond the current English-centric landscape.

The Gist

Imagine you have a brilliant, super-smart robot tutor (a Large Language Model) that has been trained to solve incredibly difficult math problems. But there's a catch: this robot only speaks English.

If you want to teach this robot to help students in Tokyo, Berlin, or São Paulo, you can't just ask it to "speak their language." You have to teach it how to think in those languages. Currently, the best training materials for these robots are all in English. It's like trying to teach a chef French cuisine using only a cookbook written in English, hoping they'll figure out the French terms on their own. It doesn't work well.

This paper introduces mAceReason-Math, a massive new library of math problems designed to fix this. Here is the story of how they built it, explained simply:

1. The Problem: The "English-Only" Barrier

For a few years, researchers have been using a special training method called RLVR (Reinforcement Learning with Verifiable Rewards). Think of this as a video game where the robot tries to solve a math problem, gets a "gold star" if it's right, and learns from its mistakes.

To make the robot really good, the problems need to be just the right difficulty—like a video game level that is challenging but not impossible. The best "levels" (datasets) exist, but they are all in English. Previous attempts to translate these problems were either too easy (like kindergarten math) or too messy to be useful for training a super-advanced robot.

2. The Solution: A High-Quality Translation Factory

The team created a dataset with over 140,000 math problems translated into 14 different languages (including German, Chinese, Japanese, Russian, and even Swahili).

But they didn't just use a basic translator app. They built a hybrid translation factory with three strict steps:

• Step 1: The Cleanup Crew (Filtering)
  Before translating, they had to clean the original English problems. Imagine a pile of old homework papers. Some had the answers written on the back, some had torn pages, and some referenced diagrams that were missing.

  • They threw out the "broken" papers (about 4%).
  • They fixed the "messy" papers (about 11%) by removing weird formatting or instructions like "Task 5.4" that didn't belong in the actual question.

• Step 2: The AI Translators (The First Draft)
  They used a powerful AI (Claude Sonnet 4) to translate the cleaned problems. But AI can be tricky with math. It might accidentally change a number or translate a formula like $x^2$ into a word, which breaks the math.

  • They gave the AI strict rules: "Do not touch the numbers or the formulas. Only translate the words around them."

• Step 3: The Human Editors (The Final Polish)
  This is the most important part. They hired native speakers (real humans who grew up speaking these languages) to review the AI's work.

  • They checked: "Does this sound like a real math problem a student in Japan would see?"
  • They caught subtle errors, like using the wrong word for "sequence" in German or formatting numbers incorrectly (e.g., using a comma instead of a dot for decimals).
  • If a translation was bad, the AI tried again, and the humans checked it again, up to five times, until it was perfect.

3. The Result: A Global Math Gym

The final product is a massive, high-quality gym for math robots.

• The Parallel Set: They have a "gold standard" set of 7,620 problems where every single problem exists in all 14 languages. This allows researchers to compare exactly how well a robot performs in French vs. Chinese using the exact same test.
• The Full Set: They also have over 10,000 problems per language, giving researchers plenty of data to train their models.

4. Why This Matters

The paper tested several robots on this new dataset. They found that:

• Bigger, smarter robots generally did better.
• However, performance varied wildly by language. Some robots were great at English math but stumbled when the problem was in Thai or Portuguese.
• This dataset is now a benchmark. It's the new "standardized test" that researchers will use to see if their multilingual math robots are actually getting smarter.

The Big Picture Analogy

Think of the previous English-only datasets as a monolingual library. You could only learn math if you spoke English.

mAceReason-Math is like building a global university where the same advanced math curriculum is available in 14 different languages, with the same high-quality textbooks, the same difficult exams, and the same rigorous grading. Now, researchers can finally teach their AI students to think logically, no matter what language they speak.

Technical Summary

Here is a detailed technical summary of the paper "mAceReason-Math: A Dataset of High-quality Multilingual Math Problems."

1. Problem Statement

Reinforcement Learning with Verifiable Rewards (RLVR), particularly when combined with Group Relative Policy Optimization (GRPO), has recently driven significant advancements in Large Language Models (LLMs) for mathematical and logical reasoning. However, a critical bottleneck exists:

• English-Centric Limitation: Current high-quality training datasets (e.g., AceReason-Math, DAPO-Math-17k) and benchmarks are almost exclusively in English.
• Difficulty Mismatch: Existing multilingual math datasets (e.g., GSM8K translations) are often too easy to provide effective training signals for state-of-the-art models, which require "Goldilocks" difficulty levels to optimize.
• Scalability Gap: There is a lack of large-scale, parallel, high-difficulty multilingual datasets necessary to study and implement RLVR in non-English contexts.

2. Methodology

The authors present mAceReason-Math, a dataset created through a rigorous, hybrid pipeline combining LLM automation with human native-speaker verification. The process involves three main stages:

A. Data Cleaning and Filtering (Base: AceReason-Math)

The authors started with the English AceReason-Math corpus and applied a two-tier filtering strategy to remove ~15% of the data:

1. Critical Issues (~4%): Samples were discarded if they contained:
   • External dependencies (links to images, diagrams, or external URLs).
   • Answer leakage (solutions embedded in the question).
   • Open-ended questions without single numerical answers (e.g., "How can you...").
   • Non-English content or missing context definitions.
   • Formatting artifacts like \boxed{} instructions.
2. Salvageable Issues (~11%): These were corrected programmatically or via LLM prompts before translation. This included removing task annotations (e.g., "Task 5.4"), fixing LaTeX extraction errors, and removing dataset construction instructions.

B. Hybrid Translation Pipeline

To scale translation to 14 languages without prohibitive human costs, the authors employed an iterative LLM-based refinement process using Claude Sonnet 4:

1. Initial Translation: LLMs translated cleaned English problems into target languages.
2. Human Validation (Pilot): A sample of 100 translations per language was reviewed by native speakers. Feedback focused on LaTeX integrity, mathematical terminology, and number formatting conventions (e.g., 12.345,67 for German vs. 12,345.67 for English).
3. Iterative Refinement: The full dataset underwent an automated grading loop. Translations were assessed against strict criteria (mathematical accuracy, LaTeX preservation, natural fluency). If issues were detected, a refinement prompt was used to correct them. This loop ran up to five times; samples failing to pass were discarded.
4. Special Handling:
   • LaTeX: Strict rules ensured mathematical notation remained untouched, with only natural language inside \text{} translated.
   • Asymptote Code: Samples containing [asy] code were translated but separated into a distinct split due to the specialized skill required to parse them.

C. Dataset Compilation

The final dataset includes:

• Languages: 14 languages (English, German, French, Spanish, Chinese, Russian, Japanese, Korean, Thai, Portuguese, Italian, Swahili, Telugu, Bengali).
• Parallel Split: 7,620 samples per language where translations exist for all 14 languages, ensuring strict comparability for training.
• Full Split: 10,000+ samples per language (varying by language), including non-parallel data.
• Test Set: A human-validated set of 190 samples per language, with LaTeX rendered via MathJax for annotator review.

3. Key Contributions

• mAceReason-Math Dataset: The first large-scale (140k+ total samples), high-difficulty multilingual math dataset specifically curated for RLVR training.
• Hybrid Pipeline: A scalable methodology combining LLM translation with native speaker verification that balances cost and quality, overcoming the "prohibitive cost" of human translation at this scale.
• Quality Control Framework: A rigorous cleaning and refinement process that addresses specific failure modes in math translation, such as LaTeX corruption, number formatting localization, and mathematical terminology accuracy.
• Open Release: The dataset is publicly released to facilitate multilingual RLVR research and benchmarking.

4. Results and Evaluation

The authors evaluated the dataset using a human-validated test set (190 samples) across various frontier and open-weight models:

• Model Performance Trends:
  • Scale Correlation: Larger models consistently outperformed smaller ones within the same family.
  • Language Variance: Significant performance gaps were observed across languages, particularly in open-weight models. For example, gpt-oss-20b achieved ~76.3% average accuracy, while SmolLM3-3B dropped to ~56.8%, with lower performance in low-resource languages (e.g., Swahili, Telugu).
  • Closed vs. Open Models: Closed models (e.g., Gemini 2.5 Flash, Claude Sonnet 4.5) generally achieved higher pass@1 accuracy (~70–81%) compared to open-weight models using avg@8 evaluation.
• Qwen3 Findings: The Qwen3 reasoning models showed exceptional performance, with the 14B variant achieving ~88.6% average accuracy, outperforming many frontier closed models. This suggests strong cross-lingual transferability of math reasoning capabilities in these models.
• Validation: The human review process was highly effective, filtering out <0.1% of translations as "problematic" after the iterative refinement loop.

5. Significance

• Enabling Multilingual RLVR: This dataset removes the English barrier for RLVR research, allowing researchers to train and evaluate GRPO and other RL methods in diverse linguistic contexts.
• Benchmarking Standard: It provides a high-difficulty benchmark to test the limits of multilingual reasoning, moving beyond simple translation tasks to complex problem-solving.
• Cross-Lingual Insights: The evaluation results highlight that while math reasoning is transferable, model performance is still heavily dependent on language-specific training data quality and volume.
• Future Research: The dataset serves as a foundation for investigating how RLVR scales across languages and whether multilingual training improves reasoning capabilities in low-resource languages.

Limitations:
The authors acknowledge that despite rigorous checks, residual translation errors may persist, particularly for languages without native speaker validators (Swahili, Telugu, Bengali). Additionally, automated answer verification tools may struggle with complex non-US number formatting or expressions.