Code2Math: Can Your Code Agent Effectively Evolve Math Problems Through Exploration?

This paper introduces a multi-agent framework that leverages code execution to autonomously evolve existing mathematical problems into structurally distinct, more challenging, and solvable variations, offering a scalable solution to the scarcity of high-quality training data for advanced mathematical reasoning.

The Gist

Imagine you are a master chef trying to train a new generation of young cooks (the AI models) to become world-class culinary artists. You have a huge library of recipes (math problems), but most of them are simple: "How to boil an egg" or "How to make toast." To train a chef to win a Michelin star, you need recipes that are incredibly complex, requiring deep intuition, creativity, and years of practice.

The problem? Writing these super-hard recipes by hand is slow, expensive, and requires a genius chef to do it.

Enter "Code2Math": The AI Sous-Chef that invents its own challenges.

This paper introduces a system where an AI doesn't just solve math problems; it invents harder versions of them using a computer code "kitchen." Here's how it works, broken down into simple concepts:

1. The Three Chefs (The Multi-Agent System)

Instead of one AI trying to do everything, the researchers set up a team of three specialized "agents" (AI assistants) that work together like a high-end kitchen brigade:

• The Innovator (Evolution Agent): This is the creative chef. It looks at a simple recipe (a "seed" problem) and says, "How can I make this harder?" It doesn't just add more ingredients; it changes the structure of the dish. It uses a computer (Python code) to test thousands of variations instantly. It asks, "If I change this number, does the dish still work? If I add this constraint, does it break the pattern?" It's like a chef trying to turn a simple soup into a complex, multi-layered soufflé that requires a secret technique to rise.
• The Inspector (Solvability Agent): This is the strict health inspector. Before the new recipe goes to the students, the Inspector checks: "Is this dish actually edible? Did the chef make a mistake in the math? Is the solution logical?" If the recipe is broken or impossible, it gets thrown in the trash.
• The Critic (Difficulty Agent): This is the food critic. They taste the new dish and ask, "Is this actually harder, or just annoying?" They make sure the new problem isn't just "boringly long" (like peeling 1,000 potatoes) but actually requires a brilliant "Aha!" moment to solve. They want the student to have to think deeply, not just grind through calculations.

2. The "Code Kitchen" (Exploration)

The secret sauce here is Code.
Usually, when an AI tries to invent a problem, it just guesses with words. But this system uses code execution as a playground.

• Analogy: Imagine the Innovator Chef is trying to build a tower of blocks. Instead of just imagining it, they have a robot arm that can actually stack 1,000 different block configurations in a second to see which ones fall over and which ones stand tall.
• The AI writes code to simulate millions of scenarios. It tests if a new math problem has a solution, finds the "hidden patterns," and ensures the difficulty is real. This turns the invention process from a "guessing game" into a "scientific experiment."

3. The Result: "Burden of Discovery"

The paper introduces a cool concept called the "Burden of Discovery."

• Old Way: A hard problem might just have big numbers. (e.g., "Add these 500 numbers.")
• New Way: A hard problem hides the key to the solution. It's like a treasure hunt where the map is torn up. The student has to find the hidden clue (the "Aha!" moment) before they can even start solving.
• The AI agents are surprisingly good at this. They created problems that were so tricky that even the smartest AI solvers (like the current state-of-the-art models) got stuck. In fact, the AI could create problems that were harder than itself could solve! It's like a teacher creating a test that is harder than the teacher can take.

4. The Catch: It's Expensive

There is a trade-off. Because the AI has to try, fail, check, and retry so many times to make a perfect problem, it takes a lot of computing power.

• Analogy: To find one perfect diamond, the AI has to dig through a mountain of dirt. For every one good problem it creates, it might "fail" or "reject" about 3 to 6 times. It's a slow, heavy process, but the result is high-quality gold.

Why Does This Matter?

Right now, AI models are getting really good at math, but they are hitting a wall because we run out of hard problems to train them on. We can't just ask humans to write more hard problems fast enough.

Code2Math shows that we can use AI to evolve its own curriculum. It's like giving the AI a gym where it can build its own weights. By letting the AI explore, fail, and refine its own challenges using code, we can generate an endless supply of high-quality, brain-teasing math problems to push the next generation of AI (and humans) to new heights.

In a nutshell: The paper proves that if you give an AI a computer to play with and a team of critics to keep it honest, it can invent its own "final boss" math problems, pushing the boundaries of what's possible in artificial intelligence.

Technical Summary

Technical Summary: Code2Math

1. Problem Statement

As Large Language Models (LLMs) approach International Mathematical Olympiad (IMO) levels of proficiency, the field faces a critical bottleneck: the scarcity of high-quality, challenging mathematical problems for training and evaluation. Manual curation of such problems is slow, expensive, and requires deep domain expertise. Existing automated generation methods often rely on simple perturbations or lack rigorous validation, resulting in problems that are either trivial, unsolvable, or lack genuine cognitive depth.

Code2Math addresses this by investigating whether code-driven agents can autonomously evolve existing mathematical problems into structurally distinct, more challenging variations. The core challenge is to generate problems that are:

1. Mathematically Sound: Logically consistent and solvable.
2. Genuinely Harder: Increasing the "Burden of Discovery" (the cognitive effort required to find the key insight) rather than just increasing computational tedium.
3. Scalable: Generated through an automated process leveraging executable code.

2. Methodology

The authors propose a Multi-Agent Framework operating under a Test-Time Scaling paradigm. The system decomposes the long-horizon task of problem adaptation into three specialized agents, utilizing a Python sandbox with libraries like SymPy, NetworkX, and Z3 for symbolic computation and constraint satisfaction.

A. The Multi-Agent System

1. Evolution Agent:

   • Role: Analyzes a seed problem and its solution to identify the "cognitive bottleneck." It then performs computational exploration (iterative hypothesis testing, enumeration, and simulation) to design a new problem.
   • Strategy: Uses a "Theory of Mind" approach to anticipate how an experienced solver would approach the problem. It deliberately conceals insights to create an "Aha moment," aiming to increase the Burden of Discovery.
   • Process: It generates multiple "rollouts" (attempts) to find a problem that satisfies the verification criteria.

2. Solvability Verification Agent:

   • Role: Acts as a strict auditor to ensure the generated problem is well-defined and solvable.
   • Process:
     • Phase 1 (Static Check): Validates domain constraints (e.g., no division by zero, valid geometric bounds) using code.
     • Phase 2 (Logic Audit): Scrutinizes the proposed solution steps for logical flaws, inconsistencies, or invalid derivations using symbolic execution. If the solution contains logical errors, the problem is rejected.

3. Difficulty Verification Agent:

   • Role: Evaluates the relative difficulty between the seed and evolved problems.
   • Criteria: Distinguishes between Artificial Complexity (tedious calculations, larger numbers) and Cognitive Depth (new insights, structural changes).
   • Scoring: Uses a 5-point scale. Scores 1-2 (Fail) indicate trivial changes or mere computational inflation. Scores 3-5 (Pass) require the problem to break standard solution templates and force a non-obvious insight.

B. Workflow

The system takes a seed problem and solution as input. The Evolution Agent generates candidate problems via code execution. These candidates are filtered by the Solvability and Difficulty agents. If a candidate fails, the Evolution Agent iterates (up to a budget of 20 rollouts) to refine the problem. Successful problems are output with a canonical solution.

3. Key Contributions

1. Novel Framework for Problem Evolution: A multi-agent system that decomposes problem adaptation into specialized roles (Evolution, Solvability, Difficulty), leveraging code execution for rigorous validation and exploration.
2. Definition of "Burden of Discovery": The paper operationalizes difficulty not as calculation volume but as the cognitive gap required to find the solution path, specifically targeting "anti-templating" capabilities that prevent rote application of standard heuristics.
3. Empirical Evidence of Capability Asymmetry: Demonstrates that agents can synthesize problems that exceed their own solving baselines, effectively creating challenges that current state-of-the-art models cannot solve.
4. High-Quality Dataset Generation: The framework successfully generates mathematically sound problems with significantly increased difficulty, validated by external judges and multiple solver models.

4. Experimental Results

The study evaluated the framework using 100 seed problems from diverse sources (IMO, AIME, textbooks) and tested them across 6 different solver models (including DeepSeek, Gemini, Qwen, and GPT-5 variants).

• Solvability & Reliability:
  • The framework achieved high Agreement Rates (AR) between the internal Solvability Agent and an external judge (GPT-5.2-High).
  • Example: DeepSeek-Reasoner achieved a 96% agreement rate (94/98), and Gemini-3-Pro achieved 100% (98/98), indicating the generated problems are logically sound.
• Difficulty Escalation:
  • Solve Rate (SR) Drop: Evolved problems consistently reduced the solve rates of all solver models compared to seed problems.
  • Example: On problems evolved by DeepSeek-Reasoner, the solve rate for the strong solver GPT-5.2-High dropped from 70% to 64%, while Gemini-3-Flash-Thinking dropped from 56% to 35%.
  • Token Consumption: The Average Token Consumption (ATC) for solving evolved problems increased significantly (e.g., median tokens rose from ~9,600 to ~17,000 for DeepSeek-Reasoner evolutions), indicating deeper reasoning chains and more extensive exploration were required.
• Efficiency & Cost:
  • The process is computationally intensive. Generating a single qualified problem required an average of 1.56 to 6.55 failed rollouts depending on the model.
  • The primary bottleneck was Solvability Verification (logical consistency), not difficulty assessment, highlighting the challenge of ensuring mathematical rigor in autonomous generation.
• Case Studies:
  • The paper provides detailed examples (e.g., transforming a simple inequality proof into a complex extremal characterization problem involving moment theory) showing how the system generalizes problems from specific cases to broader, structurally richer theorems.

5. Significance and Conclusion

Code2Math demonstrates that code-driven agents are a viable mechanism for synthesizing high-difficulty mathematical reasoning data. By combining the exploratory power of code execution with multi-agent verification, the system overcomes the limitations of static, text-based problem generation.

• Scalability: It offers a path to automatically generate the vast quantities of challenging data needed to train the next generation of reasoning models.
• Benchmarking: The evolved problems serve as a more discriminatory benchmark, exposing limitations in reasoning robustness that standard benchmarks miss.
• Future Directions: The authors suggest that while current methods involve computational overhead, future work could focus on improving rollout efficiency and extending these exploratory strategies to other structured reasoning domains beyond mathematics.

In summary, the paper establishes that autonomous problem evolution via code agents is not only possible but effective at creating mathematically valid problems that genuinely challenge the reasoning capabilities of current AI models.