🤖 machine learning

Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics

This paper introduces "Question Begets Question" (QbQ), a self-evolving curriculum that iteratively generates problem variants from a model's current strengths to overcome data scarcity and reasoning trace limitations, successfully breaking performance plateaus in competition mathematics fine-tuning by achieving a 16.5% pass@1 score on AIME.

Original authors: Longtian Bao, Jianyou Wang, Yang Zhang, Youze Zheng, Ramamohan Paturi

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Longtian Bao, Jianyou Wang, Yang Zhang, Youze Zheng, Ramamohan Paturi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a brilliant but inexperienced student how to solve the world's hardest riddles. You have a library of riddles, but it's tiny, and you don't have the answer keys with step-by-step explanations—just the final answers. You try to teach the student by giving them more riddles, but no matter how many you add, the student hits a wall. They get stuck at a certain level of skill and refuse to get any better, even with more practice. This is a common headache for scientists working on Artificial Intelligence (AI). They want AI to learn complex skills like math or law, but they often run out of good practice problems, lack detailed "thought processes" to show the AI how to think, and watch the AI hit a frustrating "ceiling" where it just stops improving.

This paper dives into that exact problem. The researchers wanted to see if they could break that learning ceiling without needing a human teacher to write out every step of the solution. They used a smart computer program (an AI model) and a set of tricky math competition problems. Instead of just feeding the AI more of the same problems, they invented a clever trick called "Question-begets-Question." Think of it like a master chef who doesn't just give the apprentice more recipes, but takes a recipe the apprentice is almost getting right and tweaks the ingredients just enough to make a new, slightly different dish. By constantly creating these new, slightly varied challenges based on what the AI is almost mastering, the researchers found a way to push the AI past its limits.

The Story of the Self-Evolving Syllabus

The researchers started with a computer model called Qwen2.5-Math-7B. At the beginning, this model was terrible at a specific type of high-school math competition called AIME. It could only solve about 5.6% of the problems on its own. The team wanted to see if they could teach it to get better, but they faced three big hurdles: not enough practice problems, no "answer keys" with the thinking process written out, and the fear that the model would just hit a wall and stop learning.

To tackle the lack of problems, they used a method called Question-begets-Question (QbQ). Imagine you have a math problem about calculating the area of a triangle. A normal way to make more practice problems would be to just change the numbers (make the triangle bigger or smaller). But the researchers' "teacher" AI did something smarter. It took a problem the student was almost getting right and applied specific "transformation rules." For example, it might change the question from finding the area to finding the radius of a circle inside that same triangle, or it might ask for the answer in a different number system. These new problems were fresh and diverse, but they still tested the exact same core skill.

However, simply generating a huge pile of these new problems wasn't enough. When they trained the model on a static mix of real and synthetic problems, the model hit a ceiling. It improved from 5.6% to about 12.5% or 14.5%, and then it stopped. No matter how much more data they fed it, the model couldn't get any better. It was like a student who memorized the answers to a specific set of practice tests but couldn't handle a new twist on the same question.

The breakthrough came when they stopped treating the training data as a static pile and started treating it as a self-evolving curriculum. Instead of giving the model a giant bag of mixed-up problems, they organized the training into rounds. Here is how the magic happened:

The Check-In: At the end of each round, they tested the model on the problems it had just learned.
The Sweet Spot: They looked for the problems the model could get right most of the time, but not every time. They called this the "mostly-right" zone.
The Seed: They took those specific "mostly-right" problems and used them as seeds to generate the next batch of new, slightly harder variants.
The Loop: They trained the model on this new batch, checked again, found the new "mostly-right" problems, and repeated the process.

The most surprising part? The researchers found that the model improved the fastest when it practiced on problems it was already good at, rather than the ones it was failing miserably at. Usually, we think you learn by fixing your biggest mistakes. But here, the AI learned best by taking its "near-misses," tweaking them slightly, and mastering those. It was like a tennis player improving not by hitting the ball into the net over and over, but by taking a serve they usually return successfully and adding a tiny bit of spin to it, over and over again.

By using this self-evolving loop for 20 rounds, the model didn't hit a wall. It kept climbing. While other methods plateaued at around 14.5%, this self-evolving approach pushed the model's success rate up to 16.5% (specifically 16.46% in their final tests) without ever showing signs of stopping. Even more impressively, the model learned to solve harder problems it had never seen before, proving that the "ceiling" wasn't a limit of the model's brain, but a limit of how the training was organized.

The researchers also made sure the AI wasn't just copying the teacher's "thought process." They never showed the AI the teacher's step-by-step reasoning; they only showed the question and the final answer. This means the AI learned to think for itself, not just mimic a human. The study suggests that the reason AI often gets stuck isn't because it's too dumb to learn more, but because we are often feeding it the wrong kind of practice. By constantly reshaping the curriculum to match exactly where the AI is right now, we can help it break through its own limits.

Technical Summary: Question Begets Question (QbQ)

Problem Statement

The paper addresses three recurring obstacles in teaching language models (LLMs) new, difficult skills:

Data Scarcity: High-quality training data for complex tasks (e.g., competition math, legal reasoning) is often limited.
Absence of Ground-Truth Reasoning: Even when data exists, correct step-by-step reasoning traces (oracles) are frequently unavailable.
Learning Ceilings: Models often hit an apparent performance ceiling where additional data yields no further improvement, regardless of the training budget.

The authors investigate these issues in a controlled setting: fine-tuning Qwen2.5-Math-7B on AIME (American Invitational Mathematics Examination) competition problems. The base model initially solves only 5.6% of problems (pass@1), placing the task well beyond its current capability.

Methodology

The proposed solution consists of two coupled components: Question-begets-Question (QbQ) for synthetic data generation and a Self-Evolving Curriculum for training organization.

1. Question-begets-Question (QbQ)

QbQ is a scalable procedure for generating diverse synthetic training data without relying on teacher reasoning traces.

Process: A teacher model (GPT-5-mini) transforms existing "seed" problems into new variants that probe the same underlying skills but with different structural aspects.
Structural Operators: To ensure diversity and avoid superficial rewrites, the teacher is conditioned on five human-designed operators:
1. Generalize, then specialize: Replace fixed quantities with parameters.
2. Parametrize and sum: Introduce indices and ask for aggregates.
3. Change the queried quantity: Retain the setup but query a different value requiring the same lemma.
4. Inverse problem: Fix the target value and solve for a parameter.
5. Add a constraint layer: Embed the computation within an additional routine.
Constraints: Variants must have answers different from their parents. The process explicitly disallows changing only numerical values.
Diversity: QbQ achieves a higher Vendi score (1.59) compared to standard synthetic augmentation (1.25), indicating greater effective diversity.

2. Self-Evolving Curriculum

Instead of training on a static dataset, the curriculum dynamically tailors training data to the model's evolving capabilities over $T$ rounds (20 rounds in the experiments).

Self-Evaluation: At each round $t$ $t$ , the current checkpoint $\pi_t$ $π_{t}$ is evaluated on the previous round's problems. Problems are categorized into four bands based on the number of correct answers ( $n$ $n$ ) out of $k=16$ $k = 16$ rollouts:
- Mastered: $n=16$
- Mostly Right (Seed Band): $8 \le n \le 15$
- Sometimes Right: $1 \le n \le 7$
- Never Right: $n=0$
Seeding Strategy: Crucially, the curriculum selects problems from the "Mostly Right" band ( $8 \le n \le 15$ ) as seeds for the next round's QbQ generation. This is a deliberate design choice to ensure the model has frequent positive signals for reinforcement learning while the problems remain challenging enough to provide gradient variance.
Training (GRPO): The model is trained using Group Relative Policy Optimization (GRPO) on the newly generated variants.
- No Oracle Reasoning: Training uses only problem statements and final answers. No step-by-step solutions or teacher reasoning traces are exposed to the model, preventing distillation concerns and modeling settings where oracles are unavailable.
- Reward: Binary correctness reward plus a small bonus for format compliance.
Evolution: After training, problems are re-evaluated. Those that become "Mastered" are discarded; those remaining in the "Mostly Right" band become seeds for the next round. Problems that drop below the band are discarded. This ensures the training distribution evolves with model competence.

Key Contributions

QbQ Method: A scalable, operator-based method for generating diverse synthetic training problems that avoids the diversity collapse common in static augmentation.
Identification of the Ceiling: Documentation that static training (both real-plus-synthetic and non-curriculum QbQ) hits a performance plateau (capping at ~12.5% and 14.5% respectively) despite increased data volume.
Self-Evolving Curriculum: A novel iterative curriculum that breaks the performance ceiling under an identical data budget, lifting pass@1 from 5.6% to 16.5% with no sign of saturation after 20 rounds.
Counterintuitive Training Principle: The finding that models improve most when trained on variants of problems they can mostly get right (near-successes), rather than on their hardest failures or mastered problems. This approach also enables generalization to harder problems never seen during training.

Experimental Results

Experiments were conducted on Qwen2.5-Math-7B using 1,005 historical AIME problems for training and held-out AIME 2025/2026 problems for testing.

Baselines:
- Base Model: 5.62% pass@1.
- Supervised Fine-Tuning (SFT) + GRPO on original data: 11.53% pass@1.
- Static Augmentation (Real + 5k synthetic): 12.05% pass@1.
- QbQ Non-Curriculum (6k synthetic, shuffled): 14.44% pass@1.
Main Result (QbQ Curriculum):
- Achieved 16.46% ± 0.45% pass@1.
- Unlike static methods, performance continued to improve through all 3,000 GRPO updates without plateauing.
- The gain of 2.02 points over the non-curriculum QbQ (which used the exact same data and compute) isolates the curriculum organization as the source of improvement.
Generalization:
- On a subset of 499 "hard" real problems where the initial model had 0% accuracy and which were never selected as seeds, the curriculum-trained model improved pass@1 from 0.0% to 5.3%.

Significance and Claims

The paper claims that the apparent learning ceiling observed in reinforcement fine-tuning is not intrinsic to the model architecture but arises from the training distribution. By dynamically curating a curriculum based on "near-successes" and generating diverse variants via QbQ, the model can continue to learn effectively without access to ground-truth reasoning traces.

The authors emphasize that this approach:

Avoids the need for expensive or unavailable expert reasoning traces.
Mitigates concerns about knowledge distillation from teacher models.
Demonstrates that models can generalize to problems harder than those encountered during the training loop.
Challenges the prevailing intuition that models learn best from their hardest failures, suggesting instead that training on the "edge of competence" (problems mostly solved) is more effective for skill acquisition in this domain.