QED-Nano: Teaching a Tiny Model to Prove Hard Theorems

1. Problem Statement

The paper addresses the challenge of training small, open-source language models (specifically a 4B parameter model) to solve Olympiad-level mathematical proof problems.

• Context: While proprietary systems (e.g., Gemini 3 Pro) and massive open models (e.g., 685B DeepSeek-Math-V2) have achieved gold-medal performance on the International Mathematical Olympiad (IMO), their training pipelines are often undisclosed, expensive, and rely on massive compute resources.
• Gap: There is a lack of understanding regarding how small models can be post-trained to handle long-horizon, globally coherent, and rigorous natural language proofs, as opposed to simple answer verification.
• Goal: To demonstrate that a 4B model can compete with much larger systems (30B–685B) and proprietary models through a specialized post-training recipe that leverages test-time compute and iterative reasoning.

2. Methodology

The authors propose a three-stage post-training pipeline for the QED-Nano model, built upon a Qwen3-4B base.

Stage 1: Supervised Fine-Tuning (SFT)

• Objective: To imbue the model with a foundational ability to write mathematical proofs and adopt a rigorous style.
• Data Source: The team distilled reasoning traces from DeepSeek-Math-V2 (a 685B model). They generated ~7,500 proof solutions for ~4,300 distinct Olympiad-level problems.
• Process: The base model was fine-tuned on these synthetic proofs.
• Challenge Identified: SFT alone led to "length explosion," where the model would generate excessively long, repetitive, or meandering text without reaching a conclusion, mimicking the superficial form of long proofs rather than the logical structure.

Stage 2: Reinforcement Learning (RL) with Rubric-Based Rewards

• Algorithm: The team used GRPO (Group Relative Policy Optimization) with an asynchronous, streaming implementation (PipelineRL).
• Reward Design: Instead of binary success/failure signals, they employed rubric-based grading.
  • A custom grader (GPT-OSS-20B) evaluates proofs against a detailed 0–7 point rubric generated by a stronger model (Gemini 3 Pro).
  • The rubric includes checkpoints for partial correctness, zero-credit items for specific failure modes, and deductions.
  • This provides dense, informative feedback, encouraging gradual improvement in reasoning rather than just guessing the final answer.
• Curriculum: The prompt set was filtered to exclude trivial problems (where the base model already passes) and problems where no rollout succeeded, focusing training on the "hard" middle ground.

Stage 3: Reasoning Cache (RC) & Test-Time Scaffolding

• Innovation: To handle the extreme length of proofs (up to 2M tokens) without the high variance of standard long-horizon RL, the authors introduced the Reasoning Cache (RC) framework.
• Mechanism:
  • The reasoning process is decomposed into iterative summarize-and-refine cycles.
  • At each step, the model generates a short reasoning segment, summarizes its progress into a compact "state representation," and conditions the next generation on both the original problem and this summary.
  • Training: The model is trained via RL to optimize this summary-conditioned generation.
  • Inference (Agent Mode): At test time, the model utilizes a scaffold (specifically RSA or DeepSeek Math scaffolds) to perform multiple parallel or sequential rollouts, using the RC mechanism to maintain coherence over millions of tokens.

3. Key Contributions

1. QED-Nano Model: A 4B parameter model that achieves state-of-the-art performance among open models on Olympiad proofs, outperforming larger open models like Nomos-1 (30B) and Qwen3-235B.
2. Open Training Pipeline: The first fully open, reproducible recipe for training a small model on complex theorem proving, including:
   • The FineProofs-SFT and FineProofs-RL datasets.
   • The QED-Nano and QED-Nano-SFT model weights.
   • Full training and evaluation code.
3. Reasoning Cache (RC) Framework: A novel method that allows small models to scale performance with test-time compute by breaking long reasoning trajectories into manageable, optimized cycles.
4. Rubric-Based RL: Demonstrating that dense, rubric-based rewards are superior to binary rewards for training long-form, rigorous reasoning in small models.

4. Results

The model was evaluated on three benchmarks: IMO-ProofBench, ProofBench, and IMO-AnswerBench.

• Performance: QED-Nano (Agent) with the RSA scaffold achieves 56.9% on IMO-ProofBench, surpassing the 30B Nomos-1 and the 120B GPT-OSS, and approaching the performance of the 685B DeepSeek-Math-V2 and the proprietary Gemini 3 Pro.
• Cost Efficiency: QED-Nano achieves these results at a fraction of the inference cost of larger models.
• Qualitative Analysis: Human evaluators noted that QED-Nano produces proofs that are more structured and explicit than other small models. While it sometimes relies on computation-heavy arguments (a "brute force" style), it avoids the circular reasoning and logical gaps common in base models.

5. Significance and Future Directions

• Democratization of Reasoning: The paper proves that model size is not the sole determinant of reasoning capability. Small, specialized models can match or exceed massive general-purpose systems if trained with the right data, rewards, and test-time scaffolds.
• Test-Time Compute Scaling: It validates the hypothesis that scaling test-time compute (via scaffolds like RC and RSA) is a more practical path to solving hard problems than scaling model parameters, especially for open-source ecosystems.
• Reproducibility: By releasing the full pipeline, the authors enable the community to study, improve, and extend these techniques, bridging the gap between proprietary "black box" systems and open research.
• Future Work: The authors suggest improving the synergy between SFT and RL to mitigate "length explosion," refining grader rubrics for stricter rigor, and exploring methods to encourage "aha" moments (novel insights) rather than just computational refinement.

In conclusion, QED-Nano represents a significant leap in open-source mathematical reasoning, demonstrating that a 4B model can be trained to solve IMO-level problems with performance rivaling systems 100x its size.