💬 NLP

Reliable Fine-Grained Evaluation of Natural Language Math Proofs

This paper introduces ProofBench, the first expert-annotated dataset for fine-grained evaluation of LLM-generated math proofs, and leverages it to develop ProofGrader, a high-accuracy evaluator that significantly outperforms naive baselines and effectively bridges the gap between automated and human assessment in mathematical reasoning.

Original authors: Wenjie Ma, Andrei Cojocaru, Neel Kolhe, Bradley Louie, Robin Said Sharif, Haihan Zhang, Vincent Zhuang, Matei Zaharia, Sewon Min

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenjie Ma, Andrei Cojocaru, Neel Kolhe, Bradley Louie, Robin Said Sharif, Haihan Zhang, Vincent Zhuang, Matei Zaharia, Sewon Min

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher grading a stack of essays written by a very smart, but sometimes overconfident, robot. The robot is trying to solve difficult math problems, like those found in the International Math Olympiad.

In the past, if the robot got the final number right, you'd give it an A. But math proofs aren't just about the answer; they are about the journey. A robot might get the right answer by accident, or it might take a wrong turn halfway through but still stumble onto the solution. If you only look at the final answer, you miss the mistakes in the middle.

This paper introduces a new way to grade these robot math proofs that is fair, detailed, and incredibly accurate. Here is the story of how they did it, broken down into simple parts.

1. The Problem: The "Black Box" of Math Grading

Currently, when we test AI on math, we mostly check if the final answer is correct (like checking if a multiple-choice bubble is filled in correctly). But for complex proofs, the answer isn't enough.

The Issue: If an AI writes a proof that looks fancy but has a hidden logical trap, a simple "Right/Wrong" check might miss it.
The Gap: We didn't have a reliable "teacher" AI that could read a proof, understand the subtle steps, and give it a score out of 7 (like a real human judge at a math competition) instead of just a "Pass/Fail."

2. The Solution: Building "ProofBench" (The Training Ground)

To teach an AI how to be a good grader, you need a massive library of examples graded by real experts.

The Dataset: The authors created ProofBench. Imagine a library containing 145 of the hardest math problems from competitions like the USAMO and IMO.
The Content: They asked top-tier AI models (like Gemini and DeepSeek) to write solutions to these problems.
The Human Touch: A team of human experts (people who have actually competed in these math olympiads) read every single AI solution. They didn't just say "Good" or "Bad." They used a 0-to-7 scoring scale, awarding points for every correct step and deducting points for every logical error.
The Result: A "Gold Standard" dataset where every AI proof has a precise, human-verified score.

3. The Experiment: Finding the Perfect "Grader"

Now that they had the "answer key" (the human scores), they wanted to build an AI that could mimic a human grader. They tested different "recipes" for this grader AI, similar to how a chef tests different ingredients:

The Chef (The Backbone Model): They tried different AI models to do the grading. They found that the smarter the grader AI is, the better it does.
The Recipe Book (Context):
- No Recipe: Just asking the AI to grade. (Result: The AI guessed wildly).
- The Answer Key: Showing the AI the correct solution. (Better, but the AI still missed nuances).
- The Rubric (The Secret Sauce): Giving the AI a detailed checklist (a "marking scheme") that says exactly what points to look for. This was the game-changer. It was like giving the grader a map of exactly where the treasure (points) is hidden.
The Voting System (Ensembling): Instead of asking one AI to grade a proof, they asked the same AI to grade it five times and took the average. This smoothed out the "bad days" the AI might have, making the score much more stable.

4. The Star Player: "ProofGrader"

By combining the smartest AI model, the detailed rubric (checklist), and the voting system, they created ProofGrader.

How good is it? It is shockingly close to a human expert.
- If a human gives a proof a 5, ProofGrader usually gives it a 4 or 5.
- The "average error" is less than 1 point on a 7-point scale.
- It is far better than simple "Right/Wrong" checkers, which often fail to distinguish between a "mostly correct" proof and a "perfect" one.

5. Why Does This Matter? (The "Best-of-N" Test)

The ultimate test of a grader is: Can it help the robot learn?
The authors ran a simulation where the AI generated 16 different proofs for a problem. They asked the grader to pick the best one.

The Binary Grader (Pass/Fail): Picked a proof that was only about 2.5/7 good. It couldn't tell the difference between a "B" and an "A."
ProofGrader: Picked a proof that was 4.14/7.
The Human Oracle: The best possible human pick was 4.62/7.

The Analogy: Imagine you are a coach picking players for a team.

The Binary Grader is like a coach who only checks if a player can run. They pick a slow runner because they can't see who is fastest.
ProofGrader is like a coach who watches the whole game, sees the strategy, and picks the player who is actually the best athlete, getting 78% of the way to the perfect choice.

Summary

This paper is about building a super-teacher AI for math proofs.

They built a massive library of expert-graded math problems (ProofBench).
They discovered that to grade well, an AI needs a detailed checklist (rubric) and needs to vote on its own answer multiple times.
The result, ProofGrader, is so good it can almost replace human experts in selecting the best math proofs.

This is a huge step forward because it means we can now train AI to write better math proofs automatically, using this "super-teacher" to give them the right feedback, rather than just telling them if they got the final number right or wrong.

1. Problem Statement

While Large Language Models (LLMs) have achieved significant success in mathematical reasoning tasks with verifiable final answers (e.g., GSM8K, MATH), generating and verifying natural language mathematical proofs remains a critical bottleneck.

The Gap: Many proof problems (e.g., USAMO, IMO) do not have a single, easily checkable final answer. Even when an answer exists, verifying the final number is insufficient to assess the validity of the reasoning, which may contain subtle intermediate errors.
The Challenge: Existing evaluation methods are either too coarse (binary correct/incorrect), too slow/costly (human expert grading), or too brittle (formal verification via Lean/Isabelle requires translation from natural language, which is error-prone).
Goal: Develop a reliable, automated, fine-grained evaluator that assigns a score (0–7) to LLM-generated proofs, matching human expert alignment to serve as a reward signal for training and a selection mechanism for inference.

2. Methodology

The authors propose a systematic framework comprising three main components: a new dataset, a systematic evaluation of design factors, and a final optimized evaluator.

A. PROOFBENCH: The Dataset

To enable rigorous evaluation, the authors constructed PROOFBENCH, the first expert-annotated dataset for fine-grained proof ratings.

Scope: 145 problems from six major competitions (USAMO, IMO, Putnam, EGMO, APMO, USA TST) spanning 2022–2025.
Generators: 435 solutions generated by state-of-the-art models: OpenAI o3, Gemini-2.5-Pro, and DeepSeek-R1.
Annotation Process:
1. Marking Scheme Generation: An LLM (Gemini-2.5-Pro) generates a problem-specific rubric (checkpoints, zero-credit items, deductions) based on reference solutions. Experts validated ~85% of these as high quality.
2. Expert Grading: Five experts with Olympiad-level experience scored each solution on a 0–7 scale using the generated rubrics.
3. Calibration: A two-stage process ensured consistency, with ~41% of solutions double-graded (87.5% agreement within 1 point).

B. Systematic Study of Evaluator Design

Using PROOFBENCH as a testbed, the authors explored the design space of LLM-based evaluators across four axes:

Backbone Model: Tested 6 models (O3, GPT-5, Gemini-2.5-Pro, O4-MINI, R1, GPT-4O).
Context: Compared four input configurations:
- REF+MS: Reference solution + Marking Scheme.
- MS: Marking Scheme only.
- REF: Reference solution only.
- NONE: No context (naive baseline).
Instruction: Tested instruction styles (NORM: flexible; STRICT: rigid adherence; BASIC: minimal guidance).
Workflow: Compared single-pass evaluation against Ensembling (averaging multiple runs) and Staged Workflows (Binary $\to$ Fine-grained; Evaluate $\to$ Reflect $\to$ Verdict).

C. PROOFGRADER

Based on the analysis, the authors introduced PROOFGRADER, the optimal configuration:

Backbone: OpenAI o3 (strongest reasoning capability).
Context: REF+MS (Reference solution + Marking Scheme).
Instruction: NORM (Flexible application of the rubric, allowing for valid alternative approaches).
Technique: Ensembling (Median aggregation of 5 independent evaluation runs).

3. Key Results

A. Evaluator Performance (Offline Metrics)

PROOFGRADER achieved a Mean Absolute Error (MAE) of 0.926 against expert scores, significantly outperforming naive baselines (MAE > 1.6).
Key Findings on Design Factors:
- Backbone > Context > Instruction: The strength of the underlying LLM is the most critical factor.
- Context is Vital: Providing a Marking Scheme (MS) or Reference Solution (REF) drastically reduces over-crediting. Without context, evaluators over-score low-quality proofs by an average of 1.7 points.
- Ensembling: Reduces variance and improves robustness. Median aggregation yielded the lowest MAE.
- Staged Workflows: Decomposing the task (e.g., Binary $\to$ Fine-grained) helped weaker models (O4-MINI) but degraded the performance of the strongest model (O3).

B. Downstream Utility (Best-of-N Selection)

The authors tested the evaluators in a Best-of-N (BoN) selection task (selecting the best proof from $N=16$ candidates) to simulate a reward model.

Performance: PROOFGRADER achieved an average human score of 4.14/7 at $N=16$ .
Gap Closure: This closed 78% of the performance gap between a naive binary evaluator (2.48) and the Human Oracle (4.62).
Comparison: PROOFGRADER outperformed computationally expensive pairwise comparison methods (Tournament/Knockout selection) while being significantly more efficient.

C. Failure Mode Analysis

Over-crediting: Occurs when proofs mimic the rubric structure or use sophisticated frameworks but contain fatal logical errors (e.g., false universal claims).
Under-crediting: Occurs when early missteps in an otherwise correct proof are penalized heavily, or when valid alternative paths are not recognized.
Contamination: Models showed higher performance on problems released before their knowledge cutoff, highlighting the need for fresh benchmarks.

4. Key Contributions

PROOFBENCH: A comprehensive, expert-annotated dataset of 435 fine-grained proof ratings across diverse competitions and models, addressing the lack of high-quality evaluation data for natural language proofs.
Systematic Design Study: The first empirical investigation into the design space of proof evaluators, establishing that strong backbones + rich context (rubrics) + ensembling is the optimal recipe.
PROOFGRADER: A high-fidelity evaluator that nearly matches human oracle performance in downstream selection tasks, demonstrating that fine-grained scoring is essential for effective reward modeling in mathematical reasoning.

5. Significance

This work bridges a critical gap in the AI for Science and Education domains. By proving that fine-grained, rubric-driven evaluation can reliably replace or augment human grading, the paper provides a scalable pathway for:

Training: Providing accurate reward signals for Reinforcement Learning (RL) to improve proof generation capabilities.
Inference: Enabling efficient Best-of-N selection to boost the quality of generated proofs without expensive pairwise comparisons.
Benchmarking: Offering a robust standard to measure progress in LLMs on complex, open-ended mathematical reasoning tasks beyond simple answer verification.

The paper concludes that while current open-source models lag behind proprietary ones in this specific task, the methodology is model-agnostic, and the framework (PROOFBENCH) serves as a foundation for future improvements in automated mathematical reasoning evaluation.