💬 NLP

Monotonic Reference-Free Refinement for Autoformalization

This paper introduces a reference-free, iterative monotonic refinement framework for full-theorem autoformalization that leverages complementary feedback from theorem provers and LLM judges to simultaneously optimize formal validity, logical preservation, mathematical consistency, and formal quality, achieving state-of-the-art performance on miniF2F and ProofNet benchmarks without ground-truth data or human intervention.

Original authors: Lan Zhang, Marco Valentino, André Freitas

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lan Zhang, Marco Valentino, André Freitas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to translate a complex story written in a casual, everyday language (like a blog post about math) into a strict, computer-readable language (like a programming code for a robot mathematician). This process is called autoformalization.

The problem is that while computers are great at checking if code is "syntactically correct" (does it have the right punctuation?), they struggle to understand if the story still makes sense or if the logic holds up. Existing methods often fix the grammar but lose the meaning, or they get the meaning right but the code crashes.

This paper introduces a new method called Monotonic Reference-Free Refinement. Here is how it works, using simple analogies:

1. The Goal: A Perfect Translation

The authors want to create a translation that is perfect in four ways:

Formal Validity (The "Syntax Check"): The code must run without errors. If it doesn't, the robot rejects it immediately.
Logical Preservation (The "Plot Check"): The translation must keep the original story's logic. You can't change the ending just because it's easier to write.
Mathematical Consistency (The "Fact Check"): All the numbers, variables, and rules must match the original story exactly.
Formal Quality (The "Style Check"): The code should be clean, concise, and easy for humans to read later.

2. The Problem: One Tool Can't Do It All

Usually, researchers use one AI model to do the whole job. But it's like asking a single person to be a grammarian, a logician, a fact-checker, and an editor all at once. They might be great at grammar but terrible at logic. Also, if the first attempt is wrong, fixing it usually requires a "gold standard" answer (the correct code) to compare against. The authors wanted a method that works without having the answer key.

3. The Solution: A Specialized Assembly Line

The authors built a system that acts like a specialized factory with different workers, each doing what they are best at. They don't need the answer key; they just need to keep improving the draft until it's perfect.

Here are the three types of "workers" (AI models) in their factory:

The "First Draft" Writers (One-Off Generators): These are specialized math AIs that take the raw story and write the very first version of the code. They are good at getting the structure right.
The "Syntax Fixers" (FV-Repairers): If the First Draft has code errors (the robot rejects it), these workers step in. They are experts at fixing broken code to make it run, ensuring the "Formal Validity" score goes up.
The "Refiners" (Recurrent Generators): Once the code runs, these workers look at the draft and try to make it better. They don't just fix errors; they improve the logic, the facts, and the style. They get feedback from "Judges" (other AIs) that say, "This part is logically weak," or "This is too wordy."

4. The "Monotonic" Rule: Never Step Backward

The most important part of this system is the Acceptance Policy. Imagine you are climbing a mountain.

In many AI systems, you might take a step up, then a step down, then up again, hoping to find the peak.
In this system, the rule is Monotonic: You only accept a new version of the code if it is strictly better (or at least not worse) than the previous one.

If a new draft is slightly better at logic but slightly worse at style, the system checks a "safety buffer" (a mathematical guarantee called a Lower Confidence Bound). It only accepts the change if it is confident that the overall quality has improved. This ensures the process never gets stuck in a loop of getting worse and worse.

5. The Result: A Self-Improving Loop

The system runs in a loop:

Generate a draft.
Check if it runs (Validity). If not, send it to the Syntax Fixer.
If it runs, send it to the Refiners to improve logic and style.
Compare the new version to the old one using the "Safety Buffer."
If the new one is certified as better, keep it. If not, keep the old one and try a different approach.

The Outcome:
The authors tested this on two difficult math benchmarks (miniF2F and ProofNet).

On the easier benchmark, they achieved 100% validity (the code always runs) and a very high overall quality score.
On the harder benchmark, they still achieved high validity and significantly better overall scores than previous methods.

In Summary:
This paper presents a "team-based" approach to translating math into code. Instead of relying on one super-AI, it uses a team of specialized AIs working in a loop, with a strict rule that every step must be an improvement. This allows them to create high-quality, error-free mathematical proofs without needing to see the correct answers beforehand.

Technical Summary: Monotonic Reference-Free Refinement for Autoformalization

1. Problem Statement

While Large Language Models (LLMs) have advanced mathematical reasoning, full-theorem autoformalization—the conversion of natural-language theorems and their accompanying proofs into complete, mechanically verifiable formal statements and proofs—remains underexplored. Existing approaches often focus on statement formalization alone, relying on "sorry" placeholders for proofs, or treat statement generation and theorem proving as isolated tasks.

The core challenge lies in the multi-dimensional nature of high-quality autoformalization. A successful formalization must simultaneously satisfy:

Formal Validity (FV): Syntactic correctness and mechanical verification by a theorem prover (e.g., Lean4).
Logical Preservation (LP): Semantic alignment with the logical structure of the original natural-language statement.
Mathematical Consistency (MC): Accurate representation of mathematical objects and operations.
Formal Quality (FQ): Structural clarity, conciseness, and reusability.

Current iterative refinement methods struggle to jointly optimize these dimensions, often improving one aspect (e.g., syntax) at the expense of others (e.g., semantic alignment). Furthermore, existing methods often rely on ground-truth formalizations or human intervention for selection, limiting their applicability in reference-free settings.

2. Methodology

The authors propose a reference-free, iterative, monotonic optimization process that operates at test time without access to ground-truth formalizations or human selection. The framework integrates feedback from theorem provers and LLM-based judges to guide a ensemble of specialized LLMs.

2.1 Objective Function

The authors define a masked composite objective $J_{OA}(x)$ to balance the four dimensions. Formal Validity ( $\pi_{FV}$ ) acts as a hard constraint (a mask), while Logical Preservation ( $\pi_{LP}$ ), Mathematical Consistency ( $\pi_{MC}$ ), and Formal Quality ( $\pi_{FQ}$ ) are treated as soft dimensions:
$J_{OA}(x) = \frac{1}{3}\pi_{FV}(x) \left( \pi_{LP}(x) + \pi_{MC}(x) + \pi_{FQ}(x) \right)$
This formulation ensures that invalid formalizations (where $\pi_{FV}=0$ ) are not selected, regardless of their soft scores.

2.2 LLM Judges and Uncertainty

Since soft dimensions are intractable to compute exactly, the framework employs LLM judges to estimate scores. To handle the inherent noise and bias of LLMs, the authors assume:

Asymptotic Stability: Judgments stabilize after calibration.
Measurable Uncertainty: A one-sided Lower Confidence Bound (LCB) can be established for the true scores.
The objective is optimized using an LCB-based estimator, ensuring that selection decisions are conservative and robust against evaluation noise.

2.3 Generator Roles and Responsiveness

The framework decomposes LLM roles into three categories, guided by a responsiveness map that models expected improvements per dimension:

One-Off Generators (OOGs): Generate initial formalizations from scratch (e.g., specialized math models like DeepSeek-Prover).
FV-Repairers (FVRs): Specifically target and correct formally invalid candidates (e.g., models capable of iterative proof revision).
Recurrent Generators (REGs): Refine the current best candidate based on feedback from LLM judges regarding specific soft dimensions (LP, MC, or FQ).

2.4 Monotonic Acceptance Policy

The process iterates through candidate generation and selection. A Conservative Acceptance Policy is applied: a new candidate is accepted only if its estimated objective score (based on the LCB) is strictly higher than the current best. This guarantees certified monotonic improvement, meaning the quality of the selected formalization never decreases across iterations. The process is proven to converge and terminate under specific conditions.

3. Key Contributions

Formal Definition and Framework: The paper is the first to formally define full-theorem autoformalization as a unified task and propose a reference-free, iterative monotonic process to solve it.
Masked Composite Objective: Introduction of an objective function that enforces Formal Validity as a hard constraint while optimizing multiple soft dimensions (LP, MC, FQ) simultaneously.
Role-Specialized Ensemble: Decomposition of the autoformalization task into OOG, FVR, and REG roles, utilizing a responsiveness map to select generators that are most effective for specific quality dimensions.
Theoretical Guarantees: Provision of theoretical proofs for certified monotonicity, convergence, and process terminability, ensuring non-decreasing progress even under realistic judge noise.
Empirical Validation: Demonstration that the framework effectively ensembles the strengths of specialized and general-purpose LLMs to achieve high-quality results on standard benchmarks.

4. Experimental Results

The method was evaluated on miniF2F-Test and ProofNet-Test using Lean4. The framework employed an ensemble of models including DeepSeek-Prover-V2-7B, Goedel-Prover-V2-8B, Seed-Coder-8B, Qwen2.5-7B, and GPT-4.1-mini (as judges).

Key Performance Metrics:

miniF2F: Achieved 100.00% Formal Validity and an overall score of 90.27%. The rate of "perfect" formalizations (all dimensions scoring 1) reached 49.18%.
ProofNet: Achieved 77.96% Formal Validity and an overall score of 52.45%, with 17.74% perfect formalizations.

Comparative Findings:

Monotonic vs. Baselines: The monotonic process significantly outperformed both Best-of-k (sampling $k$ outputs) and Iterative Self-Refinement (ISR) without acceptance policies.
Ensemble vs. Single Model: While a single strong model (GPT-5.4) showed improvement, the ensemble of smaller, specialized models assigned to specific roles achieved superior performance, particularly in Formal Validity.
Robustness: The monotonic trend persisted even when evaluated by different LLM judge families (e.g., Qwen vs. GPT) and human experts, indicating the improvements are not artifacts of judge bias.

5. Significance and Claims

The paper claims that this work offers a practical and theoretically grounded step toward reliable autoformalization. By operating in a reference-free manner, the approach is applicable even in domains lacking extensive formal libraries, potentially aiding the formalization of underexplored areas of mathematics and science.

The authors emphasize that the framework provides a principled methodology for AI-assisted autoformalization that:

Reduces the need for human oversight in large-scale formal verification pipelines.
Explicitly quantifies and controls uncertainty in judgment through LCB assumptions.
Demonstrates that combining complementary LLMs with formal verifier feedback can produce formalizations with provably non-decreasing quality.

The work concludes that while computational costs are higher than single-pass inference, the trade-off is justified by the significant gains in performance and stability, paving the way for future research in safe, reliable AI-assisted formal reasoning.