💬 NLP

ReForm: Reflective Autoformalization with Prospective Bounded Sequence Optimization

ReForm is a reflective autoformalization method that improves the semantic accuracy of translating natural language mathematics into formal statements by integrating iterative self-correction with a novel training technique called Prospective Bounded Sequence Optimization (PBSO).

Original authors: Guoxin Chen, Jing Wu, Xinjie Chen, Wayne Xin Zhao, Ruihua Song, Chengxi Li, Kai Fan, Dayiheng Liu, Minpeng Liao

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Guoxin Chen, Jing Wu, Xinjie Chen, Wayne Xin Zhao, Ruihua Song, Chengxi Li, Kai Fan, Dayiheng Liu, Minpeng Liao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Lost in Translation" Math Dilemma

Imagine you are a world-class mathematician who speaks only English. You have a brilliant idea for a new theorem, but you need to explain it to a super-intelligent robot that only understands a very strict, hyper-logical language called Lean (a formal computer language).

If you tell the robot, "The number is roughly around ten," the robot will crash. It needs to know exactly: "Is it 10.0? Is it 10.00001? Is it an integer?"

Currently, we use AI (Large Language Models) to act as the translator between English and Lean. But there is a huge problem: AI is a "one-shot" translator. It reads the English, writes the Lean code in one go, and hits "send."

Because math is incredibly sensitive, even a tiny slip—like writing "less than" instead of "less than or equal to"—makes the entire translation useless. The AI is like a student who rushes through a math test, finishes in ten seconds, and hands it in without ever checking if they made a silly mistake.

The Solution: REFORM (The "Self-Correcting Scholar")

The researchers created a new method called REFORM. Instead of being a rushed translator, REFORM turns the AI into a Reflective Scholar.

Think of the difference between these two students:

The One-Pass Student: Reads the question, writes the answer, and turns it in immediately. (This is how current AI works).
The REFORM Student: Reads the question, writes a draft, stops to critique their own work, realizes, "Wait, I forgot to mention that $x$ must be a positive number!", erases the mistake, and writes a perfect final version.

How it works (The Three Pillars):

1. The Self-Correction Loop (The "Mirror" Method)
REFORM doesn't just translate; it talks to itself. It follows a loop:

Step A: "Here is my first attempt at the Lean code."
Step B: "Let me look at that code in a mirror. Does it actually match the English question? Hmm, I missed a constraint about the variable $x$ ."
Step C: "Okay, let me fix that and try again."

2. PBSO (The "Fair Grading" System)
Training an AI to think this way is hard. If you just reward the AI for the final answer, it might learn to "cheat" by giving a correct answer but writing a lazy, useless critique in the middle.

The researchers invented PBSO (Prospective Bounded Sequence Optimization). Think of this as a teacher who doesn't just grade the final exam, but also gives points for the quality of the scratchpad work. It rewards the AI for:

Getting the final math right (The Goal).
Writing a smart, helpful critique of its own mistakes (The Process).
This ensures the AI actually learns how to think, rather than just memorizing answers.

3. ConsistencyCheck (The "Expert Audit")
To prove this worked, they didn't just trust the AI. They hired human experts to check the AI's work. They even discovered something shocking: Human experts make mistakes in formal math about 38% of the time! This proves that autoformalization is so hard that even humans need help, making the REFORM method even more important.

The Result: A Massive Leap Forward

The results were like comparing a student who guesses to a student who studies. REFORM improved the accuracy of math translation by an average of 22.6% over the previous best models.

In short, REFORM moves AI from being a fast-talking translator to a careful, thinking mathematician.

Technical Summary: REFORM: Reflective Autoformalization with Prospective Bounded Sequence Optimization

REFORM is a novel framework designed to advance the field of autoformalization—the process of translating natural language (NL) mathematical problems into machine-verifiable formal statements (e.g., in Lean 4).

1. The Problem: The Semantic Fidelity Gap

While current Large Language Models (LLMs) are proficient at generating syntactically correct formal code (code that compiles), they frequently suffer from poor semantic consistency. This means the generated formal statement often fails to capture the true mathematical intent of the original NL problem (e.g., misinterpreting quantifier scopes, overlooking constraints, or misidentifying variable domains).

The authors identify the root cause as the "one-pass generation paradigm." Most existing models attempt to translate the problem in a single forward pass, lacking the iterative self-reflection and error-correction mechanisms that human mathematicians naturally employ.

2. Methodology: The REFORM Framework

The authors propose shifting from a one-pass approach to a Reflective Autoformalization paradigm.

A. Reflective Autoformalization Paradigm
Instead of a single translation, REFORM implements a self-correction loop within a single continuous autoregressive generation:

Autoformalization: The model generates a candidate formal statement $S_t$ .
Semantic Self-Validation: The model generates a detailed critique $C_t$ , assessing whether $S_t$ faithfully captures the semantics of the original question $Q$ .
Iterative Refinement: Based on the critique, the model generates a revised statement $S_{t+1}$ , repeating the process until semantic fidelity is achieved.

B. Prospective Bounded Sequence Optimization (PBSO)
Training a model to perform this iterative loop is difficult because it involves heterogeneous rewards: the model must be rewarded for the final correct answer (task reward) and for providing accurate intermediate critiques (auxiliary reward). Standard Reinforcement Learning (RL) often fails here, as models might learn to provide "superficial" or "hallucinated" critiques to satisfy the training signal.

To solve this, the authors introduce PBSO, which features:

Heterogeneous Reward Mechanism: Assigns $r_{task}$ at the end of the sequence (for correctness) and $r_{aux}$ at intermediate steps (for critique quality).
Prospective Bounded Return: A novel RL algorithm that computes a discounted sum of future rewards for each step but clips (bounds) the returns. This prevents gradient instability caused by unbounded accumulation and ensures that intermediate validation steps are explicitly optimized to facilitate later corrections.

3. Key Contributions

New Paradigm: Reconceptualizes autoformalization as an iterative, reflective process rather than a simple translation task.
New RL Algorithm (PBSO): Enables effective credit assignment for multi-objective, multi-position reward signals in long-sequence generation.
ConsistencyCheck Benchmark: A new benchmark of 859 expert-annotated items used to evaluate the reliability of LLMs as judges for semantic consistency.
Empirical Validation: Demonstrates that the reflective approach significantly outperforms state-of-the-art (SOTA) models.

4. Results and Performance

SOTA Performance: REFORM achieves an average improvement of 22.6 percentage points over the strongest baselines across four benchmarks (miniF2F, ProofNet, Putnam, and AIME2025).
Efficiency vs. Scale: Remarkably, the REFORM-8B model outperforms the much larger Goedel-Formalizer-V2-32B in semantic consistency, proving that the reflective architecture is more impactful than mere parameter scaling.
Closing the Semantic Gap: While baselines show a massive gap between syntactic correctness and semantic consistency, REFORM's improvements are heavily concentrated on semantic fidelity, effectively transforming the model from a pattern matcher into a reasoning agent.
Benchmark Insights: The ConsistencyCheck analysis revealed that even human experts produce semantic errors in up to 38.5% of cases, highlighting the extreme difficulty of the task.

5. Significance

This paper represents a significant step toward Automated Theorem Proving (ATP). By solving the "bottleneck" of autoformalization, REFORM enables more reliable pipelines where natural language math problems can be fed into formal solvers. It demonstrates that process-oriented reinforcement learning (rewarding the way a model thinks and critiques) is superior to simple outcome-oriented learning for complex, high-precision mathematical tasks.