Does the Proof Prove It That Way? Faithful Formalization of Elements Proofs
This paper introduces Pistis, an agentic, oracle-guided system featuring a novel "OrderDecompose" search strategy that generates faithful formal Lean proofs for Euclid's Elements by rigorously aligning natural language reasoning with formal tactics, thereby outperforming prior baselines in speed, success rate, and human/LLM preference while effectively identifying gaps in mathematical arguments.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Mathematics has always relied on two distinct languages. One is the natural language we use to explain ideas, tell stories, and share insights; it is flexible, rich in context, and often skips steps that seem obvious to a human reader. The other is the formal language of proof assistants, rigid computer systems that check every single logical move to ensure a conclusion is undeniably true. For decades, researchers have worked on translating the first language into the second, a process called autoformalization. The goal was simple: take a human-written proof and turn it into code a computer can verify. But a critical problem remained. A computer could often produce a proof that was technically correct, yet it bore no resemblance to the human argument that inspired it. The computer might solve the problem using a completely different path, hiding the original reasoning behind a wall of automated shortcuts. This created a gap between truth and understanding. If the computer's proof does not follow the human's steps, we cannot use it to check if the human's reasoning was actually sound, nor can we trust it to teach us how the argument works.
A team of researchers has now tackled this gap with a new system designed to keep the computer's proof faithful to the human's original thought process. They call their system Pistis, a name drawn from the ancient Greek word for faith or trust. The researchers applied this system to the first three books of Euclid's Elements, a foundational geometry text written over two thousand years ago. Their work demonstrates that it is possible to translate these ancient arguments into a modern computer language without losing the original logic, while also uncovering hidden errors in the text that had gone unnoticed for centuries.
The core challenge the team faced was that natural language and computer logic operate on different rhythms. A human proof might say, "Let us assume this is true," and move on, expecting the reader to fill in the gap. A computer, however, demands that every single step be explicitly stated and justified. Previous attempts to translate proofs often let the computer fill in these gaps with its own logic, effectively rewriting the argument to make it easier for the machine to solve. The result was a proof that compiled successfully but failed to reflect the human author's intent. Pistis was built to prevent this. Instead of asking the computer to find any possible way to prove the statement, the system forces the computer to follow the human's specific path, sentence by sentence.
To achieve this, the researchers developed a method that breaks the translation process into two distinct phases. First, a mapping phase analyzes the natural language text and splits it into a sequence of small, atomic steps. It identifies exactly what each sentence claims and what assumptions it relies on. This creates a strict template that the computer must follow. Second, a filling phase attempts to prove each of these small steps individually. The system uses a specialized search strategy that prevents the computer from taking shortcuts or skipping ahead. If the computer cannot prove a specific step using the exact tools and references the human text mentions, it does not simply find a different way to solve the problem. Instead, it flags the issue, revealing that the original human argument might have a hole or a missing piece.
This approach proved remarkably effective when tested on Euclid's geometry. The researchers generated formal proofs for 92 propositions across the first three books. When they compared these new proofs to previous attempts, the difference was stark. The new proofs compiled more than thirty-three times faster, a significant improvement in efficiency that suggests the new method avoids the heavy, slow computations of older systems. More importantly, human experts who reviewed the proofs preferred the new system's output by a large margin. In a blinded study, reviewers found the new proofs to be far more transparent and better representations of the original textbook arguments. An artificial intelligence judge, trained to evaluate the quality of the translation, agreed, favoring the new proofs over the old ones by a ratio of more than five to one.
Beyond simply translating the text, the system acted as a rigorous checker that exposed genuine flaws in the source material. Because the system insists on following the human's logic exactly, it cannot hide errors. In one instance, the system identified a citation error in a modern translation of Euclid. The text referenced a proposition about cutting a line in half, but the citation pointed to a proposition about cutting an angle. The system flagged this mismatch, showing that the translation had linked the wrong idea to the wrong step. In another case, the system found a gap in Euclid's own reasoning where a specific scenario was left unaddressed. The researchers were able to formally demonstrate that the original argument was incomplete, a discovery that would have been difficult to make without a tool that strictly adheres to the original structure.
The success of Pistis suggests that faithful formalization is not just a technical exercise but a powerful tool for verifying human knowledge. By forcing the computer to walk the same path as the human, the system can confirm whether the reasoning holds up or where it breaks down. The researchers found that their method could accept valid arguments, refute invalid ones, and pinpoint exactly where a proof went wrong. This capability extends beyond ancient geometry; the framework is designed to work with any mathematical argument written in natural language. The work shows that we do not have to choose between the flexibility of human explanation and the rigor of machine verification. It is possible to have both, provided the machine is guided to respect the human's original voice and logic.
The study also highlighted the limits of current technology. While the system worked well for the first three books of Euclid, it could not handle every single proposition in the later books without additional human guidance. Some propositions required geometric concepts that the underlying computer system did not yet know how to handle, such as measuring the length of a curved line. The researchers noted that their system relies on a human or an advanced artificial intelligence to act as an oracle, verifying that the initial mapping of the text is correct. This means the process is not fully automatic yet, but it significantly reduces the amount of manual work required compared to previous methods.
Ultimately, the paper presents a new standard for how we interact with mathematical proofs in the age of artificial intelligence. It moves beyond the question of whether a computer can prove a theorem to the more profound question of whether the computer understands the argument. By ensuring that the formal proof mirrors the natural language argument step for step, the researchers have created a tool that can validate the reasoning behind a conclusion, not just the conclusion itself. This allows mathematicians and students to trust that the computer is not just finding a solution, but is truly following the logic of the person who wrote the proof. The work offers a path forward where human insight and machine precision work together, preserving the integrity of mathematical discovery while leveraging the power of modern verification.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.