🤖 AI

Lean Atlas: An Integrated Proof Environment for Scalable Human-AI Collaborative Formalization

This paper introduces Lean Atlas, an open-source interactive web tool featuring the Lean Compass algorithm that enables scalable human-AI collaboration in formalizing mathematics by visualizing project dependencies and automatically identifying the specific definitions and propositions requiring human semantic verification to prevent AI-generated proofs from exhibiting semantic hallucinations.

Original authors: Banri Yanahama, Akiyoshi Sannai

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Banri Yanahama, Akiyoshi Sannai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Perfectly Wrong" Proof

Imagine you hire a super-fast, super-smart robot to write a legal contract for you. The robot types out a document that is grammatically perfect. Every sentence follows the rules of English, the punctuation is correct, and the formatting is flawless. A spell-checker would give it a 100% score.

However, the robot made a subtle mistake: it swapped the word "buyer" with "seller" in the most important clause. The document is still "grammatically correct," but it means the exact opposite of what you wanted. In the world of mathematics and computer science, this is called Semantic Hallucination.

In the paper, the authors explain that AI is getting very good at writing formal math proofs (using a tool called Lean). The computer's "spell-checker" (called a Type Checker) can tell if the logic holds together. But it cannot tell if the math actually means what the human intended. The AI might prove a statement that is logically true but mathematically nonsense.

The Solution: The Human-in-the-Loop

The authors propose a new way to work: Humans and AI as a team.

The AI does the heavy lifting: writing the code, checking the logic, and building the structure.
The Human acts as the "Meaning Inspector." They don't check every single line of code (which would take forever); they only check the parts that determine what the math actually says.

But how do you know which parts to check in a massive project with thousands of lines of code? That's where Lean Atlas comes in.

The Tool: Lean Atlas (The "Map")

Think of a large math project like a giant, tangled ball of yarn. Every thread (a definition or a theorem) is connected to many others. If you pull one thread, it might tug on a dozen others.

Lean Atlas is a tool that untangles this yarn and draws a map of the connections.

It visualizes the project as an interactive web.
It distinguishes between two types of connections:
1. The "What" (Type Dependencies): These are the definitions and concepts that give the math its meaning. (e.g., "What is a 'prime number'?")
2. The "How" (Value Dependencies): These are the specific steps taken to prove something is true. (e.g., "Here is the step-by-step calculation proving 2 is prime.")

The Magic Trick: Lean Compass (The "Filter")

This is the paper's most clever idea.

Imagine you are looking at a specific theorem (a specific claim) in the middle of that giant ball of yarn. You want to know: "If I change this definition, does it break the meaning of my theorem?"

Usually, you'd have to look at every single thread connected to it. But Lean Compass is a smart filter. It uses a simple rule:

"If a connection comes from a proof (the 'How'), we can ignore it for now. The computer already checked that the proof is logical. We only need to worry about connections coming from definitions (the 'What')."

The Analogy:
Imagine you are inspecting a house for safety.

The "Proofs" are the workers who built the walls. If the workers did a bad job, the wall might fall, but the blueprint (the design) is still correct. The computer checks the workers.
The "Definitions" are the blueprints. If the blueprint says "This is a window" but the architect meant "This is a door," the house is built wrong, even if the workers followed the blueprint perfectly.

Lean Compass tells you: "Don't waste time checking the workers' steps. Just check the blueprints and the doors."

By filtering out the "proof steps," Lean Compass shrinks the list of things a human needs to check by 90% or more in many cases.

The Results: How Well Does It Work?

The authors tested this on six different projects, ranging from pure math to physics and cryptography.

The "Proof-Heavy" Projects: (Like the Prime Number Theorem). These are like houses with thousands of workers but simple blueprints. Lean Compass worked like a vacuum cleaner, removing 94–99% of the clutter. Humans only had to look at the tiny remaining core.
The "Definition-Heavy" Projects: (Like a complex cryptographic signature). These are like houses with very complex, changing blueprints. Here, the "blueprints" (definitions) are the main event, so the filter couldn't remove as much (only about 27% reduction). But it still helped.
The Mixed Projects: (Like Fermat's Last Theorem). Some parts were easy to filter, others were hard. This proved that the tool adapts to the structure of the project.

The Big Idea: "Aligned Lean Code"

The paper introduces a new quality standard called Aligned Lean Code.

Logical Correctness: The computer says, "This proof is valid."
Semantic Correctness: A human says, "This proof means what we intended."

Aligned Code is code that has passed both tests. The authors suggest that for AI to be truly useful in science, we shouldn't just trust the computer's "Yes." We need a system where humans verify the meaning, and Lean Atlas makes that verification fast and manageable.

Summary

AI is fast but can be "hallucinated" (logically correct but meaninglessly wrong).
Humans need to check the meaning, but checking everything is too slow.
Lean Atlas draws a map of the project.
Lean Compass is a filter that ignores the "proof steps" (which the computer already checked) and highlights only the "definitions" (which determine the meaning).
Result: Humans can verify the meaning of massive AI-generated math projects in a fraction of the time, ensuring the math is actually about what we think it is.

1. Problem Statement: Semantic Hallucination in AI Formalization

While AI-driven autoformalization is rapidly advancing, a critical gap exists between logical correctness and semantic correctness.

The Limitation: Proof assistants (like Lean 4) guarantee that a proof term is logically constructed for a given proposition (type checking). However, they cannot verify if the proposition or its definitions faithfully capture the intended mathematical meaning.
Semantic Hallucination: This occurs when an AI-generated formalization passes the type checker and is provable, yet fails to represent the original mathematical statement. Common patterns include definition mismatches, missing assumptions, goal substitution, and type default shifts.
The Challenge: As AI generates proofs at scale, human review becomes intractable if reviewers must inspect the entire dependency graph of a project. There is a need for a workflow that efficiently isolates the specific declarations requiring human semantic verification.

2. Methodology: Lean Atlas and Lean Compass

The authors propose a human-in-the-loop approach where AI handles logical construction, and human scientists verify semantic correctness. This is realized through two main components:

A. Lean Atlas (The Environment)

Lean Atlas is an interactive web viewer and backend tool for Lean 4 projects.

Dependency Graph Extraction: It traverses the Lean 4 environment to extract constants (theorems, definitions, inductive types, etc.) and their dependencies.
Classification: It classifies dependencies into two distinct categories:
1. Type Dependencies: Relationships where a constant appears in the type signature of another (e.g., a definition used in a theorem's proposition). These are critical for semantic meaning.
2. Value Dependencies: Relationships where a constant appears only in the implementation/value (e.g., the proof term of a theorem).
Visualization: It visualizes the dependency graph as an interactive web viewer, allowing users to filter by edge kind, confidence, and proof status.

B. Lean Compass (The Algorithm)

Lean Compass is the core algorithm designed to reduce the scope of human review.

Core Insight: It exploits an asymmetry in value dependencies:
- Value dependencies originating from a theorem's proof are guaranteed by the type checker (logical correctness) and can be pruned for semantic review.
- Value dependencies originating from a definition's implementation may contain computational content not reflected in the type signature and must be retained.
The Algorithm:
1. Input: A dependency graph $G$ and a target set of theorems $M$ .
2. Pruning Rule: Edges where the source is a theorem and the dependency site is value (i.e., proof-level dependencies) are removed.
3. Reachability: A Breadth-First Search (BFS) is performed on the pruned graph starting from $M$ to find all reachable nodes.
4. Output: A "review cone" containing only project-specific nodes whose semantic correctness could affect the target theorems.
Edge Classification: The system classifies edges into 8 kinds based on Source Kind $\times$ Dependency Site $\times$ Target Kind. Only edges #3 and #4 (theorem $\to$ value $\to$ definition/theorem) are pruned.

3. Key Contributions

Concept of "Aligned Lean Code": The authors define formalization code that has undergone human semantic verification as "aligned." This code possesses both logical correctness (from the type checker) and semantic correctness (from human review), serving as a new quality standard for AI-generated formalizations.
Lean Atlas Tool: A novel tool that visualizes dependency graphs with a specific focus on distinguishing type vs. value dependencies, enabling efficient human oversight.
Lean Compass Algorithm: A scalable method to automatically extract the minimal set of nodes requiring semantic review, significantly reducing the candidate set for human inspection in large-scale projects.

4. Results and Evaluation

The tool was evaluated on six diverse Lean 4 projects with varying structural characteristics. The primary metric was node reduction rate (the percentage of the dependency graph pruned).

Project Type	Project Name	Characteristics	Average Reduction
Proof-Heavy	PrimeNumberTheoremAnd	8 main theorems	94–99%
Proof-Heavy	Carleson	9 main theorems (large scale)	96.2%
Proof-Heavy	Brownian Motion	5 main theorems	94.4%
Mixed	FLT (Fermat's Last Theorem)	6 milestone theorems	59.8%
Mixed	PhysLib	Theoretical Physics (QM, GR, etc.)	69.0%
Definition-Heavy	XMSS	Cryptography (Hash-based signatures)	27.3%

Key Findings:

Structural Determinant: The reduction rate is determined by the theorem-to-definition ratio within the "review cone," not the project size or domain.
High Reduction: In proof-heavy projects, the algorithm successfully prunes massive amounts of proof-level dependencies (e.g., reducing a 227-node cone to 14 nodes in Brownian Motion).
Low Reduction: In definition-heavy projects (like XMSS), value dependencies from definitions must be retained because they carry computational semantics, resulting in lower reduction rates.
Cross-Domain Applicability: The approach proved effective in non-mathematical domains, including theoretical physics (PhysLib) and cryptography (XMSS), demonstrating that semantic hallucination is a general issue in formal verification.

5. Significance

Scalability: Lean Compass makes human-in-the-loop verification feasible for massive AI-generated formalizations by filtering out logically verified but semantically irrelevant code.
Quality Standard: The concept of "aligned Lean code" provides a concrete target for AI-assisted mathematics: code that is not just provable, but meaningful.
Domain Agnosticism: By addressing the gap between type-checker validity and intended semantics, this framework applies to any domain using proof assistants, including cryptography and physics, where domain-specific definitions may be unfamiliar to general proof assistants.
Future Infrastructure: The paper lays the groundwork for accumulating a corpus of "aligned" code, which can serve as high-quality training data for future AI models and a trusted reference for mathematicians.

Limitations & Future Work:
Current evaluation relies on node reduction as a proxy for time savings; actual human verification time reduction has not yet been measured. Future work includes automating confidence metadata generation via AI and expanding evaluations to broader scientific domains.