Towards Automated Formal Verification of zkEVMs Using LLM-Guided Constraint Synthesis
The paper introduces VeriSynth, a novel framework that leverages a hybrid LLM-SMT pipeline to automatically synthesize executable verification models from Rust zkEVM code, achieving over 90% bug detection rates by overcoming the limitations of manual specification and LLM hallucinations.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the internet as a giant, global digital ledger where everyone can see every transaction, but no single person controls it. This is the world of blockchain, and its most famous resident is Ethereum. To make this system faster and cheaper, developers built "Layer-2" networks that do the heavy lifting off-screen, only showing the final results to the main network. To prove these off-screen results are honest without revealing the secret details, they use a magical trick called a "Zero-Knowledge Proof." Think of it like a wizard showing you a locked box and proving, without opening it, that the treasure inside is exactly what they promised.
But here's the catch: for the magic to work, the wizard's spellbook (the computer code) must be perfect. If the wizard makes a tiny mistake in the spell—like counting the wrong amount of gold or forgetting to update the map—the box might still look locked and valid, even if it's empty or full of fake coins. This is a nightmare for security. To stop this, experts use "formal verification," a method where a super-strict robot mathematician checks the spellbook line-by-line to ensure the logic is flawless. However, writing the instructions for this robot is incredibly hard, like trying to translate a complex video game manual written in a secret language into a math textbook that the robot can read.
This is where a new tool called VeriSynth steps in. The researchers behind it realized that while Artificial Intelligence (AI) is great at writing code, it's also prone to "hallucinating"—making up facts that sound real but are wrong. So, they didn't just ask the AI to find the bugs. Instead, they built a team where the AI acts as a creative translator, turning the messy, secret code into a draft for the robot mathematician, and the robot acts as the strict judge that says, "Yes, this is correct," or "No, this is nonsense, try again."
The Problem: The Silent Bug
In the world of Zero-Knowledge Ethereum Virtual Machines (zkEVMs), the code that runs the show is written in a language called Rust. It's powerful but tricky. Sometimes, a developer might accidentally write a line of code that counts gas (the fee for a transaction) wrong, or forgets to update a memory slot. If this happens, the system might generate a "valid" proof for a transaction that is actually broken. It's like a bank teller who accidentally gives you $100 instead of $10, but the receipt says everything is perfect. Because the proof looks valid, the mistake goes unnoticed, silently breaking the security of the whole system.
The usual way to fix this is for humans to write detailed test cases, hoping they catch every possible mistake. But humans get tired, and they can't imagine every weird scenario a computer might encounter. Another approach is to use Large Language Models (LLMs)—the same kind of AI that writes essays or code—to find the bugs. But the paper argues that asking an AI to just "look for bugs" is unreliable. The AI might guess the bug is there when it isn't, or miss it entirely, because it lacks the strict, mathematical certainty needed for security.
The Solution: A Team of Translator and Judge
The authors of this paper, Shichen Huang and his team, created a framework called VeriSynth. They treat the problem like a high-stakes game of "Telephone" where the message must be perfect.
- The Translator (The LLM): First, the system breaks the complex Rust code into small, manageable chunks, like separating a giant puzzle into individual pieces. It then asks an AI to translate each piece into a "verification model." This isn't just a description; it's a set of strict mathematical rules (written in a language called Python/Z3) that tell the robot mathematician exactly how the code should behave.
- The Reference Library: To stop the AI from making things up, VeriSynth gives it a "cheat sheet" of previously verified examples. It's like showing the AI a picture of a correctly assembled Lego set before asking it to build a new one, so it knows what the pieces should look like.
- The Judge (The SMT Solver): This is the most important part. The AI's translation is never trusted immediately. It is handed to a super-strict robot mathematician (an SMT solver). The robot checks the translation against the rules. If the translation has a syntax error, a missing piece, or a logical contradiction, the robot rejects it.
- The Auto-Repair: If the robot rejects the translation, it doesn't just say "fail." It sends a specific error message back to the AI: "You used the wrong size for this number," or "You forgot to update the memory." The AI then tries to fix the mistake and sends it back. This loop continues until the robot is satisfied.
What They Found
The team tested VeriSynth on a custom "buggy" dataset they built, containing 95 different examples of code with hidden mistakes. These mistakes ranged from simple math errors to complex issues with how the system handles calls between different programs.
The results were striking:
- VeriSynth found 87 out of 95 bugs, achieving a detection rate of 91.6%.
- In contrast, if they just asked the AI to find the bugs without the strict robot judge, it only found 44 bugs (46.3%).
- Even if they let the AI chat with the robot and try again (a "conversational" approach), it only found 58 bugs (61.1%).
The paper also compared VeriSynth to the real-world testing methods used by a major project called Scroll. Scroll's team of human experts had written their own set of tests to catch bugs. Their tests caught 55 of the 95 bugs. VeriSynth caught 87, meaning it found 32 additional bugs that the human-written tests missed.
Why It Matters
The researchers emphasize that this isn't about replacing human developers or saying AI is perfect. Instead, it's about using AI as a powerful tool to do the heavy lifting of translation, while keeping a strict, unblinking robot as the final authority. The "auto-repair" feature was the most critical part; without it, the system's success rate dropped to 66.3%, proving that the ability to fix its own mistakes is what makes the system work.
The study suggests that by combining the creativity of AI with the strict logic of mathematical solvers, we can build a safety net for the future of blockchain that is far stronger than what humans can build alone. It's a reminder that in the world of digital security, the best defense is a team where the creative mind proposes, and the strict logic decides.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.