← Latest papers
🤖 AI

P3^{3}: Joint Program-and-Proof Planning for Verified Code Generation

The paper introduces P3P^3, an LLM-based agentic workflow that jointly plans programs and their formal proofs to overcome the inefficiencies of sequential generation, achieving state-of-the-art performance and significant cost reductions on verified code generation benchmarks including a new repository-derived dataset called Lean4Commit0.

Original authors: Zenan Li, Ziran Yang, Peiyang Song, Zhaoyu Li, Kaiyu Yang

Published 2026-08-11
📖 7 min read🧠 Deep dive

Original authors: Zenan Li, Ziran Yang, Peiyang Song, Zhaoyu Li, Kaiyu Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a super-smart robot to write a story. You give the robot a prompt, and it spits out a tale. But here's the catch: you don't just want a story; you want a story that is mathematically guaranteed to be true, with no plot holes, no magic that breaks the laws of physics, and no characters who vanish without explanation. This is the world of verified code generation. It's a corner of computer science where we ask Artificial Intelligence not just to write software, but to write software that comes with its own "proof of correctness"—a mathematical certificate that says, "I promise this code will do exactly what I said it would, for every possible situation."

For a long time, the standard way to do this was like a two-step dance: first, the robot writes the code (the story), and then, a separate team of robot-proof-readers tries to check if the story makes sense. If the proof-readers find a plot hole, they send the story back to the writer to fix it. The writer patches the story, sends it back, and the cycle repeats. But this paper suggests that this "write-then-check" dance is often clumsy and inefficient. It's like trying to build a bridge and then, after it's finished, realizing you forgot to put in the support beams, forcing you to tear it down and rebuild it. The authors of this paper propose a new way: instead of writing the code and then the proof separately, the robot should plan the entire bridge—both the road and the supports—at the same time, ensuring they fit together perfectly from the very first sketch.


The Problem: The "Write-Then-Check" Trap

The paper, titled "Joint Program-and-Proof Planning for Verified Code Generation," tackles a frustrating bottleneck in how AI writes verified software. Currently, most systems follow a "program-then-proof" workflow. It's like asking a chef to cook a complex meal and then, after the food is on the table, asking a food critic to prove that the ingredients were fresh and the cooking method was safe. If the critic finds a problem (like undercooked chicken), the chef has to go back, re-cook the dish, and hope the critic likes it this time.

The authors argue that this sequential approach is flawed. When the AI commits to writing the code first, it might choose a structure that looks fine on the surface but is a nightmare to prove. For example, imagine the AI writes a program to find the biggest number in a list. It might choose a method that is short and snappy to write but requires a incredibly complex, hidden mathematical rule to prove it works. Once the code is written, the AI is stuck: it has to either invent a super-hard proof to match that specific code or tear up the code and start over. This leads to a lot of wasted time, money, and "repair loops" where the AI keeps patching the code and the proof, but they never quite fit together.

The Solution: P3 (The "Hand-in-Hand" Planner

To fix this, the researchers introduce P3, a new workflow where the AI acts like a master architect who draws the blueprint for both the building and the safety inspection before laying a single brick.

Instead of jumping straight into writing code, P3 first creates a unified plan. This plan is a high-level sketch that answers two questions at once:

  1. How will the code work? (The "Program Sketch")
  2. How will we prove it works? (The "Proof Sketch")

The plan decides on the structure of the solution. It picks the right "shape" for the code (like choosing between a recursive loop or a fold) and simultaneously picks the matching mathematical rules (invariants) needed to prove that shape is safe. It's like deciding, "We will build a bridge using suspension cables, so our proof plan must include checking the tension of those cables."

Once this shared plan is locked in, the AI then "elaborates" the details. It writes the actual code and the actual proof, but it's just filling in the blanks of a pre-agreed blueprint. If the proof fails, the AI knows exactly where to look because the structure was already decided. If the plan itself is bad (e.g., the bridge design is impossible), the AI goes back to the planning stage to redraw the blueprint, rather than frantically patching the finished building.

The New Test Ground: Lean4Commit0

The authors realized that previous tests for these AI systems were too easy, like asking the robot to solve math puzzles from a textbook. Real-world software is messier. To test their new method properly, they built a new benchmark called Lean4Commit0.

They scraped 108 real-world open-source software libraries (written in Python, Rust, C/C++, and Java) and turned their core features into "verified code" challenges. Instead of simple "add two numbers" tasks, these challenges involve complex relationships between different parts of a program. For instance, in a configuration system, they might ask the AI to prove that "if you set a setting to 'High' and then later set it to 'Low', the system correctly remembers the 'Low' setting." These tasks require the AI to understand how different functions talk to each other, making them much harder than textbook problems.

What They Found: Smarter Planning Wins

The team tested P3 against four of the most powerful AI models available (including versions of Codex, Gemini, and Claude) across three different benchmarks: Verina, AlgoVeri, and their new Lean4Commit0.

The results were clear: Planning together works better than writing separately.

  • Success Rate: P3 solved more tasks than any other method in every single test. On the hardest tasks, it improved the success rate by 4.6 to 11.2 percentage points compared to the best existing methods.
  • Efficiency: It wasn't just about solving more problems; it was about solving them faster and cheaper. On difficult tasks, P3 reduced the cost of API calls by up to 40% and cut the time spent by up to 37%. This is because the AI didn't waste time trying to prove impossible things or rewriting code that was structurally wrong.
  • The "Joint" Advantage: To prove that the "joint planning" was the secret sauce, they ran a test where the AI planned the code but didn't plan the proof ahead of time. This "code-only planning" method performed worse than P3, confirming that thinking about the proof while planning the code is what makes the difference.

A Real-World Example: The Red-Black Tree

To show how this works in practice, the authors looked at a classic computer science problem: deleting a node from a "red-black tree" (a complex data structure used to organize data efficiently).

  • The Old Way (Program-then-Proof): The AI committed to a specific way of deleting the node. It turned out this way was so structurally messy that the proof required over 6,300 lines of code just to fix the holes, or it failed entirely.
  • The P3 Way: The AI first planned the deletion. It realized that a different structural approach would be easier to prove. It stuck to that plan and solved the problem in just 1,105 lines.

Why This Matters

This paper suggests that for AI to write truly reliable software, we need to stop treating the "code" and the "proof" as two separate jobs. By forcing the AI to think about the mathematical safety of the code while it is designing the code itself, we get software that is not only correct by construction but also cheaper and faster to produce. It's a shift from "fix it later" to "build it right the first time," ensuring that the software we rely on is as solid as the math that proves it works.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →