PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts
This paper introduces PoCo, an agentic framework that autonomously generates executable Foundry-compatible proof-of-concept exploits from natural-language vulnerability descriptions, significantly reducing the time and effort required for smart contract security audits.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a building inspector (a smart contract auditor) who has just found a dangerous flaw in a skyscraper's design. You know the elevator can fall if someone pushes a specific button, but to prove it to the building owner and the construction crew, you can't just say, "It might fall." You need to build a small, safe model of the elevator and actually make it drop in a controlled way to show, "See? This is real, and here is exactly how to break it."
In the world of blockchain, this "model" is called a Proof-of-Concept (PoC). It's a piece of code that demonstrates a vulnerability.
The Problem:
Writing these "drop tests" (PoCs) by hand is like trying to build a complex model airplane while blindfolded. It takes a long time, it's easy to make a mistake, and auditors are often racing against a ticking clock. If they can't build the model quickly, they might not be able to prove the building is unsafe before it gets built.
The Solution: PoCo
The paper introduces PoCo, which stands for "Proof-of-Concept." Think of PoCo not as a simple calculator, but as a robotic apprentice with a very specific job.
- The Input: You (the auditor) give the robot a note in plain English describing the problem. For example: "If a user sends money to address zero, the fees get stuck and disappear."
- The Robot's Brain (Agentic AI): Unlike older tools that just try to guess the answer once, PoCo is an "agentic" system. This means it thinks, acts, and learns in a loop, just like a human would.
- Reason: It reads your note and looks at the building's blueprints (the code).
- Act: It tries to write a script to break the elevator.
- Observe: It runs the script. Did it crash? Did it fail to compile?
- Revise: If it failed, the robot doesn't give up. It looks at the error message, figures out what went wrong (e.g., "I used the wrong tool"), and tries again.
- The Output: Eventually, the robot hands you a working, executable script that successfully demonstrates the flaw. It's ready to be put into the official report.
How They Tested It
The researchers didn't just hope it worked; they put the robot through a rigorous exam:
- The Dataset: They gathered 23 real-world security flaws from actual blockchain projects (like a collection of real broken elevators).
- The Test: They asked PoCo to build a "drop test" for each one.
- The Baselines: They compared PoCo against two other methods:
- The "One-Shot" approach: Asking a smart AI to write the whole code in one go without checking its work. (This failed most of the time, like a student guessing on a math test).
- The "Workflow" approach: Giving the AI a rigid, step-by-step checklist. (This was better, but still got stuck when the problem required looking outside the checklist).
- The Result: PoCo was the clear winner. It successfully built working "drop tests" for 50 out of 69 attempts (across different AI models), while the other methods struggled significantly.
The "Patch" Trick
How did they know the robot's "drop tests" were actually correct and not just lucky guesses? They used a clever trick called Patch-Based Validation.
- Imagine the building crew fixes the elevator (the "patch").
- The researchers took the robot's "drop test" and ran it against the fixed elevator.
- The Logic: If the robot's test fails to break the fixed elevator, it means the test was actually good—it was successfully exploiting the old broken version, and the fix worked. If the test still worked on the fixed elevator, the robot had failed to find the real problem.
Key Takeaways from the Paper
- Autonomy is Key: The robot's ability to explore the code, read error messages, and change its own plan was more important than just using the "smartest" AI model available. Even a slightly less powerful AI model performed better when it had the freedom to iterate.
- Details Matter: The more detailed the auditor's note was, the better the robot did. However, giving the robot a rigid, step-by-step script on how to break it sometimes confused it. It's better to tell the robot what is broken and why, and let the robot figure out the how.
- Safety: The robot operates in a safe, isolated sandbox (a digital "garage") so it can't accidentally break real money or real contracts.
In Summary
PoCo is a tool that turns a human's description of a security hole into a working, automated test that proves the hole exists. It saves auditors time, reduces errors, and helps developers fix vulnerabilities faster, making the blockchain ecosystem safer. The paper proves that an AI agent that can "think, act, and learn" is far superior to tools that just follow a static script or guess once.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.