Evaluating LLM-Generated ACSL Annotations for Formal Verification
This paper empirically evaluates the effectiveness of five automated systems, including three large language models, in generating and verifying ACSL specifications for 506 real-world C programs, providing new evidence on their capabilities and limitations compared to traditional rule-based approaches.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are building a massive, complex house (a software program). You want to be absolutely sure it won't catch fire, collapse, or have a door that leads to nowhere. To do this, you hire a super-strict building inspector (Formal Verification) who checks your blueprints against a strict set of safety laws (ACSL annotations).
The problem? Writing these safety laws in the inspector's strict, mathematical language is incredibly hard, boring, and prone to human error.
The Big Question: Can Artificial Intelligence (AI) write these safety laws for us, and can the inspector actually trust them?
This paper is a "taste test" where the authors tried five different "chefs" to see who could write the best safety rules for 506 different C programs (the house blueprints).
The Five Chefs (The Systems Tested)
- The Robot Chef (Rule-Based Python Script): This isn't AI. It's a rigid set of instructions. It follows a recipe exactly. It's boring, but it rarely makes a mistake.
- The Safety Guard (Frama-C RTE Plugin): This is a specialized tool designed specifically to catch runtime errors (like dividing by zero). It's like a guard dog that only barks at specific, known dangers.
- The Creative Writers (Three Large Language Models):
- DeepSeek: A very smart, modern AI.
- GPT-5: The famous, powerful AI from OpenAI.
- OLMo3: An open-source AI model.
- Think of these as brilliant but sometimes over-eager interns who know a lot of theory but might get the details slightly wrong.
The Experiment: The "Taste Test"
The authors took 506 C programs and asked each of the five chefs to write safety rules for them. Then, they handed these rules to a panel of four different "Inspectors" (SMT Solvers: Alt-Ergo, CVC4, CVC5, and Z3) to see if the rules made sense and if the house was safe.
They measured two things:
- Did the inspector understand the rules? (Proof Success)
- Did the inspector get confused and take forever to check? (Timeouts and Speed)
The Results: What Happened?
1. The Robot and the Guard Dog (The Winners)
The Rule-Based Script and the Safety Guard were the clear winners.
- Analogy: They wrote rules that were simple, dry, and perfectly formatted.
- Result: The inspectors understood them 99% of the time and finished the job in a flash. They were boring but reliable. If you need a guarantee that your software won't crash, these are the ones to use.
2. The Creative Writers (The Mixed Bag)
The AI models (DeepSeek, GPT-5, OLMo3) were much more interesting, but also much more chaotic.
- DeepSeek: This was the "star student." It wrote rules that were almost as good as the Robot Chef. It was smart enough to be precise but creative enough to handle complex situations.
- OLMo3: It did a decent job, sitting in the middle. It was stable but not as impressive as DeepSeek.
- GPT-5: This one was the most "human-like" and the most problematic. It wrote rules that sounded great and looked fancy, but they were often too vague or confusing.
- Analogy: Imagine GPT-5 writing a safety rule like, "Make sure the door is safe." The inspector (the computer) asks, "What does 'safe' mean? Is it locked? Is it fireproof? Is it the right color?" The inspector gets stuck trying to figure it out, leading to Timeouts (the inspector gives up because the task is too hard).
The Key Takeaway: The "Expressiveness vs. Stability" Trade-off
The paper found a fundamental tension, like a seesaw:
- On one side: Stability. The Robot Chef and the Guard Dog produce rules that are easy for computers to check. They are fast and reliable, but they might miss subtle, complex dangers because they are too rigid.
- On the other side: Expressiveness. The AI models (especially GPT-5) can write very detailed, nuanced rules that cover complex scenarios. However, because they are so detailed and sometimes slightly ambiguous, the computer inspectors get overwhelmed and slow down or fail.
The "DeepSeek" Sweet Spot:
DeepSeek managed to balance this best. It wrote rules that were complex enough to be useful but clear enough for the inspectors to understand quickly.
The Verdict
- Can AI write safety rules? Yes!
- Can we trust them blindly? Not yet.
- The Future: We can't just let AI write the rules and hope for the best. We need a "Hybrid" approach. We might use the AI to draft the ideas (because it's creative), but then we need a "Robot Chef" or a human to clean up the rules so the computer inspector can actually verify them without getting a headache.
In short: AI is a brilliant brainstorming partner, but for critical safety checks, we still need the boring, rigid, rule-following tools to do the final, reliable verification. The paper proves that while AI is getting better, it still needs a little help to be truly "formal" and trustworthy.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.