Intent Formalization: A Grand Challenge for Reliable Coding in the Age of AI Agents
This paper argues that "intent formalization"—the translation of informal user requirements into checkable formal specifications—is the critical challenge for ensuring the reliability of AI-generated code, proposing a spectrum of approaches from lightweight tests to full verification while highlighting the need for new metrics and human-AI interaction methods to validate these specifications.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you hire a super-fast, incredibly talented chef to cook you a meal. You tell them, "Make me something with chicken and vegetables."
The chef, who has read every recipe book in the world, immediately starts chopping, frying, and plating. In seconds, they present a beautiful dish. It looks perfect. It smells amazing. It even tastes good.
But here's the problem: Did they make what you actually wanted?
Maybe you meant "chicken soup with carrots," but they made "chicken stir-fry with broccoli." Or maybe you have a severe allergy to a specific spice they used, which you forgot to mention. The chef didn't know your specific intent; they just guessed based on what "chicken and vegetables" usually means in their training data.
This is the Intent Gap described in the paper.
The Problem: "Vibe Coding" vs. Reality
Today, AI coding tools (like GitHub Copilot or Claude) are like that super-fast chef. They can write entire software programs in seconds based on a simple sentence like "remove duplicates from this list."
But there's a catch:
- Plausible but not Correct: The code looks right and runs without crashing, but it might do the wrong thing.
- The Scale Problem: AI writes code so fast that humans can't read it all to check if it's right. We are moving from "writing code" to "vibe coding"—where we just describe what we want and hope the AI gets it right.
If the AI gets the "intent" wrong, the software is broken, even if the code is perfect.
The Solution: Intent Formalization
The paper argues that the solution isn't to make the AI write better code. The solution is to make the AI write better instructions before it writes the code.
The author calls this Intent Formalization.
Think of it like this: Instead of just telling the chef, "Make a salad," you hand them a strict checklist (a formal specification) before they start chopping.
- "The salad must have exactly 3 types of greens."
- "No nuts allowed."
- "The dressing must be on the side."
This checklist is the "formal specification." It translates your vague wish ("I want a salad") into a precise, checkable rule.
The Spectrum of Checklists
The paper suggests we don't need a perfect, 100-page legal contract for every single task. We can use a "spectrum" of checklists depending on how important the task is:
- Lightweight Tests (The "Taste Test"):
- Analogy: "If I give you the list
[1, 2, 2, 3], the result must be[1, 3]." - Use: Good for simple tasks. It's quick and catches obvious misunderstandings.
- Analogy: "If I give you the list
- Code Contracts (The "Safety Rules"):
- Analogy: "No matter what ingredients you use, the final dish must never contain more than 500 calories."
- Use: These are rules the code checks while it runs. If the code tries to break the rule, it stops immediately.
- Logical Contracts (The "Math Proof"):
- Analogy: A mathematical proof that says, "It is physically impossible for this recipe to ever produce a dish with poison."
- Use: Used for critical systems (like airplane software or bank security). It requires a computer to mathematically prove the code is correct.
- Domain-Specific Languages (The "Blueprint"):
- Analogy: You give the chef a blueprint, and the machine builds the exact dish automatically. You don't even see the cooking; the result is guaranteed to be right because the blueprint was perfect.
The Big Challenge: Who Checks the Checklist?
Here is the tricky part. If the AI writes the code, and the AI also writes the checklist, how do we know the checklist is right?
- The Oracle Problem: Usually, we check code by running tests. But who checks the specification? The only "oracle" (truth-teller) is you, the human.
- The Bottleneck: We need a way to quickly check if the AI's checklist matches your intent without you having to read a 50-page math document.
How It Works in Practice (The "TiCoder" Idea)
The paper describes a system called TiCoder that acts like a smart conversation partner.
- You say: "Find common numbers in two lists."
- The AI guesses: "Okay, I'll keep duplicates!"
- The AI immediately generates a test case: "If I give you
[1, 2, 2]and[2, 2, 3], should the answer be[2, 2]?" - You say: "No! That's wrong. I only want unique numbers."
- The AI instantly fixes its "checklist" and tries again.
This back-and-forth is much faster and cheaper than writing the whole program and then finding out it's broken.
Why This Matters
If we don't solve this, AI will just make software more abundant (lots of it, everywhere) but not more reliable (it might be full of hidden bugs).
By focusing on Intent Formalization, we turn AI from a "guessing machine" into a "precision tool." We stop asking, "Can the AI write the code?" and start asking, "Can the AI help us define exactly what the code should do, and then prove it does it?"
The Bottom Line
The paper is a call to action for researchers and companies. We need to invest in tools that help us translate our vague human wishes into precise, machine-checkable rules.
- Old Way: "Make it work." (Result: Lots of buggy code).
- New Way: "Define exactly what 'work' means, then make it work." (Result: Reliable, trustworthy AI software).
It's the difference between ordering a meal by vibe and ordering it with a precise recipe. In the age of AI agents, the recipe is the only thing standing between a great product and a disaster.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.