Context Before Code: An Experience Report on Vibe Coding in Practice
This experience report demonstrates that while "vibe coding" accelerates scaffolding for production systems, it necessitates a strategic shift in engineering effort toward explicit constraint specification and architectural auditing to ensure critical production requirements like isolation, security, and reliability are met.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are building a massive, high-tech library. In the past, you had to write every single book, catalog every shelf, and design the security system yourself, brick by brick.
Now, imagine you have a super-fast, incredibly talented robot assistant (the AI) who can write books and build shelves in seconds just because you talk to it. This is what the paper calls "Vibe Coding." You don't write the code line-by-line; you just "vibe" with the robot, giving it instructions in plain English like, "Build me a section for history books," and it does the heavy lifting.
The authors of this paper are a team of researchers who tried using this robot assistant to build two serious, real-world systems (not just practice projects). Here is what they learned, explained simply:
The Two Projects They Built
- The "Private Club" Library: A platform where different groups (like different companies or school classes) can have their own private AI assistants. The most important rule? No mixing. The history group shouldn't be able to see the science group's notes.
- The "Fact-Checker" Library: A system for students and researchers where the AI answers questions using only specific documents they upload. If the AI makes a claim, it must show exactly which page and paragraph it got the answer from (citations).
The Good News: The Robot is a Speed Demon
The robot assistant was amazing at the "boring stuff."
- The Analogy: Think of it like a construction crew that can instantly pour concrete, lay down drywall, and install standard light switches.
- The Result: The team got the basic skeleton of their libraries up and running in a fraction of the time it usually takes. They didn't have to write the basic code for logging in or saving files; the robot did it.
The Bad News: The Robot Doesn't "Get" the Rules
Here is the catch. The robot is great at doing tasks, but it's terrible at understanding consequences or security boundaries unless you scream them at it very loudly.
- The Analogy: Imagine you tell the robot, "Build a wall between the history and science rooms." The robot builds a wall, but it forgets to put a lock on the door, or it accidentally leaves a window open. It builds the wall because you asked for a wall, but it didn't understand that the purpose of the wall was to keep secrets safe.
- The Reality: The robot kept making mistakes like:
- Letting a science student read the history group's private notes.
- Trying to process huge files all at once, which crashed the system (instead of processing them slowly in the background).
- Forgetting to check if a user was actually allowed to see a specific document.
The "Non-Delegation Zones" (Where Humans Must Step In)
The authors discovered that there are certain parts of building a system that you cannot hand over to the robot. They call these "Non-Delegation Zones."
- The Architect vs. The Bricklayer: The robot is a fantastic bricklayer. It can lay bricks faster than anyone. But the Architect (the human developer) must still draw the blueprints.
- The Shift in Work: Before, developers spent 80% of their time laying bricks (writing basic code) and 20% checking the plans. Now, the robot lays the bricks in seconds. So, the human spends 0% time laying bricks and 100% time checking the blueprints, locking the doors, and making sure the robot didn't accidentally build a wall in the wrong place.
The Big Lesson: "Context Before Code"
The title of the paper, Context Before Code, is the golden rule they found.
If you just say, "Build me a chat app," the robot will build a chat app that might leak secrets or crash.
But if you say, "Build me a chat app where User A can never see User B's data, where long tasks happen in the background, and where every answer must be cited," the robot does a much better job.
In short:
The robot is a powerful tool that speeds up the "how" (how to write the code), but the human must still define the "what" and the "why" (the rules, the security, and the architecture). You can't just vibe your way to a secure, professional system; you have to be the strict boss who double-checks the robot's work constantly.
The Bottom Line: AI coding is like having a super-fast intern. It will do the work, but if you don't give it very specific instructions and check its work, it will make expensive mistakes. The job of the developer isn't disappearing; it's just changing from "writer" to "editor and architect."
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.