Building an Internal Coding Agent at Zup: Lessons and Open Questions
This paper presents Zup's internal coding agent, CodeGen, demonstrating that engineering decisions surrounding tool design, safety guardrails, and human oversight are more critical to production readiness and adoption than model quality alone.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you've hired a brilliant, hyper-intelligent intern named CodeGen. This intern is an AI that can read your code, write new features, run tests, and even fix bugs on its own. You're excited because it promises to save your team hours of work.
But here's the problem: When you first let this intern loose in your office, it's a bit of a disaster. It accidentally deletes the wrong files, tries to rewrite entire books when it only needed to change a comma, and sometimes gets confused about what it's actually looking at.
This paper is the story of how Zup Innovation (a Brazilian tech company) took their "brilliant intern" from a chaotic prototype into a reliable, production-ready employee. They discovered that the secret to success wasn't just finding a smarter AI model; it was about how you manage the intern, what tools you give them, and how you build trust.
Here is the breakdown of their journey, using some everyday analogies:
1. The "Intern" vs. The "Manager" (Architecture)
At first, the team tried to use a pre-made "management framework" (like LangChain) to run their intern. They realized this was like trying to run a complex construction project using a toy train set. The framework was designed for simple, straight-line tasks, but coding requires a loop: Think → Act → Check Result → Think Again.
The Fix: They built their own "Manager" (called Maestro) from scratch.
- Analogy: Instead of forcing the intern to follow a rigid, pre-printed checklist, they built a dynamic control room. This control room watches the intern, hands them tools, checks their work, and tells them when to stop.
- Lesson: Don't just buy a generic tool; build the specific management system your unique job requires.
2. The "Scalpel" vs. The "Sledgehammer" (Tool Design)
One of the biggest mistakes the intern made was trying to rewrite entire files at once.
- The Problem: If you ask an AI to "rewrite this 500-line file," it often gets tired, forgets the middle part, or hallucinates (makes up) code that doesn't exist. It's like asking a chef to rewrite an entire cookbook because you wanted to change one recipe.
- The Fix: They gave the intern a Scalpel (the
edittool). Instead of rewriting the whole file, the intern is forced to say: "Find this specific sentence and replace it with that specific sentence." - Lesson: Limit the scope of the AI's actions. Small, precise cuts are safer and more accurate than big, sweeping changes.
3. The "Safety Harness" (Safety & Guardrails)
The intern has access to a powerful tool: the Shell (which lets it run commands on your computer). This is like giving a new employee the keys to the entire building, including the server room and the fire alarm.
- The Risk: The intern could accidentally type
rm -rf(a command that deletes everything) or push a broken update to the whole company. - The Fix: They built a Layered Safety Harness:
- The "Read-Before-Edit" Rule: The intern must look at the file before touching it. No guessing allowed.
- The "Human-in-the-Loop" Mode: For dangerous actions (like deleting files), the intern has to hit a "Pause" button and ask a human: "Hey, I'm about to delete this. Is that okay?"
- The "Audit Trail": Every single move the intern makes is recorded in a logbook, like a security camera, so you can see exactly what happened if something goes wrong.
4. The "Training Wheels" Approach (Human Oversight)
The team realized they couldn't just flip a switch and say, "Now you are 100% autonomous." That's too scary for developers.
- The Strategy: They introduced Progressive Trust.
- Phase 1 (Training Wheels): The intern suggests a change, but a human must click "Approve" before it happens.
- Phase 2 (Confidence): As the intern proves it's reliable, the human starts approving fewer things.
- Phase 3 (Autonomy): Eventually, the intern runs on its own for routine tasks.
- Analogy: It's like teaching a kid to ride a bike. You start with training wheels (approval mode), then you run alongside them, and finally, you let them ride solo. You don't just throw them on the bike and hope for the best.
5. The "Memory" System (State Management)
Coding tasks can take a long time. If the intern gets distracted or the internet cuts out, it shouldn't forget what it was doing.
- The Fix: They built a system that saves the "conversation history" and the "current state" of the project. If the intern gets disconnected, it can reconnect and say, "Okay, I was just about to fix the login button. Let's pick up right there."
The Big Takeaway
The paper's main message is this: The AI model itself isn't the hero; the engineering around it is.
You can have the smartest AI in the world, but if you give it a sledgehammer instead of a scalpel, or if you don't have a safety harness, it will fail. The real magic happens in the design of the tools, the safety rules, and the way humans build trust with the machine.
In short: Don't just buy a smarter brain; build a better body and a better environment for it to live in. That's how you turn a prototype into a product that people actually use.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.