TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code
TraceCoder is a trace-driven multi-agent framework that enhances automated debugging of LLM-generated code by leveraging fine-grained runtime traces for causal analysis, a historical lesson learning mechanism to avoid repetitive failures, and a rollback strategy to ensure iterative improvements, achieving significant gains in accuracy and cost-efficiency over existing baselines.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you ask a very talented, but occasionally confused, robot to write a piece of computer code for you. Sometimes, the robot gets it right. But often, especially for tricky tasks, it writes code that looks good on the surface but has hidden, subtle bugs that cause it to fail.
Current methods for fixing these bugs are like a teacher who only says, "Wrong," and gives the robot a blank piece of paper to try again. The robot guesses, fails, gets told "Wrong" again, and keeps guessing the same wrong things over and over. It's stuck in a loop of frustration.
TraceCoder is a new system designed to break this cycle. Think of it not as a single robot, but as a team of three expert detectives working together to solve a mystery, plus a smart notebook and a safety net.
Here is how the team works, using simple analogies:
1. The Detective Team (The Multi-Agent Framework)
Instead of one robot guessing blindly, TraceCoder splits the job into three specialized roles:
The Instrumentation Agent (The "Spy"):
Imagine the code is a black box. The Spy doesn't just wait for the box to break; it secretly installs tiny cameras and microphones inside the machine while it runs. These "cameras" (called diagnostic probes) record exactly what the code is doing, step-by-step, like a flight recorder on a plane. This gives the team a clear view of the internal chaos, rather than just seeing the final crash.The Analysis Agent (The "Sherlock Holmes"):
This agent looks at the footage from the Spy. Instead of just seeing "It failed," it sees why it failed. "Ah, I see the code tried to divide by zero here," or "It thought the number 0 was positive." Crucially, this agent also checks the Smart Notebook (see below) to make sure they don't make the same mistake twice. It figures out the root cause of the problem.The Repair Agent (The "Mechanic"):
Once the Analysis Agent says, "The problem is here, and here is the plan to fix it," the Mechanic gets to work. It carefully edits the code to match the plan. It doesn't just guess; it follows a specific blueprint created by the detective team.
2. The Smart Notebook (Historical Lesson Learning Mechanism)
One of the biggest problems with current AI is that it has a short memory. If it tries to fix a bug and fails, it often forgets why it failed and tries the exact same wrong fix again.
TraceCoder has a Smart Notebook. Every time the team tries to fix something and fails, they write down:
- What we tried.
- Why it didn't work.
- What we learned from the failure.
Before the team tries a new fix, they read the notebook. This prevents them from falling into the same "loop of failure" and helps them avoid repeating past mistakes. It's like a student who reviews their old test papers to ensure they don't get the same question wrong twice.
3. The Safety Net (Rollback Mechanism)
Sometimes, a repair attempt makes things worse. Maybe the Mechanic fixes one bug but accidentally breaks something that was working fine. This is called "Performance Degradation."
TraceCoder has a Safety Net. It constantly keeps a copy of the "best version so far." If a new attempt makes the code worse or doesn't improve it, the system immediately hits "Undo" and reverts to the last good version. This ensures the team never gets stuck in a downward spiral; they always stay on a path that is at least as good as where they started.
The Results: Why It Matters
The paper tested this system on several difficult coding challenges. Here is what they found:
- It's much more accurate: TraceCoder fixed significantly more bugs than other methods. On some difficult tests, it improved the success rate by over 34% compared to the best existing tools.
- It stops the "loop of failure": By using the Spy (to see inside the code) and the Notebook (to learn from history), it avoids the repetitive guessing that traps other systems.
- It handles complex logic: It is particularly good at fixing subtle logic errors (like thinking 0 is a positive number) that are hard to spot just by looking at the final error message.
In summary: TraceCoder changes the game from "Guess and Check" to "Observe, Learn, and Fix." By giving the AI a way to see inside its own code, a way to remember past mistakes, and a safety net to prevent backsliding, it acts much more like a human expert programmer debugging a complex problem.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.