SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests
SWE-Doctor is a novel software engineering agent that improves patch generation by utilizing runtime diagnoses derived from multi-faceted bug reproduction tests to overcome the limitations of directly using these tests as generation targets, thereby achieving state-of-the-art resolution rates on SWE-bench benchmarks.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a very smart, but slightly literal-minded robot assistant (an AI agent) whose job is to fix broken code in a software program. You give it a complaint from a user: "This button doesn't work when I type Chinese characters." The robot's goal is to write a "patch" (a code fix) to solve the problem.
Usually, these robots try to fix the code by guessing, checking if the test passes, and trying again. But this paper introduces a new, smarter robot called SWE-Doctor.
Here is how SWE-Doctor works, explained through simple analogies:
The Problem: The "One-Test" Trap
The researchers first tried a common idea: "Let's give the robot a test that proves the bug exists, and tell it to fix the code until the test turns green (passes)."
They found this didn't work well for two reasons:
- The "Partial Fix" Problem (Fail-to-Pass): Imagine a car has two broken headlights. You give the mechanic a test that only checks the left headlight. The mechanic fixes the left one, the test turns green, and they stop. But the right headlight is still broken! The robot fixed part of the problem because it was only looking at one specific test.
- The "Misleading Clue" Problem (Fail-to-Fail): Sometimes, the robot creates a test that is broken in a weird way that doesn't match the real problem. If the robot tries to fix the code just to make that specific broken test pass, it might break the car even more or fix the wrong thing entirely. It's like trying to fix a flat tire by listening to a radio that is playing static; the noise (the test failure) doesn't tell you where the actual problem is.
The Solution: SWE-Doctor
SWE-Doctor changes the game. Instead of just saying, "Make this test pass," it acts like a detective and a doctor.
Step 1: The Multi-Faceted Exam (Multi-Faceted BRT Generation)
Instead of writing just one test, SWE-Doctor breaks the user's complaint into different "facets" or angles.
- Analogy: If a patient says, "My stomach hurts," a bad doctor might just check if they can eat an apple. A good doctor checks if they can eat an apple, if they can drink water, and if they can walk.
- SWE-Doctor creates several different tests to check every part of the complaint. This ensures the robot doesn't stop after fixing just one small part of the issue.
Step 2: The Autopsy Report (Runtime Diagnosis)
This is the most important part. When the tests run and fail, SWE-Doctor doesn't just look at the "FAIL" sign. It puts the code under a microscope (a debugger) to see exactly what happened inside the machine while it was failing.
- Analogy: Imagine a car breaks down. A simple mechanic looks at the dashboard light and guesses. SWE-Doctor opens the hood, looks at the engine while it's sputtering, checks the oil pressure, and listens to the specific sound of the grinding gear.
- It writes a "diagnosis report" that says: "The error happened in this specific file, at this exact moment, because this value was wrong." It turns the confusing "FAIL" into a clear map of where the problem is.
Step 3: The Guided Surgery (Patch Generation)
Finally, SWE-Doctor gives the robot the "Multi-Faceted Exam" results and the "Autopsy Report."
- Analogy: Instead of telling the robot, "Fix the car," the doctor says, "Here is a list of all the things that need to work (the exam), and here is a map showing exactly which gear is grinding (the diagnosis). Please fix the gear, but make sure you don't break the other parts we checked."
- Before submitting the fix, SWE-Doctor does a final check: "Did you fix all the things on the list, or just the one that was easiest?" This prevents the "Partial Fix" problem.
The Results
The researchers tested SWE-Doctor on thousands of real-world software bugs (using a benchmark called SWE-bench).
- It works better: SWE-Doctor fixed significantly more bugs than the previous best robots (about 8% to 9% more on the hardest problems).
- It finds unique solutions: It solved many problems that the other robots couldn't solve at all.
- It's not just for Python: They even tested it on Go (another programming language) and it worked there too, proving the "doctor" method isn't tied to just one type of code.
In short: SWE-Doctor stops the AI from blindly guessing to make a single test pass. Instead, it acts like a thorough doctor who runs multiple tests, examines the internal symptoms of the failure, and uses that deep understanding to perform a complete and accurate repair.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.