Writing Bug Reports for Software Repair Agents: What Information Matters Most?
This paper investigates how to optimize bug reports for AI software repair agents by analyzing real-world issues and finding that reports containing localization cues and suggested fixes significantly improve agent success rates, whereas traditional human-centric details like reproduction steps offer less benefit.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you're a master detective, but instead of solving crimes, you're a super-smart AI robot hired to fix broken code in a massive digital city. Usually, when a human finds a bug, they write a long, detailed diary entry for a human developer: "Hey, I clicked the red button, the screen turned purple, and here are the 15 steps I took to make it happen. Also, I think it's in the kitchen, but I'm not sure."
But what if your new boss isn't a human? What if your boss is an AI agent that reads that diary and tries to fix the problem instantly? A new study by researchers at the University of Lugano and the University of Sannio suggests that the "perfect" bug report for a human is actually a terrible instruction manual for a robot.
Here's the scoop: The researchers took 500 real-life bug reports from a famous testing ground called SWE-bench Verified. They filtered out the ones that weren't actual bugs, leaving 441 real trouble spots. Then, they handed these reports to three different AI "brains" (GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash) and watched to see which reports helped the AI fix the code and which ones left it scratching its digital head.
The Big Surprise: Less Story, More Map
The study suggests a radical idea: AI agents don't need a novel; they need a treasure map.
When humans write bug reports, they love to include the "story": exactly what happened, what they expected to happen, and a step-by-step guide on how to break it again. The researchers found that for AI, this "story" part is often just background noise. In fact, when they ran simulations, they found that removing the "steps to reproduce" or the "expected behavior" didn't really hurt the AI's chances of fixing the bug, as long as the report still had something more important.
What the AI craves is direction.
The data suggests that the two things that make an AI super successful are:
- Localization Cues: Telling the AI where to look. Did the bug happen in a specific file? A specific function? Or just "somewhere in the kitchen"? The study found that reports pointing to specific lines of code or functions made the AI significantly more likely to succeed. It's like giving the detective a map that says, "The clue is in the third drawer," instead of "Look around the house."
- Suggested Fixes: Telling the AI how to fix it. This was the biggest winner. Whether the human wrote a snippet of code or just said, "Try changing this variable to zero," the AI's success rate jumped. It's as if the human detective didn't just point to the clue but also handed the robot the exact tool needed to crack the safe.
What the AI Ignores (and What It Hates)
The researchers explicitly argue against the idea that "more information is always better."
- Length is a trap: They found that longer bug reports actually made it harder for the AI to succeed. It seems that extra words often just add confusion or hide the important clues.
- Links are risky: If a report just says, "Check out this other website for details," the AI tends to fail. The study suggests that AI agents can't easily "go look" at external links the way humans can. If you want the AI to fix it, you have to summarize the important parts of that link right there in the report.
- The "Story" isn't the key: While humans think the "reproduction steps" are the most important part of a bug report, the study suggests that for an AI, these are less critical than knowing where the bug is or how to fix it.
How Sure Are We?
The researchers didn't just guess; they ran the numbers. They used a statistical model to measure how much each piece of information boosted the AI's chances.
- They found that suggested fixes (in code or plain English) were the strongest boost. In their simulations, having a suggested fix in natural language doubled the odds of the AI succeeding (an odds ratio of 2.01).
- Localization cues (like pointing to a specific line of code) also gave a solid boost, increasing the odds of success by about 52% (an odds ratio of 1.52).
- However, they also ran a "what-if" experiment (an ablation study) where they took 65 perfect reports and surgically removed specific parts. They found that if you took away both the location clues and the fix suggestions, the AI's success rate tanked. But if you removed the "story" parts (like reproduction steps), the AI could still often solve the puzzle if it had the map and the tool.
The Takeaway for the Future
The paper suggests that we need to change how we talk to our robot coworkers. Instead of writing a long, emotional diary entry about how the software broke, we should write a concise "repair specification."
Think of it like this: If you're asking a human to fix a leaky faucet, you tell them, "It drips when I turn the handle, and I think the washer is old." But if you're asking a robot, you should say, "The leak is in the washer (location), and replacing it with a rubber one should stop it (suggested fix)."
The study suggests that by shifting our focus from "telling the story" to "giving the instructions," we can help AI agents fix software bugs much faster. It's not about writing more; it's about writing sharper.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.