Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports
This paper introduces a standardized taxonomy for subclassifying invalid bug reports and evaluates various AI approaches, finding that Retrieval Augmented Generation excels at root-cause identification while agentic web search is most effective for generating actionable no-code fixes.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you run a busy, high-tech repair shop. Every day, hundreds of people drop off their gadgets, claiming something is broken. But here's the catch: many of these gadgets aren't actually broken.
Some people just forgot to plug them in (Configuration Issues). Some are asking for a new feature that doesn't exist yet (Feature Requests). Some are holding the device upside down and asking why the screen is dark (User Error). And some are just asking, "How do I turn this on?" (Questions).
In the software world, these are called Invalid Bug Reports.
The problem is that your team of expert engineers (the developers) spends hours trying to "fix" these non-problems. It's like a master mechanic trying to rebuild an engine when the car just needs a new battery. It's a waste of time and money.
This paper introduces a new AI-powered "Triage Assistant" designed to solve this exact problem. Here is how it works, broken down simply:
1. The Goal: Sort the Trash from the Treasure
The researchers wanted to build a system that can instantly look at a complaint and say:
- "This is a real broken part; send it to the engineers."
- "This isn't broken; here is a simple instruction on how to fix it yourself without touching any tools."
They call these simple instructions "No-Code Fixes." Think of them as a "Try turning it off and on again" or "Check your settings menu" guide, but generated automatically by a smart computer.
2. The New "Rulebook" (Taxonomy)
Before building the AI, the researchers created a new, organized rulebook for classifying these complaints. Instead of just saying "Invalid," they sorted them into specific buckets, like:
- "It's not broken, it's just working as designed" (You asked for a feature that doesn't exist).
- "You're using the wrong version" (You're running an old version of the app; update it!).
- "It's not your fault, it's someone else's" (The problem is with a different app or website you are connected to).
- "I can't see the problem" (You didn't tell us enough details to reproduce the error).
3. The Experiment: Three Different "Detectives"
To see which AI method works best, they tested three different "detectives" on a dataset of real bug reports from the Brave web browser (a popular open-source project).
- Detective A (The "Vanilla" LLM): This is a smart AI that just reads the complaint and guesses the answer based on everything it learned during its training. It's like a knowledgeable librarian who has read every book but hasn't looked at the specific shelf you are standing in.
- Detective B (The "RAG" Detective): This AI is given a magnifying glass and a library card. Before guessing, it searches through the project's own history, past bug reports, and official manuals to find similar cases. It's like a mechanic who checks the specific car's service history before diagnosing the problem.
- Detective C (The "Web Search" Detective): This AI is an explorer. If it doesn't know the answer, it goes out to the live internet to check for recent news, updates, or discussions about the specific error. It's like a mechanic who calls the manufacturer's hotline or checks online forums to see if a new patch was released yesterday.
4. The Results: Who Won?
The researchers found that different detectives are better at different jobs:
For Sorting the Complaints (Subclassification):
The Library Card Detective (RAG) was the best overall. By looking at the project's own history, it was slightly better at figuring out why a report was invalid. It was especially good at spotting when a user just couldn't reproduce a bug or when they were asking for a new feature.- The Weak Spot: Even the best detective struggled with "Wrong Version" reports. It's hard for an AI to know if a bug is fixed in a version released yesterday unless it has real-time access.
For Writing the Solution (No-Code Fixes):
The Explorer Detective (Agentic Web Search) won here. When it needed to tell a user how to fix their problem, going out to the live web allowed it to find the most up-to-date solutions and workarounds. It was the most successful at giving users a helpful, actionable answer.
5. The Big Takeaway
The paper concludes that you can't just use one "smart AI" for everything.
- If you want to categorize a problem, give the AI access to the company's internal history (RAG).
- If you want to solve a problem for a user, let the AI go out and check the live web (Agentic Search).
By using these tools, the "repair shop" can automatically send simple instructions to the 40% of customers who just need a little guidance, freeing up the human engineers to focus only on the gadgets that are truly broken.
What the paper does NOT claim:
- It does not claim this system is perfect; it still struggles with some specific types of errors (like "Wrong Version").
- It does not claim this works for every single software company in the world; they tested it specifically on the Brave browser.
- It does not claim that human support staff are obsolete; the goal is to reduce their repetitive work, not replace them entirely.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.