HAFix: History-Augmented Large Language Models for Bug Fixing
This paper introduces HAFix, a novel approach that leverages rich historical repository data through seven specific heuristics and an aggregation strategy to significantly enhance Large Language Models' bug-fixing performance, demonstrating substantial improvements over non-historical baselines while identifying optimal prompt styles and analyzing practical deployment costs.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a master detective trying to solve a crime (a software bug). Usually, when you arrive at the scene, you only look at the room exactly as it is right now. You see the broken window and the scattered papers. This is how most current AI coding assistants work: they look at the code right now and try to guess how to fix it.
But what if you could also look at the history of the room? What if you could see the security footage from last week, read the diary entries of the people who lived there, and see exactly how the room changed over time? You might realize, "Ah! The window was broken because someone tried to hang a heavy picture here three months ago, and the wall was weak!"
This is the core idea behind HAFix, a new method described in the paper. It teaches Large Language Models (LLMs)—the "super-smart" AI brains behind tools like GitHub Copilot—to act like detectives who don't just look at the present, but also study the past.
Here is a simple breakdown of how it works, using everyday analogies:
1. The Problem: The "Amnesiac" Detective
Most AI coding tools are like detectives with amnesia. They see a bug (a mistake in the code) and try to fix it based only on the code currently on the screen.
- The Limitation: They miss the context. Why was this line written this way? Who changed it last? What happened right before the bug appeared? Without this history, the AI often guesses wrong or misses the root cause.
2. The Solution: HAFix (The "Time-Traveling" Detective)
The researchers created HAFix (History-Augmented LLMs for Bug Fixing). Instead of just showing the AI the current broken code, they give it a "time capsule" of the project's history.
They use seven different "historical clues" (heuristics) to help the AI:
- The "Who" Clue: Who touched this file last? (The "Blame" commit).
- The "What" Clue: What other files changed at the same time?
- The "How" Clue: How did the code look before the mistake was made?
- The "Story" Clue: What was the developer trying to do when they made the change?
The Analogy: Imagine you are trying to fix a leaky faucet.
- Standard AI: Looks at the dripping faucet and tries to tighten a random nut.
- HAFix: Looks at the faucet, plus the plumber's logbook from last year, plus the photos of the pipes before they were installed, plus the notes on why the previous plumber chose that specific brand of washer. It uses all that history to find the real cause of the leak.
3. The Experiment: Testing the Theory
The researchers tested this on two massive libraries of real-world bugs (one for Python, one for Java). They used three different "AI brains" (Code Llama, DeepSeek-Coder, etc.) to see if giving them history helped.
The Results were like finding a treasure map:
- Better Fixes: When the AI had access to history, it fixed significantly more bugs. On average, it improved the success rate by nearly 50% compared to the "amnesiac" version.
- The "Aggregation" Trick: They found that no single clue was perfect. Sometimes the "Who" clue was best; other times, the "How" clue was best. So, they created a strategy called HAFix-Agg, which asks the AI to try all seven historical clues and combine the results. It's like asking a panel of seven different experts for advice and taking the best answer from the group. This worked even better than any single clue.
4. The Cost: Is it Worth the Time?
You might think, "If I have to read the whole history book, won't it take forever?"
- The Trade-off: Yes, looking at history takes more time and computer power (tokens).
- The Smart Shortcut: The researchers discovered a clever trick called "Early Stopping." Imagine you are reading a book to find a specific answer. If you find the answer on page 10, you don't need to read pages 11 through 500.
- The Result: By stopping the AI as soon as it finds a working fix, they cut the time and cost by 69% to 73% while still keeping the high success rate.
5. The Secret Sauce: How to Ask the Question
The paper also tested how to ask the AI for help. They tried three ways of phrasing the request:
- Instruction: "Here is the code, here is the bug, fix it."
- Label: "Here is the code, and I've put a red tag on the broken line."
- Mask: "Here is the code, but I've hidden the broken line. Fill in the blank."
The Winner: The simple "Instruction" style worked best. It turns out that for these AI models, just clearly telling them "Here is the bug, here is the history, please fix it" is more effective than trying to be fancy with tags or hiding parts of the code.
Summary: Why This Matters
This paper proves that history matters. In software development, understanding how code evolved is just as important as seeing the code itself.
- For Developers: It means future AI tools will be much smarter, acting like senior engineers who remember the project's entire journey, not just the current moment.
- For the Future: It suggests that we shouldn't just build bigger AI models; we should build AI models that are better at reading the past.
In short, HAFix teaches AI to stop guessing and start learning from experience, making software repair faster, cheaper, and much more accurate.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.