💬 NLP

PatchRecall: Patch-Driven Retrieval for Automated Program Repair

PatchRecall is a hybrid retrieval approach that combines codebase matching with history-based analysis to balance high recall and conciseness in selecting relevant files for Automated Program Repair, thereby improving effectiveness on benchmarks like SWE-Bench without significantly increasing the number of retrieved files.

Original authors: Mahir Labib Dihan, Faria Binta Awal, Md. Ishrak Ahsan

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mahir Labib Dihan, Faria Binta Awal, Md. Ishrak Ahsan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master mechanic (an AI) hired to fix a specific problem in a massive, sprawling city of buildings (a software codebase). The city has thousands of buildings, and the problem description you received is vague: "The traffic light at the main intersection is blinking red when it should be green."

Your job is to find the exact blueprints for the specific building that needs fixing, read them, and make the repair.

The Problem: The "Needle in a Haystack" Dilemma

In the past, when mechanics tried to fix these cities, they used a simple search engine. They would type "traffic light" into a giant library of blueprints.

The Issue: The search engine would pull out 50 blueprints. Maybe 2 of them were actually about the traffic light, but the other 48 were about street signs, bus stops, or power lines.
The Consequence: The mechanic gets overwhelmed by all that extra noise. They might fix the wrong building, or get so confused by the irrelevant papers that they give up.
The Reality: In 8 out of 10 cases, the fix only requires changing one single file (one single blueprint). But the old search methods kept handing them a whole stack of useless papers.

The Solution: PatchRecall (The "Smart Detective")

The authors of this paper, PatchRecall, built a new kind of detective to help the mechanic. Instead of just searching the library, this detective uses two different strategies at the same time and combines the results.

Strategy 1: The "Direct Search" (Looking at the City)

This is the old method, but improved. The detective looks at the current problem description and searches the city's blueprints for keywords.

Analogy: If the problem is "leaky faucet," this strategy finds all blueprints that mention "plumbing" or "water."
Weakness: It's good at finding words, but bad at understanding the meaning. It might grab a blueprint for a "water fountain" when you needed a "kitchen sink."

Strategy 2: The "History Book" (Looking at Past Cases)

This is the new, clever part. The detective has a massive notebook of past repairs from similar cities.

Analogy: The detective thinks, "Wait, three years ago, someone had a 'blinking traffic light' problem. They fixed it by changing the code in the 'Admin Panel' building. Let's look at what they did."
How it works: The system finds the 10 most similar past problems, looks at which files were changed to fix them, and suggests those same files for the current problem.
Strength: It learns from experience. Even if the words are slightly different, the pattern of the fix is often the same.

The Magic Mix: "Fusion"

Now, the detective has two lists of suspects:

List A: Files found by searching the current city (Direct Search).
List B: Files found by looking at past repair logs (History Book).

The detective doesn't just pick one list. They merge them.

If a file appears on both lists, it's a top suspect.
If a file is only on one list, it gets a lower score.
They create a final, short list of the top 5 most likely files to fix.

Why This Matters

The paper tested this on a famous benchmark called SWE-bench (a giant test of real-world software bugs).

Before: The AI would grab too many files, get confused, and fail to fix the bug.
With PatchRecall: The AI gets a shorter, smarter list of files. It finds the right file more often (higher "recall") without getting bogged down by junk data.

The Bottom Line

Think of PatchRecall as upgrading from a generic search engine to a seasoned detective.

The search engine just matches words.
The detective matches words AND remembers how similar problems were solved in the past.

By combining these two approaches, the AI can pinpoint the exact file that needs fixing much faster and more accurately, making automated software repair actually work in the real world.

1. Problem Statement

Automated Program Repair (APR) relies heavily on the initial step of file retrieval: identifying the specific files within a large codebase that require modification to resolve a reported issue.

The Trade-off: There is a critical tension between recall (ensuring the correct file is included in the candidate set) and efficiency/conciseness (avoiding the inclusion of too many irrelevant files).
- Retrieving too few files risks missing the target, leading to guaranteed failure.
- Retrieving too many files introduces "noise," overwhelming Large Language Models (LLMs) and degrading their ability to generate correct patches.
Current Limitations: Existing benchmarks (like SWE-bench) and baselines (like BM25) often fail to localize edits precisely. Empirical analysis shows that while traditional methods retrieve a set of files, over 80% of real-world issues require changes to only a single file. Current retrieval strategies often fail to distinguish this single critical file from thousands of irrelevant ones, leading to high noise levels and low resolution rates.

2. Methodology: PatchRecall

The authors propose PatchRecall, a hybrid retrieval framework designed to balance high recall with a concise candidate set. The method integrates two complementary strategies:

A. Two-Stream Retrieval

Codebase Retrieval (Direct):
- Uses the BM25 algorithm (sparse retrieval) to match the issue description directly against the files in the target repository.
- This captures lexical similarities between the issue text and the code.
History-Based Retrieval (Patch-Driven):
- Leverages the SWE-bench-unverified dataset (1,794 issues) as an auxiliary knowledge base.
- When a new issue arrives, a Sentence Transformer (dense retrieval, all-mpnet-base-v2) finds the top-10 most semantically similar past issues.
- The system extracts the files modified in the patches associated with those past issues.
- These files are aggregated and scored based on their frequency and relevance to the current issue.

B. Hybrid Scoring and Re-ranking

The system merges the candidate lists from both streams and applies a min-max normalization to make scores comparable. A hybrid score $H(f)$ is calculated for each file $f$ :

$H(f) = \alpha \cdot s'_{ST}(f) + (1 - \alpha) \cdot s'_{BM25}(f)$

$s'_{ST}$ : Normalized score from the Sentence Transformer (history-based).
$s'_{BM25}$ : Normalized score from the BM25 retrieval (codebase-based).
$\alpha$ : A weighting hyperparameter (tuned between 0 and 1).

The final top- $k$ files are selected based on $H(f)$ and fed into the downstream LLM for patch generation.

3. Key Contributions

Empirical Insight: The authors conducted a failure analysis on SWE-bench, revealing that:
- Failures are concentrated in specific complex repositories (e.g., Django).
- 80%+ of tasks require edits to only one file, yet traditional retrieval often returns many irrelevant files.
- Current APR agents struggle disproportionately with complex, multi-file tasks (>4 hours difficulty).
Novel Framework: Introduction of PatchRecall, which uniquely combines semantic similarity (via Sentence Transformers on historical issues) with lexical similarity (BM25 on the current codebase).
Efficiency: The method achieves higher recall without significantly increasing the number of retrieved files, directly addressing the noise problem in APR.

4. Experimental Results

Experiments were conducted on the SWE-bench-verified dataset (500 human-validated issue-patch pairs).

Baseline Comparison:
- Sentence Transformers alone outperformed both BM25 and TF-IDF across all retrieval depths ( $k$ ), proving the value of semantic understanding over pure keyword matching.
- BM25 performed competitively at low $k$ but saturated quickly.
Hybrid Performance:
- The hybrid approach (with $\alpha$ between 0.4 and 0.6) achieved the highest recall.
- Specifically, the hybrid method outperformed the pure BM25 baseline used in state-of-the-art frameworks like MAGIS.
- The results demonstrate that incorporating historical patch data significantly improves the localization of the correct files compared to relying solely on the current codebase text.

5. Significance and Impact

Bottleneck Resolution: The paper identifies file localization as a primary bottleneck in APR. By improving the retrieval stage, the authors enable downstream LLMs to work with cleaner, more relevant contexts.
Scalability: The approach is lightweight and modular, meaning it can be integrated into existing multi-agent systems (like MAGIS) or standalone APR pipelines without requiring complex architectural overhauls.
Future Directions: The authors suggest future work could involve adaptive weighting (dynamically tuning $\alpha$ based on issue complexity) and integrating structural signals (ASTs, dependency graphs) for even finer-grained localization.

In conclusion, PatchRecall demonstrates that leveraging historical repair patterns alongside direct code search creates a more robust retrieval system, moving the field closer to practical, scalable automated program repair for large-scale software repositories.