💻 computer science

SweRank: Software Issue Localization with Code Ranking

This paper introduces SweRank, an efficient retrieve-and-rerank framework for software issue localization that leverages the newly constructed SweLoc dataset to achieve state-of-the-art performance on benchmarks while surpassing the cost and latency of closed-source LLM agent-based approaches.

Original authors: Revanth Gangi Reddy, Tarun Suresh, JaeHyeok Doo, Ye Liu, Xuan Phi Nguyen, Yingbo Zhou, Semih Yavuz, Caiming Xiong, Heng Ji, Shafiq Joty

Published 2026-04-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Revanth Gangi Reddy, Tarun Suresh, JaeHyeok Doo, Ye Liu, Xuan Phi Nguyen, Yingbo Zhou, Semih Yavuz, Caiming Xiong, Heng Ji, Shafiq Joty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a massive mystery inside a library that contains billions of books (the software code). A patron (the developer) comes to you with a complaint: "The book on page 4,000,000 is missing a chapter, and now the story doesn't make sense!"

Your job is Software Issue Localization: finding the exact page, chapter, or even the specific sentence in that giant library where the mistake happened, so you can fix it.

This paper introduces a new detective tool called SWERANK. Here is how it works, explained simply:

The Problem: The "Over-Thinker" vs. The "Overwhelmed"

Currently, there are two main ways to solve this mystery:

The "Super-Detective" Agents (The Old Way):
Imagine hiring a brilliant but expensive detective who has to physically walk through the library. They have to read a book, check the index, ask a librarian, walk to another shelf, read another book, and repeat this process 10 times before they find the mistake.
- The Catch: This takes a long time (high latency) and costs a fortune (high API costs) because the detective is a high-paid AI (like Claude-3.5) who charges by the minute. Also, if they get distracted or make one wrong turn, the whole investigation fails.
The "Keyword Search" (The Traditional Way):
Imagine using a simple search engine that just looks for matching words. If the patron says "missing chapter," the engine finds every book that has the words "missing" and "chapter" in them.
- The Catch: Software bugs are rarely described with simple keywords. They are described as messy stories like, "The app crashed when I tried to upload a photo, and now my screen is blue." Simple search engines get confused by these long, messy descriptions and can't find the right code.

The Solution: SWERANK (The "Smart Librarian")

The authors created SWERANK, which is like hiring a Smart Librarian who uses a two-step process to find the mistake instantly and cheaply.

Step 1: The Quick Scan (The Retriever)

First, the librarian has a super-fast memory. They don't read every word of every book. Instead, they look at the patron's messy complaint and instantly scan the entire library to pull out the top 50 most likely suspects (files or functions).

How they do it: They were trained on a massive new dataset called SWELOC. Think of SWELOC as a giant training manual made from thousands of real-life "complaints" and the "exact pages" where the errors were fixed in the past. This teaches the librarian to understand the story of a bug, not just the keywords.

Step 2: The Deep Dive (The Reranker)

Now, the librarian has 50 suspects. Instead of guessing, they take these 50 candidates and read them carefully side-by-side with the complaint. They act like a judge, saying, "Okay, Candidate A looks promising, but Candidate B actually matches the 'blue screen' story perfectly. Let's move Candidate B to the top of the list."

The Result: They produce a final, highly accurate list of the exact code that needs fixing.

Why is SWERANK a Game Changer?

It's Cheaper: The "Super-Detective" agents cost about $0.66 per mystery to solve. SWERANK costs a tiny fraction of that (pennies) because it doesn't need to take 10 long walks through the library; it just does a quick scan and a focused review.
It's Faster: The agents take minutes to solve a problem. SWERANK does it in seconds.
It's Smarter: Because it was trained on SWELOC (real-world bug reports), it understands that a bug description is a "story of failure," not a "request for a new feature." It knows how to read between the lines of a messy complaint.

The Analogy of the "Training Data" (SWELOC)

The authors realized that previous training data was like teaching a mechanic how to build a car, but then asking them to fix a car that broke down. They built SWELOC, a dataset specifically made of "broken car stories" paired with the "exact wrench" needed to fix them. This allowed their model to learn the specific skill of finding the break, rather than just building things.

The Bottom Line

SWERANK proves that you don't need a slow, expensive, multi-step AI agent to fix software bugs. You just need a highly trained, efficient ranking system that can quickly narrow down the search and then intelligently pick the winner. It's the difference between hiring a detective to walk the whole city versus using a satellite map to pinpoint the exact house in seconds.

In short: It finds the needle in the haystack faster, cheaper, and more accurately than the current state-of-the-art methods.

1. Problem Statement

Software Issue Localization is the task of identifying specific code locations (files, classes, or functions) relevant to a natural language issue description (e.g., a bug report or feature request). This is a critical prerequisite for automated software repair.

The paper identifies two main limitations in current approaches:

Agent-Based Systems: Recent methods use LLM-based agents (e.g., SWE-Agent, LocAgent) that iteratively explore codebases using tools like grep or file traversal. While effective, these systems suffer from high latency, significant API costs (e.g., ~$0.66 per example with Claude-3.5), and brittleness due to complex multi-step reasoning chains.
Traditional Code Ranking Models: Existing retrieval models are typically optimized for "Query-to-Code" (finding code that implements a function) or "Code-to-Code" retrieval. They struggle with issue localization because:
- Issue descriptions are often verbose and describe observed failures rather than desired functionality.
- They lack training data specifically pairing bug reports with the exact code modifications required to fix them.

2. Methodology

The authors propose SWERANK, a highly efficient retrieve-and-rerank framework designed specifically for software issue localization.

A. Dataset Construction: SWELOC

To bridge the data gap, the authors constructed SWELOC, a large-scale dataset curated from public GitHub repositories.

Source: Derived from 3,387 high-quality Python repositories (top 11,000 PyPI packages), excluding those in existing benchmarks (SWE-Bench, LocBench).
Extraction: Identifies Pull Requests (PRs) that resolve linked issues and include test file modifications. The issue description serves as the query, and the modified functions serve as positive examples.
Quality Control:
- Consistency Filtering: Uses a pre-trained embedding model to ensure the positive code snippet is semantically closer to the issue description than other unrelated functions in the repository (Top-K filtering, $K=20$ ).
- Hard Negative Mining: Selects negative examples (unmodified functions) that are semantically similar to the positive code but irrelevant to the issue, making the training task more challenging.

B. The SWERANK Framework

The framework employs a two-stage architecture:

SWERANKEMBED (Retriever):
- Architecture: A bi-encoder model generating dense vector representations for both issue descriptions and code functions.
- Training: Optimized using InfoNCE contrastive loss. It learns to maximize the similarity between an issue and its corresponding positive code while minimizing similarity with hard negatives.
- Variants:
  - Small: 137M parameters (initialized from CodeRankEmbed).
  - Large: 7B parameters (initialized from GTE-Qwen2-7B-Instruct).
SWERANKLLM (Reranker):
- Architecture: An instruction-tuned Large Language Model (LLM) acting as a listwise reranker.
- Input: Takes the issue description and the top- $K$ candidates retrieved by SWERANKEMBED.
- Training Objective: Unlike standard listwise training that requires a full ground-truth ranking of all candidates, SWERANKLLM is trained to generate only the identifier of the single correct positive function. This adaptation allows training on datasets where only the positive is known, not the relative ranking of negatives.
- Variants:
  - Small: 7B parameters (CodeRankLLM).
  - Large: 32B parameters (Qwen-2.5-32B-Instruct).

3. Key Contributions

SWERANK Framework: A novel, cost-effective retrieve-and-rerank pipeline that outperforms complex agent-based systems in both accuracy and efficiency.
SWELOC Dataset: A new, large-scale, high-quality dataset specifically for issue localization, featuring rigorous consistency filtering and hard-negative mining. It serves as a valuable resource for fine-tuning various existing models.
Training Strategy Innovation: A modified listwise training objective for LLM rerankers that works effectively even without a full ground-truth permutation of negative candidates.
Comprehensive Evaluation: Extensive benchmarks demonstrating state-of-the-art (SOTA) performance across file, module, and function granularities.

4. Experimental Results

The framework was evaluated on SWE-Bench-Lite and LocBench.

Performance vs. Agents:
- SWERANK significantly outperforms agent-based methods (e.g., SWE-Agent, LocAgent, OpenHands) across all granularities.
- On SWE-Bench-Lite, SWERANKLLM-LARGE achieves 83.21% Acc@1 (File) and 95.99% Acc@1 (Module), surpassing the best agent-based baselines (e.g., LocAgent with Claude-3.5 at 77.74% Acc@1 File).
- Even the SWERANKEMBED-SMALL (137M) model outperforms 7B parameter embedding models and many agent systems.
Cost and Efficiency:
- Cost: Agent-based methods cost ~$0.66 per example. SWERANKLLM costs approximately $0.011–$ 0.015 per example (a ~60x reduction).
- Latency: SWERANK processes an instance in ~12.5 seconds (on a single A100), whereas LocAgent takes ~85 seconds.
- Token Efficiency: SWERANK uses ~3x fewer input tokens and significantly fewer output tokens by avoiding iterative reasoning loops.
Generalizability:
- Fine-tuning various retrievers (Arctic-Embed, CodeRankEmbed) and rerankers (Qwen-2.5, Llama-3.1) on SWELOC yields consistent performance gains.
- The model generalizes well to non-Python languages (SWE-Bench Multilingual), outperforming proprietary retrievers like Gemini-Embedding.
Complexity Analysis:
- SWERANK maintains robust performance on complex issues (requiring changes to 2–3 functions) where agent performance degrades significantly.
- The reranker effectively captures cross-function dependencies that the bi-encoder retriever might miss.

5. Significance

Paradigm Shift: The paper demonstrates that framing software issue localization as a ranking problem is more efficient and effective than the prevailing agentic reasoning approach. It challenges the notion that complex, multi-step tool use is necessary for high-accuracy code localization.
Practical Deployment: By drastically reducing cost and latency while improving accuracy, SWERANK makes real-time, automated issue localization feasible for integration into IDEs and CI/CD pipelines.
Community Resource: The release of SWELOC addresses a critical data scarcity in the field, providing a standardized, high-quality benchmark for training and evaluating future issue localization models.

In conclusion, SWERANK establishes a new state-of-the-art for software issue localization, proving that specialized, data-driven ranking models can outperform expensive, general-purpose agent systems in this specific domain.