One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents

Technical Summary: One Tool Is Enough: Reinforcement Learning of LLM Agents for Repository-Level Code Navigation

Problem Statement
Locating specific files and functions that require modification within large-scale software repositories is a significant challenge due to the vast size and structural complexity of codebases. Existing Large Language Model (LLM) agents typically approach this as a repository-level retrieval task, relying on multiple auxiliary tools (e.g., SearchClass, GetImports). However, these multi-tool pipelines often overlook the actual logic of code execution, complicate model control, and suffer from error propagation. Furthermore, while post-training via Supervised Fine-Tuning (SFT) or Reinforcement Learning (RL) has shown promise, directly training agents to fix software issues is computationally prohibitive due to the need for Docker-based execution environments for evaluation. Consequently, the authors focus on issue localization—identifying the relevant code regions—as a tractable proxy that enables verifiable, string-level evaluation suitable for scalable training.

Methodology
The authors propose RepoNavigator, an LLM agent designed for repository-level issue localization. The methodology rests on three core components:

1. Single Execution-Aware Tool (jump): Instead of a suite of high-level abstraction tools, RepoNavigator utilizes a single tool called jump. This tool leverages a language server (specifically Pyright for Python) to resolve the definition of a given symbol (variable, function, class) at a specific source location. This design mirrors the actual flow of code execution (sequential execution and jumps) rather than high-level abstractions that disappear after compilation. The tool performs static analysis, including syntactic scope resolution (LEGB rule), type-based attribute resolution, and import dependency graph traversal.
2. Agent Architecture: The agent operates in a reason → act → observe loop. At each step, the policy network decides whether to generate a natural-language reasoning step or to emit a JSON-formatted tool call. Tool calls are constrained via decoding to ensure valid syntax. The agent continues this loop until it predicts the final set of relevant code locations.
3. Reinforcement Learning (RL) Training: RepoNavigator is trained end-to-end directly from a base pretrained model (Qwen2.5 series) without relying on distillation from closed-source models. The authors employ Group Relative Policy Optimization (GRPO). The reward function is a hybrid of two components:
   • Outcome Reward: Based on the Dice coefficient between the predicted locations and the ground-truth locations (derived from golden patches).
   • Tool Reward: Based on the success rate of tool calls (penalizing format errors or unresolved symbols).
     This hybrid reward structure provides dense signals for tool mastery while optimizing for the final localization accuracy.

Key Contributions

1. First RL-Trained Localization Agent from Pretrained Models: RepoNavigator is the first repository-level localization agent trained using reinforcement learning directly from a pretrained open-source model, eliminating the need for distillation from closed-source teacher models (e.g., Claude).
2. Execution-Aligned Navigation: The design of a single jump tool aligns the agent’s navigation with actual execution semantics, simplifying the action space and reducing the brittleness associated with multi-tool chaining.
3. Efficiency and Scalability: The paper demonstrates that a unified, structurally grounded tool combined with RL training offers a more efficient and scalable solution than complex multi-tool pipelines, achieving state-of-the-art performance with smaller model sizes.

Results
Experiments were conducted on the SWE-bench Verified and SWE-bench Pro (Python subset) datasets. The results indicate:

• Superior Performance: RepoNavigator achieves state-of-the-art performance in issue localization. Notably, the 7B model outperforms 14B baselines, the 14B model surpasses 32B competitors, and the 32B model exceeds closed-source models like GPT-5 and Claude 3.7 Sonnet on most metrics (Sample-F1 and IoU).
• Training-Free vs. RL: Even without RL training (training-free), RepoNavigator’s single-tool design outperforms many multi-tool baselines. However, RL training (GRPO) significantly boosts performance, validating the effectiveness of the hybrid reward mechanism.
• Downstream Impact: When integrated into the repair phase of the Agentless framework, RepoNavigator’s localization results led to higher issue resolution rates compared to baselines.
• Cost Efficiency: RepoNavigator consumes significantly fewer tokens (avg. 2.68k) compared to baselines like LocAgent (48.86k) and RepoSearcher (26.66k), demonstrating higher cost-efficiency.
• Tool Simplicity: Ablation studies show that adding additional tools (e.g., GetClass, GetFunc) to the jump tool degrades performance, confirming that fewer, more capable tools are preferable for this task.

Significance
The paper argues that the complexity of multi-tool pipelines is unnecessary and detrimental for repository navigation. By abstracting navigation to a single, execution-aware jump operation, the agent’s action space is simplified, reducing exploration complexity and error propagation. The success of training directly from pretrained models via RL challenges the prevailing paradigm that requires a "cold start" via SFT or distillation. The authors conclude that integrating a single, structurally grounded tool with RL provides a robust, efficient, and scalable framework for repository-level code navigation, with potential for extension to other programming languages supported by language servers.