Automating Android Build Repair: Bridging the Reasoning-Execution Gap in LLM Agents with Domain-Specific Tools

1. Problem Statement

While Large Language Models (LLMs) have shown promise in Automated Program Repair (APR), their application to Android build failures remains underexplored and challenging.

• The Challenge: Android applications frequently fail to build due to transient, environment-specific issues (e.g., dependency conflicts, configuration errors, missing resources) rather than just logical code bugs.
• The Gap: Existing LLM agents often possess the high-level reasoning to understand why a build fails but struggle to translate that reasoning into effective low-level execution. When equipped with a general-purpose shell, agents frequently fail to sequence complex commands correctly or get trapped in trial-and-error loops, leading to low resolution rates.
• Data Scarcity: There is a lack of benchmarks for Android build repair. Existing datasets often capture repositories at a single broken point in time without providing the "ground truth" (the subsequent commit that fixed the issue), making verification difficult.

2. Methodology

The authors propose a two-pronged approach: creating a rigorous benchmark and developing a specialized agent architecture.

A. AndroidBuildBench (The Benchmark)

To address the data gap, the authors curated AndroidBuildBench, a dataset of 1,019 reproducible build failures.

• Source: Derived from the commit histories of 43 popular open-source Android projects (filtered for Java/Kotlin usage, activity, and Gradle build scripts).
• Construction: The benchmark pairs a "failing commit" with the "subsequent fixing commit." This ensures every problem has a verified, feasible solution.
• Categories: The dataset includes three types of errors:
  1. Human-Committed: Natural transient failures during development.
  2. Augmented Dependency: Simulated configuration issues (reverting build files to create mismatches).
  3. LLM-Generated: Failures caused by AI-assisted code generation, reflecting modern development workflows.
• Error Types: The benchmark covers Syntax Errors, Missing Resource Files, Configuration Errors, Library Unavailability, and NDK Errors (though NDK errors were rare in the curated set due to the requirement of a successful endpoint).

B. GradleFixer (The Agent)

The authors introduce GradleFixer, an LLM agent designed specifically for Android build repair.

• Core Strategy: Tool Bridging. Instead of giving the agent a general-purpose shell (which allows arbitrary command execution), GradleFixer replaces the shell with domain-specific tools.
• Tool Abstraction: The agent is equipped with three high-level, API-like tools:
  1. run_build: Executes the standard build command (./gradlew assembleDebug).
  2. run_gradle: Executes specific Gradle tasks (e.g., dependency checks).
  3. change_java_version: Switches the active JDK version in the environment.
• Hypothesis: By constraining the action space to relevant operations and presenting them in an API-like format, the agent can focus its reasoning on debugging logic rather than command syntax and sequencing.

C. Experimental Setup

• Models: The study primarily uses Gemini-2.5-Pro as the core LLM.
• Baselines:
  • Coding-Assistant: Can only edit files, no execution.
  • Hierarchical Agent: A multi-agent framework.
  • Gemini-CLI (Shell): The standard open-source agent with full shell access.
  • Gemini-CLI (No Shell): Read/Write only.
• Environment: Standardized Linux with multiple JDK versions installed, running builds in isolated Jupyter Notebooks.

3. Key Contributions

1. AndroidBuildBench: A new, verified benchmark of 1,019 real-world Android build errors, solving the verification problem found in prior datasets.
2. GradleFixer & Tool Bridging: A novel agent architecture that demonstrates that domain-specific abstractions significantly outperform general-purpose shells.
3. Empirical Validation of the Reasoning-Execution Gap: The paper provides strong evidence that LLMs often fail not due to a lack of knowledge, but due to the inability to execute that knowledge correctly in a raw shell environment.
4. Cost-Effectiveness Analysis: Demonstrates that a smaller, cheaper model (Gemini-2.5-Flash) equipped with the right tools outperforms a larger, more expensive model (Gemini-2.5-Pro) using a general shell.

4. Results

• Performance: GradleFixer achieved a Pass@1 resolve rate of 81.4%, significantly outperforming the state-of-the-art shell-based agent (Gemini-CLI), which achieved only 65.1%.
• Tool Specificity: An ablation study showed that performance improves as tools become more specific. The agent performed best with the most constrained tool (run_build), suggesting that focus is more critical than raw capability.
• Tool Bridging vs. Prompting: Providing "Tool Usage Guidance" via prompts improved the shell agent's performance slightly (58.9%), but it still lagged far behind GradleFixer (74.0% in ablation). This confirms that structural constraints (tools) are more effective than semantic guidance (prompts).
• Failure Analysis:
  • The primary cause of failure for GradleFixer was the magnitude of code changes. Cases involving large diffs (>700 lines) were significantly harder to fix, regardless of the error type.
  • Shell-based agents frequently fell into "red herring" loops (e.g., chasing version warnings instead of root causes) or failed to sequence commands correctly.
• Cost Efficiency: Successful repairs with GradleFixer consumed significantly fewer tokens than failed attempts by shell agents. A smaller model with GradleFixer outperformed a larger model with a shell, highlighting the economic value of domain-specific tooling.

5. Significance and Future Work

• Paradigm Shift: The paper argues that for specialized domains, the design of an agent's toolset is more critical than the size of the underlying LLM. "Tool Bridging" is proposed as a generalizable strategy to connect high-level reasoning with low-level execution.
• Practical Impact: Automating build repair can lower the barrier to entry for Android development, enabling "vibe-coding" where non-experts can experiment without fear of breaking the build.
• Future Directions:
  • Fine-tuning smaller models on domain-specific datasets.
  • Extending Tool Bridging to other ecosystems (iOS, Web).
  • Developing agents that can automatically generate and refine their own domain-specific tools.

In conclusion, the paper establishes that bridging the gap between reasoning and execution via domain-specific tools is the key to unlocking the full potential of LLMs in complex software engineering tasks like Android build repair.