Technical Summary: Automatically Enhancing the Quality of Android App Bug Reports

Problem Statement

Most defects in mobile applications are visually observable on the device screen, yet automated mechanisms for detecting and reporting them are often unavailable, particularly for issues that do not produce explicit signals like crashes. Consequently, users, testers, and developers must manually submit bug reports (BRs). These reports are frequently incomplete, ambiguous, or inaccurate, often lacking critical information such as the Observed Behavior (OB), Expected Behavior (EB), and Steps to Reproduce (S2Rs). This issue is particularly acute for UI-centric defects, where end-users struggle to precisely describe application internals, execution states, and the sequence of GUI interactions required to reproduce a fault.

Existing approaches to improving bug reports either leave the burden of correction on the user or generate content primarily from the original report text. These methods fail to ensure that generated information accurately reflects the application's actual behavior, potentially producing plausible but incorrect reproduction steps that do not correspond to valid GUI interactions. The core challenge is not merely improving the bug description but recovering the missing connections between the reported bug information and the application's underlying UI execution.

Methodology: BUGSCRIBE

The paper presents BUGSCRIBE, an LLM-powered approach that automatically enhances user-written Android bug reports. The central insight is that high-quality bug report generation should be formulated as a problem of connecting bug descriptions with the application's UI execution information. BUGSCRIBE employs a component-specific grounding strategy, providing the Large Language Model (LLM) with the most relevant context for generating each specific component of the bug report (OB, EB, and S2Rs).

The system operates through three main phases:

1. App Execution Model Generation:
   BUGSCRIBE constructs a graph-based execution model of the application. Nodes represent unique GUI screens (distinguished by UI component hierarchies), and edges represent user interactions (e.g., taps). This model is enriched with UI metadata, including component types, IDs, labels, and descriptions. The model is built using traces from both automated exploration (using a modified CRASHSCOPE) and manual app usage.

2. Contextual Information Extraction:
   The system annotates the user-submitted report to identify sentences expressing OB, EB, and S2Rs. It then extracts three key app information elements to serve as context for the LLM:

   • App Screen Descriptions: LLM-generated summaries of screen functionality, layout, and available actions.
   • GUI Interactions: A tuple of (action, target element, source screen, target screen) extracted from the execution model.
   • Buggy Screen Localization: The specific screen where the fault occurs, identified by the LLM ranking screens based on the reported bug and screen descriptions.

3. Bug Report Generation:
   Using a zero-shot, task-decomposition prompting strategy, BUGSCRIBE generates the report components sequentially:

   • S2R Generation: The LLM maps the original report's S2R sentences to the execution model's interaction sequences to infer a valid reproduction path leading to the buggy screen.
   • OB/EB Generation: The LLM generates the Observed and Expected Behaviors by mapping the original report's descriptions to the contextual app information, including the suspected buggy screen, the generated S2Rs, and screen descriptions.
   • Assembly: The system assembles a structured report including a title, OB, EB, S2Rs, and environmental details.

Key Contributions

The paper makes four primary contributions:

1. BUGSCRIBE: An LLM-powered approach that links report information to app-specific execution information to generate accurate, complete, and correct OB, EB, and S2R descriptions.
2. Data-Driven Methodology: A methodology identifying specific app information elements (e.g., GUI interactions vs. screen descriptions) that are most effective for generating different report components.
3. Empirical Evaluation: A comprehensive evaluation demonstrating that BUGSCRIBE improves original bug reports and outperforms LLM-based baselines that lack app-specific information.
4. Replication Package: A publicly available dataset of 58 bug reports (10 development, 48 test), ground truth OB/EB/S2Rs, code, prompts, and experimental infrastructure.

Experimental Results

The authors evaluated BUGSCRIBE on a test set of 48 bug reports from 26 Android applications, comparing it against the original reports and three LLM-based baselines (GPT-5.4 and CLAUDE-Opus-4.6 without app context, and U2SBR).

• S2R Quality: BUGSCRIBE significantly outperformed baselines and original reports. It improved S2R quality by 44.1% to 82.3% (measured by F1 score). Specifically, it generated substantially more correct steps (532–538 vs. 196–345 for baselines) and significantly reduced extra and missing steps.
• OB/EB Quality: BUGSCRIBE improved OB and EB quality by 3.8% to 35.2%. It generated more correct information elements (158–165 vs. 122 for original reports) and drastically reduced missing elements (0–3 vs. 59 for original reports).
• Context Effectiveness: The study found that different components benefit from different context. S2R generation relies heavily on GUI interactions and the buggy screen, while OB/EB generation benefits from generated S2Rs, buggy screen descriptions, and screen-level context.
• Cost: The average processing time was approximately 2.3 minutes per report, with costs ranging from $0.44 to $0.92 per report depending on the LLM used.

Significance and Claims

The paper claims that the primary significance of this work lies in shifting the paradigm of bug report enhancement from text-based rewriting to execution-grounded reasoning. By connecting user reports with the application's actual UI execution model, BUGSCRIBE ensures that generated reproduction steps and behavior descriptions are not just linguistically plausible but technically valid within the context of the specific application.

The authors assert that their component-specific grounding strategy is novel, as it tailors the context provided to the LLM based on the specific needs of the report component being generated. This approach addresses the limitation of prior work, which often generates "plausible but incorrect" descriptions because it lacks the ground truth of the application's execution flow. The results suggest that integrating dynamic app analysis with LLM reasoning is a viable path toward automatically producing high-quality, actionable bug reports that can be directly used by developers for diagnosis and reproduction.