BugScope: Learn to Find Bugs Like Human

1. Problem Statement

The rapid acceleration of code generation via Generative AI has made software auditing critical. However, existing bug detection methods face significant limitations:

• Traditional Static Analysis (e.g., Infer, CodeQL): These tools rely on structural reasoning and graph-based analysis. They often fail to detect bugs requiring deep semantic understanding (e.g., complex data flows across files or mathematical reasoning about variable ranges) because they abstract away code semantics or require manual, labor-intensive rule modeling.
• LLM-Based Auditors (e.g., Cursor BugBot, Claude Code): While capable of deep semantic reasoning, they struggle with:
  • Context Limitations: They cannot effectively retrieve relevant code fragments across large codebases without hallucinating or missing dependencies.
  • Misalignment: They lack the structured, disciplined reasoning process of human auditors, leading to excessive false positives (overly permissive) or missed bugs (overly restrictive).
  • Domain Specificity: General-purpose LLMs do not inherently align with the specific anti-patterns and detection logic required for diverse bug types (e.g., Out-of-Bounds, Divide-by-Zero, Memory Leaks).

2. Methodology: The BUGSCOPE Framework

BUGSCOPE is a multi-agent framework designed to emulate the human auditing process. It operates in two distinct phases: a Learning Stage and an Auditing Stage.

A. Learning Stage (Guideline Synthesis)

Instead of prompting an LLM directly with a bug definition, BUGSCOPE uses a "learning from examples" approach to distill reusable guidelines.

1. Information Extraction: The system ingests a real-world bug report and extracts structured data (fix commits, relevant functions, file paths, and code bodies).
2. Data Augmentation: To overcome the scarcity of examples, the LLM is guided to generate positive (buggy) and negative (correct) code examples across three dimensions:
   • Seed Identification: Enumerating program constructs that instantiate bugs (e.g., division operators for Divide-by-Zero).
   • Context Retrieval: Exploring how to trace data/control dependencies backward/forward and defining termination conditions (when to stop retrieving code to avoid noise).
   • Detection Logic: Identifying "evidence patterns" (sufficient conditions) that confirm a bug, such as boundary checks that implicitly allow zero values.
3. Guideline Synthesis: From these augmented examples, the system synthesizes two core artifacts:
   • Retrieval Strategy: A set of rules for a Context Retrieval Agent (including a Tree-sitter based seed extractor and traversal rules).
   • Detection Prompt: A structured prompt for a Bug Detection Agent containing specific anti-pattern specifications and reasoning constraints.

B. Auditing Stage (Multi-Agent Execution)

Given a target repository, the framework executes a disciplined workflow:

1. Context Retrieval Agent:
   • Parses the codebase using Tree-sitter to build an AST and inter-procedural call graph.
   • Identifies "seeds" (suspicious operations) using the synthesized extractor.
   • Performs selective inter-procedural expansion, tracing data dependencies (parameters, return values) only as far as the synthesized termination conditions dictate. This balances cost and coverage.
2. Bug Detection Agent:
   • Receives the retrieved context and applies the synthesized Detection Prompt.
   • Performs step-by-step semantic reasoning to identify evidence patterns.
3. Validation:
   • A validator module checks generated bug reports to ensure all assumptions are grounded in the code semantics, filtering out hallucinations.

3. Key Contributions

• Human-Inspired Workflow: The first framework to explicitly structure LLM-based auditing into Seed Identification, Context Retrieval, and Bug Detection, mirroring the disciplined process of human experts.
• Automated Guideline Synthesis: A novel method to automatically distill domain-specific retrieval strategies and detection logic from a small set of real bug reports via data augmentation, eliminating the need for manual rule engineering.
• Neural-Symbolic Integration: Combines the precision of static analysis (Tree-sitter AST parsing, data-flow tracing) with the semantic reasoning capabilities of LLMs.
• Termination Conditions: Introduces specific stopping criteria for context retrieval to prevent infinite loops and reduce noise, a critical factor for scalability.

4. Experimental Results

The authors evaluated BUGSCOPE on a curated dataset of 33 real-world bugs (from 21 projects) and large-scale real-world codebases (including the Linux kernel).

• Controlled Benchmark (33 Bugs):

  • Performance: Achieved 86.05% Precision, 87.88% Recall, and an F1 score of 0.87.
  • Comparison: Significantly outperformed leading baselines:
    • Claude Code (Opus 4.6): F1 = 0.51
    • RepoAudit: F1 = 0.38
    • Cursor BugBot: F1 = 0.43
    • Meta Infer: F1 = 0.05
  • BUGSCOPE successfully detected bugs across diverse anti-patterns (OOB, DBZ, Memory Leaks) where baselines often failed to identify any true positives.

• Real-World Deployment:

  • Applied to 6 major open-source C/C++ projects (e.g., vim, git, systemd), discovering 80 new bugs (76.92% precision), 73 of which were confirmed or fixed by developers.
  • Applied to the Linux kernel, uncovering 184 previously unknown bugs (91.07% precision on a subset), with 78 fixed and 7 explicitly confirmed by maintainers.

• Ablation Studies:

  • Removing the Context Retrieval Agent dropped recall to 42.42%.
  • Removing the Learning/Alignment Stage (synthesizing prompts directly from reports without augmentation) caused false positives to increase six-fold, dropping precision significantly.

5. Significance

BUGSCOPE represents a paradigm shift in automated software auditing. It moves beyond "prompt engineering" or "static analysis augmentation" to a learning-based adaptation where the model internalizes the process of auditing.

• Scalability: By using structured retrieval and termination conditions, it makes LLM auditing feasible for massive codebases (millions of lines) where holistic analysis is impossible.
• Reliability: The validation step and structured reasoning drastically reduce hallucinations and false positives, making the tool practical for industrial use.
• Generalizability: The framework is not limited to specific bug types; it successfully generalized to new anti-patterns in the Linux kernel without retraining, demonstrating its potential as a universal auditing assistant.

In conclusion, BUGSCOPE demonstrates that by aligning LLMs with human-like, step-by-step auditing workflows and distilling specific domain guidelines, it is possible to achieve state-of-the-art bug detection performance that surpasses both traditional static analyzers and current commercial AI coding assistants.