🤖 AI

Improved Bug Localization with AI Agents Leveraging Hypothesis and Dynamic Cognition

This paper introduces CogniGent, a novel agentic AI framework that leverages causal reasoning, call-graph analysis, and hypothesis-driven dynamic cognition to significantly outperform existing traditional and LLM-based methods in software bug localization.

Original authors: Asif Mohammed Samir, Mohammad Masudur Rahman

Published 2026-02-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Asif Mohammed Samir, Mohammad Masudur Rahman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a crime in a massive, sprawling city (the software codebase). The only clue you have is a vague witness statement: "Something went wrong when I tried to save my file, and now my data is gone."

Traditional methods of finding the culprit (the bug) are like handing that witness statement to a librarian who only knows how to match words. If the witness says "file" and the crime happened in a room labeled "File," the librarian points there. But what if the crime actually happened in the "Database" room, but the witness just used the wrong word? Or what if the crime was a chain reaction where one person pushed another, who then pushed a third, causing the final disaster? Word-matching librarians often miss these connections.

Enter CogniGent: The Detective with a Brain and a Team.

This paper introduces a new AI system called CogniGent. Instead of just matching words, CogniGent acts like a team of human detectives who use hypothesis testing and dynamic reasoning to solve the case.

Here is how it works, broken down into simple steps:

1. The "Hypothesis" Phase (The Detective's Gut Feeling)

When a human developer gets a bug report, they don't just look at the code immediately. They ask themselves, "What could possibly cause this?" They form a theory (a hypothesis).

CogniGent does the same: It reads the bug report and asks, "If the system is failing here, maybe the problem started way back in the 'Login' module, or maybe it's a 'Database' issue?" It generates several theories about where the bug might be hiding.

2. The "Click2Cause" Phase (Following the Trail)

In the real world, if a developer suspects a specific line of code, they might "Ctrl+Click" on it to see what other functions it calls, and then click on those to see what they call. They follow the trail of breadcrumbs.

CogniGent's Superpower: It uses a special algorithm called Click2Cause. Instead of blindly searching the whole city, it dives deep into one specific path (like following a single suspect down a dark alley) to see how the error spreads. It checks: "If this function fails, does it break the next one? Does that break the one after that?"
The Metaphor: Imagine a game of "Telephone." If the first person whispers the wrong message, the last person hears gibberish. CogniGent traces the message back from the gibberish to the very first whisper to find out who started it.

3. The "Scratchpad" (Keeping the Clues Organized)

AI models often get confused if you give them too much information at once (like trying to read a whole library book in one second).

CogniGent's Trick: It uses a "scratchpad." As it investigates a specific path, it writes down its thoughts and evidence on a temporary notepad. Once it finishes that path, it clears the notepad and moves to the next theory. This keeps its "brain" focused and prevents it from getting overwhelmed by irrelevant code.

4. The "Observer" (The Judge)

Once the investigation team has gathered evidence for all their theories, a final "Observer" agent steps in. It looks at the evidence, compares it against the original bug report, and decides: "Okay, Theory A is weak, but Theory B has strong proof." It then ranks the suspects (the code files) from most likely to least likely to be the culprit.

Why is this a big deal?

The researchers tested this system on 591 real bug reports from famous software projects (like Apache HBase and Spring). They compared CogniGent against:

Old-school search engines (which just match words).
Other AI tools (which are getting better but still struggle with complex chains of errors).

The Results:
CogniGent was a massive success.

It found the buggy code 23% to 38% more accurately than the best existing methods.
It was especially good at finding bugs that were spread across multiple files (the "chain reaction" bugs), which usually stump other tools.

The Bottom Line

Think of traditional bug-finding tools as a flashlight that only shines on the spot you are looking at. If the bug is in the dark corner behind the wall, the flashlight misses it.

CogniGent is like a detective with a map, a magnifying glass, and a team. It doesn't just look at the spot; it asks why the spot is dark, follows the trail of clues, tests its theories, and finds the root cause even if it's hidden deep in the code's history. This makes fixing software faster, cheaper, and less frustrating for developers.

1. Problem Statement

Software bugs impose significant financial costs and consume approximately 50% of developers' time. While automated bug localization is a long-standing challenge, existing approaches suffer from three critical limitations:

Isolated Analysis: Traditional methods (Spectrum-based and Information Retrieval/IR) often analyze code components (methods, documents) in isolation, failing to account for inter-component dependencies and error propagation across the codebase.
Lack of Causal Reasoning: Current techniques, including recent LLM-based agents, rely heavily on textual or semantic similarity between bug reports and code. They often lack the ability to perform causal reasoning to trace symptoms back to root causes, especially when bug reports describe failure symptoms rather than specific code locations.
Context Management Issues: Agentic AI techniques that iteratively explore code often struggle with context overload. As the context window grows with new information, LLM performance tends to degrade, and irrelevant information can obscure the signal.

2. Methodology: CogniGent

The authors propose CogniGent, a novel agentic framework that emulates Dynamic Cognitive Debugging—a human-like process of hypothesis formulation, evidence gathering, and testing. The system utilizes a multi-agent workflow orchestrated via LangGraph, integrating six specialized agents:

A. Core Workflow Components

Graph-Based Indexing: Source code is indexed as a directed heterogeneous graph (using Neo4j) where nodes represent code segments (methods, constructors) and edges represent relationships (invocation, inheritance). This allows for structural navigation beyond simple text matching.
Preprocessing & Retrieval:
- Restructuring Agent: Uses a lightweight LLM to clean and restructure bug reports, removing noise (e.g., emotional language) to improve query quality.
- Retrieval & Filtering Agents: Use IR (Lucene) to fetch initial candidates, followed by an LLM-based Intelligent Relevance Feedback (IRF) step to filter candidates based on contextual reasoning rather than just keyword matching.
Dynamic Cognitive Debugging (The Core Innovation):
- Hypothesis Agent: Generates multiple hypotheses regarding the root cause of the bug based on the report and filtered code, assigning confidence scores.
- Investigation Agent (Supervisor & Explorer):
  - The Supervisor (built on the ReAct framework) guides the exploration.
  - The Explorer uses a novel algorithm called Click2Cause. Unlike Breadth-First Search (BFS) used in prior work (e.g., LocAgent), Click2Cause employs Depth-First Search (DFS) to mimic a developer's "Ctrl+Click" navigation.
  - It traverses the call graph recursively, using a scratchpad-based context management strategy. This keeps the reasoning context focused on the current path, preventing context overload and allowing the agent to backtrack and prune unproductive paths dynamically.
Observer Agent: An independent agent evaluates the explored code chains and evidence against the original hypotheses to assign a final suspiciousness score, mitigating bias from earlier stages.

B. Final Ranking

The system produces a ranked list of faulty components at two granularity levels: Methods and Documents. The ranking combines confidence scores from the investigation and observation phases.

3. Key Contributions

Dynamic Cognitive Debugging Emulation: A novel agentic workflow that replicates human debugging practices by formulating and testing hypotheses to reason about causal relationships between bug symptoms and code.
Click2Cause Algorithm: A repository-level code navigation algorithm using DFS and scratchpad-based context management to efficiently traverse call graphs without context overload.
Multi-Agent Architecture: A pipeline of six role-specific agents (Restructuring, Retrieval, Filtering, Hypothesis, Investigation, Observer) that collaboratively localize bugs.
Comprehensive Evaluation: Extensive testing on a curated dataset of 591 bug reports from 15 Java systems (132 versions), comparing against six state-of-the-art baselines.

4. Experimental Results

The authors evaluated CogniGent using Mean Average Precision (MAP), Mean Reciprocal Rank (MRR), and HIT@K metrics.

Performance vs. Baselines:
- Document Level: CogniGent (using the Devstral model) achieved a MAP of 0.407, representing a 23.33% improvement over the best LLM-based baseline (LocAgent) and a 23.33% improvement over traditional IR baselines.
- Method Level: Achieved a MAP of 0.226, a 38.57% improvement over traditional baselines.
- MRR Gains: Significant improvements of 25.14% to 53.74% were observed across both granularity levels compared to baselines.
Robustness:
- Multi-Component Bugs: CogniGent excelled at localizing bugs spanning multiple documents or methods, outperforming competitors in covering buggy components within the Top-5 results.
- Bug Types: Performed consistently across Natural Language (NL), Program Elements (PE), and Stack Trace (ST) bug reports, though ST reports remained the most challenging.
Statistical Significance: Wilcoxon signed-rank tests confirmed that CogniGent's improvements over baselines (specifically BLIZZARD and LocAgent) are statistically significant ( $p < 0.05$ ) with medium to large effect sizes.
Ablation Study: Removing the Investigation module caused the largest performance drop (11.57% at document level), confirming that causal reasoning and dependency traversal are the primary drivers of success.

5. Significance

This research bridges the gap between human-like cognition and agentic automation in software engineering. By moving beyond surface-level text matching to causal reasoning and dependency-aware navigation, CogniGent addresses the "black box" nature of many LLM-based tools. It demonstrates that simulating the iterative, hypothesis-driven debugging process of human developers significantly improves the accuracy of automated bug localization, offering a scalable and practical solution for identifying root causes in complex, interconnected codebases. The authors have released a replication package to ensure reproducibility.