A Grounded Theory of Debugging in Professional Software Engineering Practice

Technical Summary: A Grounded Theory of Debugging in Professional Software Engineering Practice

Problem Statement
Debugging is a central yet complex activity in software engineering, yet most engineers are never explicitly taught how to debug. While prior research has documented various debugging strategies and tool usage, there is a lack of theoretical understanding regarding how experienced developers reason about bugs within large, real-world codebases. Existing knowledge is largely derived from surveys, interviews with hypothetical scenarios, or observational studies where participants worked in unfamiliar codebases without long-term stakes. These methods often fail to capture the nuanced, iterative reasoning processes employed by professionals in their actual work environments.

Methodology
The authors employed a constructivist grounded theory approach to inductively develop a theory of debugging from observational data. The study proceeded in two stages:

1. Participant Observations (P1–P7): Seven professional software engineers (average 6.5 years of experience) were observed performing 11 debugging tasks on their own familiar codebases (ranging from 10k to >1M lines of code) across diverse domains (AI, finance, medicine, OS). Sessions involved screen sharing, think-aloud protocols, and follow-up interviews.
2. Streamer Analysis (S1–S5): To validate and enrich the theory, the authors analyzed 3.1 hours of recordings from five professional live-coding streamers working on six debugging tasks. These streamers verbalized their thought processes, approximating a think-aloud protocol.

Data analysis involved open coding, focused coding, and the creation of analytic memos to identify recurring patterns. The researchers prioritized coding verbal transcripts to capture reasoning processes over fine-grained coding of screen actions. The study achieved theoretical saturation, with the streamer data refining existing categories rather than generating new ones.

Key Contributions
The paper contributes a grounded theory of professional debugging that surfaces human-centered dimensions of the practice. The primary contributions include:

• A Structured Process Model: Debugging is theorized as a structured, iterative diagnostic process comprising four high-level steps: (1) Initial bug reproduction, (2) Mental model development, (3) Fix development, and (4) Fix validation.
• Mental Model Development as Core: The study identifies "mental model development" as the dominant phase, consuming approximately 57% of debugging time. This process involves updating a mental model of the system to guide information gathering.
• Knowledge Avoidance Strategy: Contrary to advice suggesting developers should construct a thorough understanding of a system, the authors observed a "knowledge avoidance" approach. Developers aim to construct a "good-enough" mental model sufficient to fix the bug, avoiding the costly effort of exhaustive system comprehension.
• Uncertainty and Speculation: The study challenges the view that debugging is strictly a scientific hypothesis-testing process. Instead, developers often posit vague explanations or speculations, refining them iteratively based on gathered evidence rather than constructing formal, testable hypotheses.
• Adaptive Strategies: Developers dynamically alternate between navigation (static inspection) and execution (runtime analysis) strategies, and between forward tracing (predicting outcomes) and backward tracing (reasoning from failure), adapting these modes based on codebase complexity and familiarity.
• Socio-Technical Integration: The research highlights the heavy reliance on external resources (colleagues, version control history, web searches, AI tools) to complement code-based evidence, a dimension often underreported in prior literature.

Results

• Time Allocation: On average, developers spent 12% of time reproducing issues, 57% updating mental models, 13% fixing issues, and 18% validating fixes.
• Iterative Cycles: Tasks averaged 1.65 fix-validate cycles. In 9 of 17 tasks, developers returned to the mental model update step after an initial fix attempt failed.
• Strategy Usage:
  • Navigation vs. Execution: Developers combined static navigation (reading code, searching) with execution (breakpoints, logs). Execution was used more frequently for complex issues to verify speculations, while navigation was used for initial scoping.
  • Tracing Modes: Backward tracing (working from the bug to the cause) was dominant in low-familiarity contexts and at high abstraction levels (e.g., version history). Forward tracing (tracing execution flow) was more common in high-familiarity contexts and at granular levels (line-by-line reading).
  • External Resources: Developers utilized web searches, AI tools (LLMs), and social interactions (colleagues, chat) to fill knowledge gaps. AI was used selectively to understand specific code segments or syntax but was often insufficient for generating maintainable fixes.
• Experience Impact: Experience allowed developers to form hypotheses quickly and take shortcuts. Familiarity with the codebase reduced the need for extensive backward tracing, whereas low familiarity necessitated "archaeological" digging through version history and external documentation.

Significance and Claims
The paper claims to be the first to use grounded theory analysis to study the debugging process in situ. Its significance lies in:

• Bridging Theory and Practice: It reconciles discrepancies between interview-based reports (which often cite scientific hypothesis testing) and observational studies (which show specific, often vague, strategies) by showing that developers use speculative reasoning that resembles hypotheses but is too vague to be formally testable.
• Human-Centered Perspective: It emphasizes that debugging is as much about reasoning and managing uncertainty as it is about implementing code fixes.
• Implications for Tool Design: The authors suggest future tools should support mental model management by tracking untested speculations, visualizing confidence levels in code regions, and integrating socio-technical resources (history, expertise) more effectively.
• Implications for Education: The findings suggest that debugging education should move beyond rigid hypothesis testing to teach students how to manage uncertainty, engage in speculative reasoning, and navigate complex, unfamiliar codebases using a mix of technical and socio-technical strategies.

The authors maintain a modest stance, noting that their sample was limited by the requirement for participants to share code, and that the think-aloud protocol may have influenced behavior. However, they argue their findings offer a more representative view of professional practice than studies relying on unfamiliar codebases or hypothetical scenarios.