DebugHarness: Emulating Human Dynamic Debugging for Autonomous Program Repair

1. Problem Statement

The paper addresses the critical bottleneck in software security: while automated tools (like fuzzers) efficiently discover bugs, fixing deep-rooted, low-level memory safety vulnerabilities (e.g., use-after-free, heap buffer overflows) remains a labor-intensive task requiring expert manual analysis.

Existing Automated Program Repair (APR) systems, particularly those leveraging Large Language Models (LLMs), largely treat bug fixing as a static text-generation problem. They rely on static artifacts such as issue reports, stack traces, and source code. This approach fails for complex C/C++ vulnerabilities because:

• Static analysis lacks dynamic execution context (runtime memory states, register values).
• It cannot trace the temporal propagation of corrupted states from the crash site back to the root cause (which may occur in a different file or function).
• Current LLM agents act as passive code readers, missing the interactive, hypothesis-driven debugging workflows used by human systems engineers.

2. Methodology: DebugHarness

The authors propose DebugHarness, an autonomous LLM-powered debugging agent that emulates human dynamic debugging practices. Instead of static analysis, it actively queries the live runtime environment. The system operates as an end-to-end harness with three core phases:

A. Signature-Driven Initialization

• Input: A reproducible crash trigger (Proof-of-Concept input) and a sanitizer report (e.g., AddressSanitizer).
• Mechanism: The system parses the crash signature to classify the vulnerability type (e.g., heap-buffer-overflow vs. use-after-free).
• Action: It injects domain-specific troubleshooting guidelines into the LLM's system prompt. These guidelines provide error-specific heuristics (e.g., "audit heap metadata for UAF") and enforce a rule: the LLM must formulate a hypothesis before proposing code changes.

B. Interactive State Introspection (The Core Innovation)

This phase bridges the gap between static code and dynamic execution. The LLM orchestrates a suite of system tools:

• Language Server (clangd): Maps sanitizer stack frames to static source code locations.
• Standard Debugger (GDB): For execution control and forward stepping.
• Time-Travel Debugger (Mozilla rr): Allows deterministic record-and-replay. The agent can reverse-execute (reverse-continue) to trace memory corruption backward from the crash to the root cause without restarting the session.
• Memory Introspection Tools (pwndbg): Provides deep heap and memory inspection.
• Context Management: To handle verbose debugger outputs, the LLM can generate Python scripts to distill raw memory dumps into semantic summaries, preventing context window exhaustion.

The agent enters a closed-loop hypothesis-testing cycle:

1. Formulate a hypothesis about the root cause.
2. Issue commands to debuggers to verify assumptions (e.g., set watchpoints, reverse-execute).
3. Analyze the dynamic feedback to refine the hypothesis.

C. Patching and Validation

• Once the root cause is isolated, the LLM synthesizes a patch (multi-hunk unified diff).
• Deterministic Correction: A heuristic algorithm automatically fixes minor formatting errors in the LLM's generated diff (e.g., line number offsets) to prevent premature validation failures.
• Validation Loop: The system recompiles the application and re-runs the PoC and test suite. If the patch fails, compiler warnings or new crash logs are distilled and fed back to the LLM to refine the patch.

3. Key Contributions

1. Paradigm Shift: Moves automated repair from static code generation to dynamic, interactive debugging. It is the first framework to integrate interactive debugging primitives (watchpoints, reverse execution) directly into the LLM reasoning loop.
2. System Implementation: A robust client-server architecture using LangChain, clangd (for static context), and Model Context Protocol (MCP) to interface with debuggers (rr, gdb, pwndbg).
3. Signature-Driven Strategy: A novel prompting mechanism that tailors the LLM's investigation strategy based on the specific vulnerability class identified in the crash report.
4. Comprehensive Evaluation: Rigorous testing on SEC-bench, a dataset of 200 real-world C/C++ security vulnerabilities.

4. Experimental Results

The evaluation was conducted on SEC-bench (200 vulnerabilities across 29 projects like ImageMagick and mruby) using multiple LLM backbones (DeepSeek-V3.2, GLM-5, Gemini-3 Flash).

• Resolution Rate: DebugHarness successfully patched ~90% of the vulnerabilities (ranging from 89.5% to 94.5% depending on the LLM).
• Comparison with Baselines:
  • DebugHarness: ~90%
  • VulnResolver (SOTA static agent): 67.5%
  • PatchAgent (SOTA static agent): 57.5%
  • General Agents (SWE-agent, OpenHands): <40%
  • Result: DebugHarness shows a relative improvement of over 30% compared to the strongest static baselines.
• Ablation Study:
  • Removing dynamic debugging (w/o debugger) dropped the success rate to 77.0%.
  • Removing advanced tools like rr and pwndbg (GDB-only) dropped the rate to 82.0%.
  • Conclusion: Dynamic introspection is critical, especially for temporal bugs like Use-After-Free (UAF), where success rates dropped from 86.7% to 70.6% without reverse execution.
• Cost Efficiency: The average cost per vulnerability was $0.09, comparable to or better than baselines, despite the higher resolution rate.

5. Significance and Impact

• Bridging the Gap: DebugHarness effectively bridges the gap between the reasoning capabilities of LLMs and the dynamic intricacies of low-level systems programming.
• Human Emulation: It validates the hypothesis that emulating human interactive debugging workflows (hypothesis, inspection, verification) is strictly necessary for resolving non-trivial memory safety issues that static analysis cannot solve.
• Future Directions: The modular design suggests potential for extending to concurrency bugs (data races), logic bugs, and other languages, as well as integrating more diagnostic tools (e.g., Valgrind, strace).

In summary, DebugHarness represents a fundamental advancement in Automated Program Repair, demonstrating that dynamic, evidence-based debugging is superior to static code generation for fixing complex, real-world security vulnerabilities.