💻 computer science

Debug Like a Human: Scaling LLM-based Fault Localization to Processor Design via Block-Level Instruction-Oriented Slicing

BluesFL is a novel block-level fault localization framework that leverages LLMs and a human-inspired instruction-oriented slicing algorithm to achieve state-of-the-art bug detection accuracy and low cost in large-scale RISC-V processor designs.

Original authors: Zizhen Liu, Xiaoguang Mao, Deheng Yang, Jiayu He, Yihao Qin, Guangda Zhang, Yan Lei, Jianjun Xu, Jiang Wu

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zizhen Liu, Xiaoguang Mao, Deheng Yang, Jiayu He, Yihao Qin, Guangda Zhang, Yan Lei, Jianjun Xu, Jiang Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a massive, incredibly complex clockwork machine (a computer processor) that has stopped ticking correctly. This machine is made of millions of tiny gears and springs (lines of code). When it breaks, the machine doesn't just stop; it might start spinning a gear the wrong way or skipping a beat.

Finding exactly which gear is broken is like looking for a needle in a haystack, but the haystack is the size of a city, and the needle is invisible. This is the problem the paper "Debug Like a Human" tries to solve.

Here is how the authors, led by Zizhen Liu and his team, built a new tool called BluesFL to help fix this, explained simply:

The Problem: Too Much Noise

Currently, when engineers try to find bugs in these giant machines, they face four big headaches:

The "Whole Library" Problem: The code is so huge that if you show it all to a computer brain (an AI), the AI gets overwhelmed and confused, like a student trying to read an entire encyclopedia to find one typo.
The "Crowded Room" Problem: The machine processes many tasks at once. Traditional tools look at the whole room and can't tell which specific person (instruction) caused the mess.
The "Blindfold" Problem: Old tools look at the code structure but ignore the actual numbers and values flowing through the machine. It's like trying to fix a car engine by looking at the blueprints but ignoring the sound the engine makes.
The "Human" Gap: Humans are great at debugging because they follow a trail: "If this gear is wrong, it must be because that other gear pushed it." Computers struggle to mimic this specific chain of reasoning.

The Solution: BluesFL (The Smart Detective)

The authors created BluesFL, a system that teaches an AI to debug exactly the way a human engineer does. They did this in three clever steps:

1. Cutting the Haystack into Manageable Piles (Code Blockization)

Instead of showing the AI the entire 19,000-page manual at once, they chopped the code into small, logical "blocks."

The Analogy: Imagine the machine is a giant city. Instead of giving the detective a map of the whole world, they give them a map of just one neighborhood. They group together all the streets (lines of code) that talk to each other, so the AI only has to focus on one small, relevant neighborhood at a time.

2. Tracing the Footprints (Instruction-Oriented Slicing)

When the machine makes a mistake, it happens at a specific moment in time. The authors built a special algorithm called Blues that traces the "footprints" of the specific instruction that failed.

The Analogy: Imagine a crime scene. Instead of interviewing every person in the city, the detective asks, "Who was in the room exactly when the alarm went off?" The Blues algorithm builds a timeline of only the specific gears and wires that were active during the mistake. It ignores everyone else who was just standing around doing nothing.

3. Reading the Clues (Signal Values)

This is the most human-like part. The AI doesn't just look at the code; it looks at the values (the numbers) flowing through the wires at the exact moment of the crash.

The Analogy: A human mechanic listens to the engine. If a gear is supposed to turn at 100 RPM but is turning at 50, that's a clue. BluesFL lets the AI "read" these numbers from the machine's "waveform" (a recording of its heartbeat). This helps the AI understand why the mistake happened, not just where it happened.

How It Works in Practice

When a bug is found, BluesFL acts like a detective:

It gets a report saying, "The machine jumped to the wrong address at time 19."
It uses the Blues algorithm to build a path of only the code blocks involved in that specific jump.
It asks the AI: "Look at this small block of code. Here are the numbers flowing in. Does this look like the culprit?"
If the AI says "Maybe," it digs deeper, following the numbers backward to the source.
It keeps going until it finds the specific line of code that is broken and ranks it as the most likely suspect.

The Results

The team tested this on a real, open-source computer processor (the Ibex RISC-V core).

The Old Way: The best existing tools could only find the broken gear in about 7 out of 100 cases.
The New Way (BluesFL): Their system found the broken gear in 24 out of 100 cases (a huge jump!).
Cost: It was also cheap to run, costing the equivalent of about 25 cents per bug found.

The Bottom Line

The paper claims that by teaching AI to think like a human—by focusing on small, relevant pieces of code, following the specific trail of the mistake, and reading the actual numbers involved—we can find bugs in giant computer processors much faster and more accurately than before. It's not about making the AI smarter in general; it's about giving it the right map and the right magnifying glass.

Technical Summary: Debug Like a Human

Problem Statement
Fault localization (FL) in modern processor design is a critical bottleneck, consuming approximately 60% of hardware development effort. While Large Language Models (LLMs) have shown promise in module-level hardware design, applying them to large-scale, project-level processor designs remains challenging due to four specific limitations:

Lack of Processor-Specific Techniques: Existing FL methods designed for general hardware (e.g., memory controllers) often fail on processors because they do not account for the instruction-driven, complex nature of processor architectures.
Ineffective Instruction Coverage: Traditional spectrum-based methods rely on coarse-grained coverage over simulation intervals. In pipelined processors, where instructions span multiple modules and cycles, this leads to coverage pollution from "in-flight" instructions, reducing accuracy.
Underutilization of Signal Values: Current hardware FL techniques rarely leverage concrete signal values (waveforms) because interpreting them requires advanced semantic comprehension, which was previously difficult for automated tools.
Context Window Constraints: Processor projects often exceed LLM context limits. Providing the entire codebase causes models to lose focus, while providing too little context hides critical dependencies.

Methodology: BluesFL
The authors propose BluesFL, a block-level, LLM-based fault localization framework inspired by human debugging workflows. The system operates on a co-simulation framework where a hardware model and a software reference model are compared to generate test reports indicating architectural state mismatches.

The methodology consists of four key stages:

Dataflow-Based Code Blockization:
To address context limits, the SystemVerilog codebase is partitioned into disjoint code blocks based on dataflow relationships rather than file structure. These blocks are categorized into four types: ModInputBlock, ModOutputBlock, AssignBlock, and AlwaysBlock. This approach clusters statements with data dependencies, preserving local semantics while creating compact units suitable for LLM analysis.
Block-Level Instruction-Oriented Slicing (Blues):
To address the complexity of instruction execution, the authors introduce the Blues algorithm. This algorithm constructs an "Instruction Execution Path" graph starting from a failing signal and timestamp.
- Intra-Block Analysis: Analyzes dataflow within a block to identify driven signals. It calculates new timestamps ( $t'$ ) based on whether the block is combinational (same time) or sequential (previous cycle).
- Inter-Block Analysis: Traces backward to find the blocks that drive the identified signals, linking them in the execution path graph.
- Result: This process filters out irrelevant code, creating a path of only the blocks actually covered during the execution of the failing instruction, annotated with precise timestamps.
LLM Reasoning with Tool Calls:
The LLM acts as a debugging assistant that iteratively traverses the execution path. At each state $(b, t)$ , the LLM:
- Receives the code block context and driven signals (with timestamps).
- Uses a read_values tool to query specific signal values from the waveform at the annotated time.
- Uses a check_signals tool to query the execution path for blocks driving specific signals.
- Decides whether the current block is suspicious, appending it to a queue if so, or requesting further inspection.
Ranking:
Once the LLM determines the root cause is found or the analysis is complete, it assigns a confidence score (0–1) to each suspicious block, producing a ranked list of fault locations.

Key Contributions

BluesFL Framework: The first application of LLM-based fault localization specifically to real-world, project-level processor designs.
Dataflow-Based Blockization: A method to compress large codebases into semantically rich, compact blocks that fit within LLM context windows.
Blues Algorithm: A novel slicing technique that guides LLMs to focus exclusively on instruction-covered blocks and incorporates time-annotated signal values to mimic human reasoning about dynamic processor behavior.
Empirical Validation: Evaluation on a 19K-line open-source RISC-V core (Ibex) with 119 injected bugs.

Experimental Results
Evaluated on the Ibex processor core, BluesFL demonstrated significant improvements over state-of-the-art baselines (including static slicing, spectrum-based, and existing LLM-based methods like LiK and Agentless):

Accuracy: BluesFL correctly localized 24 bugs at Top-1, representing a 242.9% improvement over the best existing baseline (LiK, which found 7 bugs).
Efficiency: The average cost to localize a single bug was $0.257.
Ablation Studies: Removing signal values or the instruction execution path significantly degraded performance. Specifically, without signal values, the LLM checked 609% more blocks, increasing the cost per bug to $1.821.
Model Robustness: The framework performed competitively across different LLM backends (GPT-4o-mini, GPT-4o, Claude-3.5-Sonnet), with Claude-3.5-Sonnet achieving the highest Top-1 score (30 bugs) at a slightly higher cost.

Significance and Claims
The paper claims that BluesFL successfully scales LLM-based fault localization to the processor domain by bridging the gap between human debugging intuition (tracing instruction paths and signal values) and automated analysis. By mimicking the human process of narrowing down suspicious regions through dataflow and time-annotated execution paths, the framework overcomes the limitations of context size and coverage pollution that plague existing methods. The authors position this work as a foundational step toward agile, automated verification for complex processor designs, noting that while current results are promising, further work is needed to improve long-distance cross-block reasoning to handle cases where the debugging process terminates prematurely.