💻 computer science

UCAgent: An End-to-End Agent for Block-Level Functional Verification

The paper introduces UCAgent, an end-to-end agent that automates block-level functional verification by utilizing a pure Python environment, a 31-stage fine-grained workflow with automated checkers, and a Verification Consistency Labeling Mechanism to achieve high coverage and uncover design defects without relying on fragile LLM-generated SystemVerilog code.

Original authors: Junyue Wang, Zhicheng Yao, Yan Pi, Xiaolong Li, Fangyuan Song, Jinru Wang, Yunlong Xie, Sa Wang, Yungang Bao

Published 2026-03-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junyue Wang, Zhicheng Yao, Yan Pi, Xiaolong Li, Fangyuan Song, Jinru Wang, Yunlong Xie, Sa Wang, Yungang Bao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a massive, incredibly complex Lego castle. You have the blueprints (the design), but before you can show it off, you need to make sure every single brick is in the right place, every door opens, and the tower doesn't collapse when you blow on it.

In the world of computer chips (Integrated Circuits), this "checking" process is called Functional Verification. It's the most expensive, time-consuming, and frustrating part of making a chip. For years, it has been like trying to find a needle in a haystack while wearing blindfolded gloves.

Enter UCAgent. Think of UCAgent not as a human engineer, but as a super-smart, tireless robot assistant designed to do all the checking for you, from start to finish.

Here is how UCAgent works, explained through simple analogies:

The Problem: Why Humans (and Old Robots) Struggle

Traditionally, engineers wrote code in a very strict, difficult language called SystemVerilog to test chips.

The Language Barrier: Imagine asking a brilliant writer to write a legal contract in a language they barely know (like ancient Sumerian). They might make grammar mistakes or invent words that don't exist. This is what happens when AI tries to write chip-testing code; it's not trained enough on that specific "ancient language."
The Memory Gap: A chip design is like a 1,000-page novel. If you ask an AI to read the whole thing and then write a test for page 500, it often forgets what happened on page 10. It gets confused, hallucinates (makes things up), and the test fails.
The "Lost in Translation" Issue: If the blueprint says "Red Door," the test might check for a "Crimson Gate." The test passes, but it's checking the wrong thing. The connection between the plan and the test gets broken.

The Solution: UCAgent's Three Superpowers

UCAgent solves these problems with three clever tricks:

1. The "Universal Translator" (Python Environment)

Instead of forcing the AI to write in the difficult "ancient language" (SystemVerilog), UCAgent translates the chip into Python.

The Analogy: Imagine you are a master chef (the AI) who is amazing at cooking Italian food (Python) but terrible at cooking Japanese sushi (SystemVerilog). Instead of forcing you to make sushi, UCAgent gives you a machine that turns the sushi ingredients into Italian pasta. Now, you can cook perfectly!
How it works: UCAgent converts the chip design into a Python package. The AI writes the tests in Python (which it loves), and the system runs them against the chip. This avoids the AI making syntax errors.

2. The "Strict Foreman" (The 31-Stage Workflow)

Old AI systems tried to do everything in one giant leap. UCAgent breaks the job down into 31 tiny, manageable steps, like a recipe with 31 specific instructions.

The Analogy: Imagine building a house. You don't just say, "Build a house." You say: 1. Pour foundation. 2. Check foundation. 3. Build walls. 4. Check walls.
The Safety Net: After every single step, a digital inspector (an automated checker) looks at the work. If the AI made a mistake in step 3, the inspector stops the process immediately, says "Fix this," and the AI tries again. It doesn't let the AI move to step 4 until step 3 is perfect. This prevents small errors from piling up into a disaster.

3. The "Digital Tag System" (Verification Consistency Labeling)

This is the glue that keeps everything connected. UCAgent forces the AI to put digital tags on everything it creates.

The Analogy: Imagine you are organizing a massive library. You don't just write "Book" on a shelf. You write a specific code: History-WorldWar2-Chapter3.
How it works:
- When the AI reads the blueprint, it tags a feature: Tag: Red-Door.
- When it writes the test, it must use that same tag: Tag: Red-Door.
- If the AI tries to write a test for a "Blue-Gate" (which wasn't in the plan), the system immediately yells, "Stop! That tag doesn't exist in the blueprint!"
- This ensures the AI never hallucinates features that don't exist and never forgets to test something that does exist.

The Results: What Did It Achieve?

The researchers tested UCAgent on real computer chips (like a UART, which is like a digital walkie-talkie, and a Floating Point Unit, which does math).

Speed & Accuracy: It successfully automated the entire process, achieving nearly 100% coverage (meaning it tested almost every possible scenario).
Bug Hunter: It didn't just follow orders; it actually found new bugs in designs that human engineers had missed for a long time.
Efficiency: It turned a process that usually takes weeks of human labor into a task that can be done in hours or days.

The Bottom Line

UCAgent is like giving a super-intelligent intern a set of strict rules, a universal translator, and a tagging system. Instead of letting the AI guess and make mistakes, it guides the AI step-by-step, ensuring that every test it writes is accurate, consistent, and actually checks what the chip is supposed to do.

It's a massive leap forward, moving chip verification from "hope we didn't miss anything" to "we know we checked everything."

1. Problem Statement

Functional verification is the most resource-intensive phase of Integrated Circuit (IC) development, consuming over 70% of project timelines. Despite massive investments, traditional methods (constrained-random, formal verification) struggle with the exponential growth of design complexity, leading to low first-silicon success rates (14% in 2024).

While Large Language Models (LLMs) show promise in code generation, applying them to end-to-end hardware functional verification faces three critical bottlenecks:

Limited Accuracy in HDL Generation: LLMs are trained on vast amounts of Python data but lack proficiency in Verilog/SystemVerilog (HDLs) due to data scarcity and complex syntax, leading to unstable or incorrect verification code.
Workflow Fragility: Complex verification tasks exceed LLM context windows, causing "context rot," hallucinations, and error accumulation. Simple workflows fail to adapt to diverse hardware complexities (e.g., needing mock modules for dependencies).
Verification Consistency: Maintaining strict traceability between design specifications, coverage models, and test cases is difficult. LLMs often alter terminology or hallucinate invalid scenarios, breaking the semantic link required for reliable verification.

2. Methodology: UCAgent Architecture

The authors propose UCAgent, an end-to-end autonomous agent that automates block-level functional verification. It relies on three core mechanisms to overcome the challenges above:

A. Pure Python Verification Environment

To bypass the limitations of LLMs in generating SystemVerilog, UCAgent shifts the verification implementation entirely to Python.

PyDUT: Uses the tool Picker to convert the RTL Design Under Test (DUT) into a Python package.
Toffee: A Python-based verification framework that provides high-level abstractions (e.g., clock control, signal wrapping), shielding the LLM from low-level simulation details like delta-cycles.
Benefit: Leverages the LLM's superior proficiency in Python to generate testbenches and test cases reliably.

B. Configurable 31-Stage Fine-Grained Workflow

Instead of a monolithic prompt, the verification process is decomposed into a 31-stage workflow (e.g., feature extraction, coverage modeling, test case generation, bug analysis).

Automated Checkers: Each stage has a dedicated automated checker that validates the output against predefined rules before the workflow proceeds.
Feedback Loop: If a stage fails, the checker returns specific error messages to the LLM for self-correction, preventing error accumulation.
Configurability: The workflow is defined via YAML, allowing stages (like mock generation for dependencies) to be inserted or skipped based on the specific hardware module's complexity.

C. Verification Consistency Labeling Mechanism (VCLM)

To ensure traceability, UCAgent introduces a hierarchical labeling system (FG, FC, CK) that must be embedded in all generated artifacts:

FG (Function Group): Top-level categories (e.g., <FG-ARITHMETIC>).
FC (Function Checkpoint): Specific features within a group (e.g., <FC-VFADD>).
CK (Check Point): Specific verification conditions (e.g., <CK-FP32>).
Mechanism: Checkers parse these labels at every stage (Spec $\to$ Coverage Model $\to$ Test Case). If a label in a test case does not exist in the specification, or if a specification requirement lacks a corresponding test, the checker rejects the output. This formalizes semantic correspondence and prevents hallucinations.

3. System Implementation

UCAgent is built on four modules:

Agent Middleware: Uses LangChain and ReAct reasoning to manage LLM interactions and tool calling (via Model Context Protocol).
Workflow Management: Orchestrates the 31-stage sequence, managing state persistence and transitions.
Stage Validation: Integrates the checkers and VCLM to enforce quality and consistency.
Verification Execution Environment: Runs the Python-based testbenches using Pytest, collecting metrics (coverage, pass rates) for feedback.

4. Experimental Results

The authors evaluated UCAgent on five hardware modules (UART, Integer Divider, FPU, ICache, Page Table Walker) using three LLMs (Claude-Sonnet-4.5, GPT-5, Qwen3-Coder-Plus).

Coverage Performance:
- Achieved up to 98.5% Code Coverage and 100% Functional Coverage on modules like the Integer Divider.
- UART-16550 reached ~80% code coverage and >96% functional coverage.
Bug Discovery:
- Successfully identified previously unknown design defects in the unverified LaneFAdd module (specifically boundary condition bugs in FP16/BF16 handling).
- Traced these bugs back to specific functional points using VCLM, aiding rapid localization.
Efficiency:
- End-to-end verification times ranged from 40 to 180 minutes depending on module complexity and model.
- The checker-driven self-correction loop allowed the system to recover from high failure rates during test case generation (e.g., Claude-Sonnet-4.5 recovered from 67 failures on UART to achieve closure).
Model Comparison:
- Claude-Sonnet-4.5: Highest code coverage and finest functional granularity but slower due to iterative refinement.
- GPT-5: Most stable functional coverage (consistently 100%).
- Qwen3-Coder-Plus: Fastest execution but lower coverage due to coarser decomposition.

5. Key Contributions

Paradigm Shift: Proposes a Python-native verification environment for LLMs, effectively bypassing the data scarcity and syntax complexity of SystemVerilog.
Robust Workflow: Introduces a 31-stage fine-grained workflow with automated checkers, solving the issue of error accumulation in long-horizon tasks.
Consistency Mechanism: Develops VCLM, a formal labeling system that enforces strict traceability between specifications, coverage models, and test cases, solving the "hallucination" problem in verification logic.
Practical Validation: Demonstrates that an AI agent can autonomously discover real hardware bugs and achieve industry-standard coverage metrics without human intervention in the loop (except for massive scale modules).

6. Significance

UCAgent represents a significant step toward fully autonomous hardware verification. By decoupling verification logic from the limitations of HDL generation and enforcing strict consistency through labeling, it addresses the primary bottlenecks preventing AI from replacing human verification engineers. The framework proves that LLMs can not only generate code but also manage complex, multi-stage engineering workflows with high reliability, potentially reducing the 70% verification time burden in IC design cycles.

Limitations noted: The system still requires manual configuration for workflow decomposition in massive industrial designs, and verification completeness is ultimately bounded by the base LLM's domain knowledge (e.g., specific RISC-V edge cases). Future work aims to integrate dynamic knowledge injection and fully autonomous task decomposition.