🤖 AI

Veri-Sure: A Contract-Aware Multi-Agent Framework with Temporal Tracing and Formal Verification for Correct RTL Code Generation

Veri-Sure is a contract-aware multi-agent framework that ensures silicon-grade RTL correctness by aligning agent intent through design contracts, performing precise localized repairs via static dependency slicing, and validating outputs through a hybrid pipeline of trace-driven temporal analysis and formal verification, all evaluated on the newly introduced industrial-grade VerilogEval-v2-EXT benchmark.

Original authors: Jiale Liu, Taiyu Zhou, Tianqi Jiang

Published 2026-01-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiale Liu, Taiyu Zhou, Tianqi Jiang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a complex, high-speed machine (like a futuristic robot) based on a set of instructions written in plain English. You ask a super-smart AI to translate those instructions into the actual blueprints (code) for the machine.

The problem is, the AI is great at writing sentences, but it often makes tiny, invisible mistakes in the blueprints that only show up when the machine starts running at full speed. In the real world of chip design, these mistakes are expensive—they can cost millions of dollars to fix once the chip is manufactured.

This paper introduces VERI-SURE, a new "team of AI experts" designed to fix these mistakes before the chip is ever built. Here is how it works, using simple analogies:

1. The Problem: The "Telephone Game" and the "Blind Spot"

When you ask a single AI to write code, two things go wrong:

The Telephone Game: If you ask the AI to fix a bug, it might misunderstand your original goal. It changes the code to fix one thing but accidentally breaks the original intent.
The Blind Spot: The AI usually checks its work by running a simulation (a test drive). But just like a test drive might miss a rare engine failure that only happens on a specific bumpy road, the simulation often misses tricky timing errors that only happen in the real world.

2. The Solution: A Specialized Construction Team

Instead of one AI doing everything, VERI-SURE sets up a construction crew where every member has a specific job. They all agree on a "Design Contract" first.

The Architect (The Contract Maker): Before anyone writes a single line of code, this agent translates your messy English instructions into a strict, mathematical "contract." It defines exactly how the machine should behave, what the buttons do, and how fast it should run. This ensures everyone on the team is reading from the same map.
The Coder (The Builder): This agent writes the actual blueprints (the code) based strictly on the contract.
The Verifier (The Safety Inspector): This agent runs the test drive (simulation). If the machine fails, it doesn't just say "It broke." It looks at the crash data.

3. The Magic Trick: "Surgical Repair" vs. "Demolition"

When the test drive fails, older AI systems often panic and try to demolish the whole building and start over. This is risky because they might forget how to build the walls correctly.

VERI-SURE uses Surgical Repair:

The Detective (Trace Analysis): It looks at the "black box" data (waveforms) to find the exact second the machine failed.
The Surgeon (Dependency Slicing): Instead of guessing, it traces the wires back to find the exact small block of code causing the problem. It isolates that tiny piece.
The Patch: The AI only rewrites that tiny, broken block. The rest of the machine stays exactly the same. This prevents the "Telephone Game" where fixing one thing breaks another.

4. The Super-Check: "Math Proof" vs. "Test Drive"

Sometimes, a test drive isn't enough to prove a machine is safe.

The Asserter (The Rule Enforcer): This agent checks if the machine follows the rules of time (e.g., "Did the light turn on exactly when the button was pressed?").
The Boolean Proofer (The Math Wizard): For the logic parts, this agent doesn't just run tests; it uses mathematical proofs to guarantee that the code cannot possibly be wrong, no matter what inputs you give it. It's like proving a bridge will never collapse using physics equations, rather than just driving a car over it once.

5. The New Test Track: "The Harder Obstacle Course"

To prove their team is the best, the authors built a new, harder obstacle course called VERILOGEVAL-V2-EXT.

The old tests were like driving in a parking lot (easy, short tasks).
The new test includes driving through a storm, navigating a maze, and handling heavy cargo (industrial-grade tasks like complex communication protocols and memory control).

The Result

When they put their team (VERI-SURE) on this new, hard obstacle course:

Standalone AIs (the lone geniuses) got about 76% of the tasks right.
Old AI Teams (without the surgical repair or math proofs) struggled with the hardest tasks.
VERI-SURE achieved 93% success, even on the hardest, most complex tasks.

In short: VERI-SURE doesn't just ask an AI to "write code." It creates a disciplined team that agrees on a plan, fixes only the broken parts with surgery, and uses math to prove the fix is perfect, ensuring the final chip works exactly as intended.

Technical Summary: VERI-SURE

1. Problem Statement

The deployment of Large Language Models (LLMs) for Register-Transfer Level (RTL) design in Electronic Design Automation (EDA) faces significant hurdles in achieving silicon-grade correctness. While LLMs have advanced in software code generation, RTL generation presents unique challenges due to the concurrent nature of hardware, strict cycle-level timing, and protocol semantics. The paper identifies three primary bottlenecks in current LLM-based RTL pipelines:

Limited Evaluation Reliability: Simulation-centric evaluations often suffer from insufficient test coverage, failing to catch subtle temporal corner cases.
Debugging Inefficiencies: Iterative debugging often leads to "regressions and repair hallucinations" caused by whole-file regeneration, which destabilizes unrelated logic.
Semantic Drift: As natural language intent is reinterpreted across multiple agent handoffs or iterations, the original design intent can degrade, leading to inconsistent implementations.

Existing benchmarks, such as VerilogEval-v2, are criticized for being dominated by short, pedagogical modules that under-represent industrial needs like advanced arithmetic units, communication protocols, and Clock Domain Crossing (CDC) patterns.

2. Methodology: The VERI-SURE Framework

VERI-SURE is a contract-aware multi-agent framework designed to close the loop between code synthesis, verification, and localized repair. It moves beyond one-shot translation to an automated generate-verify-debug cycle.

Core Components

Design Contract (Architect Agent): The framework begins by distilling natural language specifications into a structured, JSON-format design contract. This contract explicitly defines interface semantics, clock/reset conventions, cycle-accurate behavior, and corner cases. It serves as a shared "source of truth" for all subsequent agents to prevent semantic drift.
Multi-Agent Workflow:
- Coder Agent: Generates synthesizable RTL code based on the contract.
- Verifier Agent: Produces self-checking testbenches aligned with the contract.
- Debugger Agent: Leads the autonomous debugging loop when simulation fails.
- Asserter Agent: Generates targeted SystemVerilog assertions to detect sequential or protocol violations (e.g., wrong-edge sampling, reset mismatches).
- Boolean Proofer Agent: Performs formal verification on combinational logic to validate candidate fixes and prevent regressions.

Key Mechanisms

Trace-Driven Temporal Analysis & Static Slicing: When simulation fails, the system does not rely solely on textual logs. It analyzes Value Change Dump (VCD) waveforms to identify the earliest divergence time ( $t_f$ ). It then employs static dependency slicing to compute the "cone of logic" influencing the failing signals. This isolates the failure to a minimal set of suspect code blocks (line ranges), rather than the entire file.
Localized Patching: Guided by the slicing analysis, the Debugger agent applies precise, block-level edits (read/replace) only to the suspect blocks. This avoids the instability of whole-file rewrites. A rollback rule ensures that patches are only kept if they improve the failure signature (e.g., later first failure time or fewer mismatches).
Multi-Branch Verification: The framework integrates two verification branches:
1. Trace-driven Temporal Analysis: For sequential and timing issues.
2. Formal Verification: Combines assertion-based checking (via the Asserter) and Boolean equivalence proofs (via the Boolean Proofer using SymbiYosys) to verify combinational constraints and prove correctness beyond finite simulation.

3. Benchmark: VERILOGEVAL-V2-EXT

To better evaluate industrial-grade RTL generation, the authors introduce VERILOGEVAL-V2-EXT. This benchmark extends the original VerilogEval-v2 by adding 53 new, industrial-grade design tasks, bringing the total to 209 problems.

Content: The new tasks cover complex domains such as communication protocols (UART, FIFO), advanced arithmetic, DSP kernels, and multi-cycle datapaths.
Stratification: Problems are classified into Easy, Medium, and Hard difficulty levels based on structural signals (lines of code, number of assign/always blocks, datapath width) and manual review. This allows for finer-grained analysis of failure modes.

4. Experimental Results

The authors evaluated VERI-SURE against 15 standalone LLMs (including commercial closed-source and open-weight models), single-agent feedback loops, and prior multi-agent systems (MAGE, VerilogCoder).

Performance: VERI-SURE achieved a state-of-the-art 93.30% functional Pass@1 on the VERILOGEVAL-V2-EXT benchmark, with 100% syntax success.
Hard Tasks: The framework showed its greatest advantage on "Hard" tasks, achieving an 85.07% functional pass rate, significantly outperforming the best standalone model (Gemini-3-Pro at 62.69%) and other agentic baselines.
Backbone Agnosticism: While VERI-SURE performed best with GPT-5.2, it also significantly boosted the performance of the open-weight DeepSeek-3.2 model, raising its functional pass rate from 61.24% (standalone) to 71.29%.
Ablation Studies:
- Removing the Tracing, Slicing, and Patching mechanism caused the largest performance drop (11.0% overall, 16.4% on Hard tasks), confirming that precise localization is critical.
- Removing Formal Verification reduced the Hard task pass rate by 11.9 percentage points, demonstrating its necessity for debugging deep logical bugs that simulation misses.
- The Contract mechanism contributed a 2.9% drop in overall performance when removed, highlighting its role in maintaining intent consistency.

5. Significance and Claims

The paper claims that VERI-SURE demonstrates that automated RTL design can move closer to "silicon-grade correctness" by shifting from simulation-centric evaluation to a hybrid approach combining trace-driven analysis and formal verification.

Key claims include:

Mitigation of Semantic Drift: By enforcing a structured design contract, the framework aligns agent intents and prevents the gradual reinterpretation of requirements.
Stability in Repair: The dependency-slicing-guided patching mechanism effectively reduces regression risks and repair hallucinations associated with whole-file regeneration.
Beyond Simulation: The integration of formal proofs allows the system to verify functional correctness in ways that pure simulation cannot, particularly for combinational equivalence and complex timing constraints.
Industrial Relevance: The introduction of VERILOGEVAL-V2-EXT provides a more realistic and discriminative testbed for evaluating end-to-end RTL generation capabilities, addressing the gap between academic benchmarks and industrial IP block requirements.

The authors conclude that while large-scale pretraining provides a strong foundation, the structured agentic workflow with formal verification is essential for achieving high reliability in complex, multi-cycle hardware designs.