RefEvo: Agentic Design with Co-Evolutionary Verification for Agile Reference Model Generation

1. Problem Statement

As System-on-Chip (SoC) complexity increases, the industry's "shift-left" strategy requires rapid development of high-fidelity reference models (e.g., in SystemC) for early architecture exploration and verification. While Large Language Models (LLMs) show promise in code generation, their application to hardware modeling faces three critical hurdles:

• Rigid Workflows: Static, linear generation processes fail to adapt to varying design complexities, leading to inefficiency and errors in complex logic.
• Catastrophic Forgetting: Long interaction sessions required for iterative refinement exceed LLM context windows, causing the model to lose track of critical initial specifications (e.g., register maps, protocols).
• Coupled Validation Failure: A critical reliability issue where an LLM generates both the Design Under Test (DUT) and the Testbench (TB). The TB is often inadvertently tailored to match the DUT's hallucinations, resulting in a false-positive "PASS" despite the model being flawed. Current static verification flows cannot detect this shared hallucination.

2. Methodology: The RefEvo Framework

RefEvo is a dynamic multi-agent framework designed to treat hardware modeling as a co-evolutionary process rather than a static generation task. It comprises three core innovations:

A. Dynamic Design Planner (Agent 1)

Instead of a fixed workflow, a "Brain" agent autonomously analyzes the semantic complexity of the design specification.

• Function: It evaluates the design across three dimensions: Interface Protocol (e.g., AXI vs. GPIO), State Space (e.g., FSM complexity), and Concurrency (e.g., multi-clock domains).
• Output: It generates a structured Execution Plan (JSON) that decomposes tasks and selects the appropriate routing strategy, ensuring simple logic is handled efficiently while complex systems receive tailored workflows.

B. Symbiotic Verification Loop (Agents 2, 3, & 4)

This is the core mechanism addressing "Coupled Validation Failure." It employs a Dialectical Arbiter to oversee a symbiotic relationship between generation and verification.

• Dual-Generator Setup: The Modeler Agent generates the SystemC Reference Model, while the Verifier Agent constructs the Testbench (TB) concurrently.
• Dialectical Arbitration: A dedicated Arbiter Agent acts as the central authority, comparing simulation traces against the immutable Specification Oracle. It resolves mismatches via four routing paths:
  1. Syntax Repair: Fixes compilation errors.
  2. Design Refinement (Red Path): If the model violates the spec, the Arbiter directs the Modeler to fix the logic.
  3. TB Realignment (Blue Path): If the model matches the spec but the TB fails (indicating a hallucination in the verification logic), the Arbiter directs the Verifier to correct the TB. This is the key mechanism for breaking shared hallucinations.
  4. Success: Confirms functional equivalence and outputs the Golden Model.

C. Spec Anchoring Context Management

To prevent catastrophic forgetting during long-turn interactions:

• The Anchor (Immutable): The initial Design Specification is pinned at the top of the context window and never evicted, serving as the ground-truth Oracle.
• The Summary (Compressed): Historical interaction rounds are summarized to preserve the reasoning chain while saving tokens.
• The Workspace (Dynamic): Recent error logs and code snippets are processed here.
  This strategy ensures 100% recall of critical constraints regardless of session length.

3. Key Contributions

1. Agile Dynamic Planning: Introduction of a planner that autonomously decomposes specifications based on semantic complexity, moving beyond rigid linear workflows.
2. Co-Evolutionary Verification: A novel dialectical mechanism that validates both the model and the testbench against a specification oracle, effectively solving the "Coupled Validation Failure" problem where static flows fail.
3. Spec Anchoring Strategy: A context compression technique that guarantees lossless recall of specifications, enabling long-horizon reasoning without catastrophic forgetting.
4. Comprehensive Benchmarking: Evaluation on 20 diverse hardware modules (from simple logic to complex protocols like AXI DMA and FFT) using five state-of-the-art LLMs.

4. Experimental Results

RefEvo was evaluated against four baselines (Naive, Flow Only, FixedTB, and RefEvo) across five LLMs (including Gemini-2.5-Pro, GPT-5.1, and Claude-Opus-4.1).

• Success Rate: RefEvo achieved a 95% pass rate on the benchmark, significantly outperforming static baselines (which ranged from 0% to 35%). Notably, it resolved complex modules (e.g., fpu_div, keccak) that consistently failed in baseline modes.
• Ablation Study:
  • Moving from "Flow Only" to "FixedTB" eliminated syntax errors but left functional failures.
  • Enabling "Co-Evolution" (RefEvo) significantly reduced functional failures, proving that many errors were due to incorrect verification logic (TB hallucinations) rather than flawed models.
• Efficiency: The Spec Anchoring strategy reduced token consumption by an average of 71.04%. For complex designs, this resulted in absolute savings of over 73,900 tokens per session while maintaining 100% specification recall.
• Robustness: The framework demonstrated model-agnostic improvements, consistently boosting pass rates across all tested LLMs regardless of their baseline capabilities.

5. Significance

RefEvo represents a paradigm shift in LLM-aided EDA. By moving from static code generation to a dynamic, co-evolutionary agent system, it addresses the fundamental reliability issues of hardware modeling.

• Reliability: It solves the "sycophantic" validation problem where LLMs validate their own hallucinations, ensuring that the generated reference models are truly "golden."
• Scalability: The context management strategy makes high-fidelity modeling feasible for complex SoCs without incurring prohibitive token costs.
• Impact: The framework paves the way for fully automated, high-fidelity SoC verification, enabling more efficient hardware-software co-design workflows in the "shift-left" era.