🤖 AI

ContractHIL-HLS: Contract-Aligned Multi-Agent Workflow with Hardware-in-the-Loop Feedback for HLS Design

This paper introduces ContractHIL-HLS, a contract-aligned multi-agent workflow that integrates structured natural language-to-contract translation with hardware-in-the-loop feedback to significantly improve pass rates in HLS design tasks and optimize board-level post-quantum cryptography accelerators.

Original authors: Jingbo Zhang, Haoxiang Sun, Wenbo Wang, Wenbo Zhang

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Jingbo Zhang, Haoxiang Sun, Wenbo Wang, Wenbo Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a brilliant, hyper-fast robot how to build a complex machine, like a custom video game console or a super-secure vault. You give the robot instructions in plain English, like "Make it fast and secure." The robot, powered by a Large Language Model (an AI that reads and writes code), tries to build it. But here's the catch: the robot is great at writing the idea of the code, but it often forgets the tiny, boring rules that real machines need to actually work, like how much power they use or if the parts fit together. This is the world of High-Level Synthesis (HLS), where humans try to use AI to turn simple ideas into actual hardware designs for chips. The big question isn't just "Can the AI write code?" but "Can the AI build something that actually works on a real board without us having to fix every single mistake by hand?"

This is where a team of researchers from Beijing University of Technology steps in with a new idea called ContractHIL-HLS. Think of it as a "smart project manager" for the AI. Instead of just letting the AI chat back and forth with you, they force the AI to sign a formal "contract" first. This contract is a strict, structured list of rules: "Here is exactly what the machine must do, here are the things it absolutely cannot do, and here is how we will test it." If the AI tries to break the rules, the system catches it immediately. Even cooler, they connect this process to real hardware. If the AI designs a chip that runs too hot or is too slow, the real machine sends a "failure report" back to the AI, which then has to rewrite the design to fix it. It's like a video game where the character doesn't just guess the solution; they get hit by a trap, see exactly why they failed, and try again with a better plan.

The Problem: AI is Great at Chatting, Bad at Building

Usually, when you ask an AI to design hardware, it acts like a creative writer. It generates code based on your prompt, but it often forgets the "fine print." Maybe it uses a feature that the hardware chip doesn't support, or it creates a design that looks good on paper but crashes when you try to run it. Previous methods tried to fix this by having the AI play different roles, like "Architect" or "Coder," but the AI would often forget what the Architect said by the time the Coder started working. The design intent got lost in the conversation.

The researchers realized that instead of relying on a long, messy conversation, they needed a structured contract. Imagine you are ordering a custom pizza. Instead of just saying, "I want a delicious pizza," you fill out a form: "Crust: Thin. Toppings: Pepperoni only. No mushrooms. Must be ready in 10 minutes." This form is the "contract." In the world of chip design, this contract translates your vague idea into a strict set of rules that the AI must follow.

The Solution: The Three-Agent Workflow

The team built a system with three specialized "agents" (AI helpers) that work together like a well-oiled machine:

The Contract Agent: This agent takes your messy, natural language request (e.g., "Make a secure message system") and turns it into that strict "pizza order" contract. It defines the interface, the constraints (what is forbidden), and the rules for testing.
The HTML Agent: This agent takes the contract and turns it into a clean, structured webpage (HTML). Why a webpage? Because it's easy for humans to read, easy for the next AI agent to understand, and easy for computer scripts to check. It acts as a permanent, unchangeable record of the rules.
The Hardware-in-the-Loop (HIL) Agent: This is the builder. It looks at the HTML contract, writes the code, and then—here is the magic part—actually tests it on real hardware or simulates it. If the hardware says, "Hey, this design uses too much memory," the HIL agent sees that evidence, reads the contract, and rewrites the code to fix it. If it can't fix it, the system knows to "roll back" and try a different approach.

What They Found: Small Wins and Big Leaps

The team tested this system in two very different ways to see if it really worked.

1. The "Small Design" Test (HLS-Eval)
First, they tested the system on 94 small, standard coding tasks. They compared three methods:

Direct: Just asking the AI without any special rules.
Contract: Using the structured contract but without the real hardware feedback.
ContractHIL-HLS: Using the full system with the contract and hardware feedback.

The results were clear. The "Direct" method passed the tests about 64.0% of the time. When they added the Contract, the success rate jumped to 70.2%. The full system reached 70.4% for single attempts and 76.6% if you let the AI try five times. This proves that simply forcing the AI to follow a structured contract makes it much more reliable, even before it touches real hardware.

2. The "Real World" Test (PQC Case Study)
Small tests are great, but do they work for big, complex systems? To find out, the team built a secure message accelerator using Post-Quantum Cryptography (PQC)—a super-secure way to send messages that even future quantum computers can't crack. This is a massive engineering challenge involving memory, timing, and power.

They started with a single design (one "bitstream") that took 207.3 ms to process a message. The system analyzed the hardware feedback and realized the design was too crowded and slow. Following the contract rules, the AI split the design into two separate parts (a "dual-bitstream" organization). One part handled the sending, and the other handled the receiving.

The result? The new design processed messages in just 52.4 ms. That is a massive speedup! It also used less power and passed all the timing checks. The system successfully used the "contract" to guide the AI from a slow, single design to a fast, split design, all while keeping the security rules intact.

Why This Matters

The paper shows that we don't need to wait for AI to become a "genius" on its own. Instead, we can build a workflow where the AI is guided by a strict, unbreakable contract and corrected by real-world evidence. The researchers found that this approach doesn't just make the AI write code faster; it makes the code work.

However, the paper is careful to note that this isn't a magic wand for everything. In their tests, there was one group of tasks (called "machsuite") where the AI still failed completely, even with the contract. This suggests that while the contract helps the AI follow rules, it can't fix it if the AI simply doesn't know the underlying math or logic yet.

In short, ContractHIL-HLS is a bridge between the messy world of human ideas and the rigid world of hardware chips. It turns the AI from a creative writer into a disciplined engineer, ensuring that when we ask for a secure, fast machine, we actually get one that works.

Technical Summary: ContractHIL-HLS

Problem Statement

While Large Language Models (LLMs) have demonstrated the ability to generate hardware code, practical High-Level Synthesis (HLS) engineering requires workflows that maintain design intent across iterations, ensuring alignment with Electronic Design Automation (EDA) feedback, host control, memory ownership, and Field-Programmable Gate Array (FPGA) deployment measurements. Existing benchmarks (e.g., Chip-Chat, RTLLM, HLS-Eval) show models can generate valid code, but they often fail to establish a workflow where design intent remains traceable against tool feedback and deployment results. Current approaches typically rely on conversational role prompts or conversation history as state, which can lead to constraints introduced by an "architect" prompt being omitted by a subsequent "coder" prompt. Furthermore, existing LLM-assisted HLS work remains primarily kernel-centric, lacking coverage for bitstream partitioning, live-state ownership, and cross-stage security boundaries.

Methodology

The paper proposes ContractHIL-HLS, a contract-aligned multi-agent workflow designed to transform imprecise natural language requests into traceable HLS implementations. The workflow is distinguished by three core mechanisms:

Structured Contract as an Execution Artifact: Instead of relying on loose conversational history, the workflow introduces a formal contract ( $C = \langle R, I, F, H, K, O, V, B \rangle$ ) that encodes requirements into explicit fields: Role/Platform, Public Interface, Functional Intent, Hard/Negative Constraints, Compatibility, Optimization Intent, Validation Evidence, and Rollback Policy. This contract serves as the semantic alignment and task-execution artifact.
Agent Decomposition by Semantic Lowering: Agents are defined by their transformation tasks rather than conversational roles:
- Contract Agent: Normalizes natural language into the structured contract.
- HTML Agent: Renders the contract into a stable, inspectable HTML artifact containing sections, tables, and checklists. This artifact is dual-readable by humans and scripts.
- Hardware-in-the-Loop (HIL) Agent: Implements and revises the design using the HTML artifact combined with measured hardware evidence.
Hardware-in-the-Loop Feedback Loop: The workflow integrates feedback from HLS tools (Vitis), implementation tools (Vivado), and board runtime measurements (PYNQ) directly into the generation and decision loop. This extends LLM-assisted design from kernel code toward system- and board-level closure. The loop includes bounded implementation requests, low-cost rejection checks (parsing, compilation, testbench), synthesis/implementation checks, and a keep-or-rollback decision based on measured evidence (e.g., Worst Negative Slack, resource utilization, power, runtime).

Key Contributions

The paper outlines three primary contributions:

Structured Contract as an Execution Artifact: Requirements are encoded as explicit fields (interface, constraints, validation, optimization, rollback) to align generation with a shared engineering object rather than repeated prompt text.
Persistent HTML and HIL Feedback Loop: The contract is rendered into structured HTML, and tool outputs (HLS, Vivado, deployment metrics) are fed back as first-class workflow states to drive implementation and rollback decisions.
Agent Coordination from Benchmarks to Engineering Cases: The workflow demonstrates that a contract-aligned approach improves outcomes on 94 local HLS-Eval tasks and successfully carries the design loop into a board-tested Post-Quantum Cryptography (PQC) secure-message system.

Results

HLS-Eval Evaluation (Small-Design Setting)

Evaluated on 94 locally executable HLS-Eval tasks using a Vitis HLS toolchain:

Baseline vs. Contract: The introduction of the structured contract alone improved the estimated single-sample testbench pass rate (pass@1) from 64.0% (Direct prompt) to 70.2%.
Full Workflow: The complete ContractHIL-HLS flow achieved a 70.4% pass@1 and 76.6% pass@5.
Token Efficiency: The "Contract" setting used fewer tokens (3555) than the "Direct" setting (3741) by replacing loosely repeated instructions with named fields, suggesting improved reliability without simply increasing prompt length.
Category Analysis: Improvements were observed across most benchmark families (e.g., chstone, polybench, rosetta), though the machsuite group remained at 0.0% pass rate, indicating that the workflow improves reliability for plausible implementations but cannot compensate for missing algorithmic capabilities.

PQC Case Study (Board-Level Deployment)

Validated on a Post-Quantum Cryptography (ML-KEM/ML-DSA) secure-message accelerator:

Performance: The workflow successfully transitioned from a monolithic single-bitstream design to a dual-bitstream organization (splitting KEM and DSA/XOR logic). This reduced the average text runtime from 207.3 ms to 52.4 ms.
Efficiency: The conservative Energy-Delay Product (EDP) improved from 171.3 mJ·s to 23.6 mJ·s.
Timing and Resources: The split design resolved timing violations (WNS improved from 1.570 ns in the monolithic image to 0.769 ns and 0.171 ns in the split images) and redistributed memory pressure, with both images achieving positive routed timing slack.
Validation: The workflow preserved receiver-boundary rules and decrypted-message verification, ensuring that state ownership was visible and correct.

Significance and Claims

The paper claims that ContractHIL-HLS addresses the gap between isolated code generation and traceable workflow execution. Its significance lies in:

State Alignment: Demonstrating that converting implicit natural language requirements into stable, inspectable structures (contracts/HTML) significantly improves the reliability of hardware code generation before any board feedback is available.
System-Level Closure: Proving that the same contract-aligned workflow can scale from kernel-level benchmarks to complex, board-tested systems involving bitstream partitioning, host orchestration, and security boundaries.
Evidence-Driven Revision: Establishing a mechanism where tool output (QoR, timing, power) becomes a first-class artifact that drives rollback decisions, preventing the silent propagation of designs that fail specific gates (e.g., timing or resource constraints).

The authors maintain a modest stance, noting that the workflow is a state-alignment mechanism that improves reliability when the model is close to a plausible implementation, rather than a universal solution for all algorithmic gaps. The PQC case serves as a deployment stress test for system completeness rather than a claim of primitive-level latency superiority.