Technical Summary: HLS-Seek

Problem Statement

High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into Register Transfer Level (RTL) hardware. The Quality of Results (QoR)—specifically execution latency and resource utilization (LUTs, DSPs, BRAMs, FFs)—is critical for real-world accelerator deployment. However, QoR is determined by microarchitectural structures, pragma configurations, and code optimizations that cannot be accurately inferred from software code analysis alone; it requires expensive synthesis and implementation evaluation.

Existing Large Language Model (LLM) approaches for HLS code generation primarily focus on functional correctness (generating compilable, runnable code) but are QoR-agnostic. They treat all compilable designs equally, regardless of their hardware performance. While Reinforcement Learning (RL) with real synthesis feedback (synthesis-in-the-loop) has shown promise in the RTL domain, applying it to HLS is prohibitively expensive. A single HLS synthesis run can take minutes to hours, and RL training typically requires thousands of evaluations, making the approach infeasible at scale.

Methodology

The authors propose HLS-Seek, a QoR-aware Natural Language to HLS framework that replaces expensive real-synthesis RL with a comparative proxy reward model enhanced by an uncertainty-aware switching mechanism.

1. Core Insight

The authors observe that RL for HLS does not require absolute QoR values (exact latency/resource numbers) from synthesis. Instead, it only requires relative comparisons between candidate designs to estimate group-relative advantages. This insight allows for the training of a lightweight proxy model to predict which of two designs is better, bypassing the need for synthesis during most training steps.

2. Three-Stage Training Pipeline

HLS-Seek employs a three-stage training process:

• Stage 1: Diversity-Oriented Supervised Fine-Tuning (SFT): The base LLM is fine-tuned on a broad HLS corpus with QoR descriptions to expose the model to diverse pragma configurations and coding patterns, establishing a foundation in HLS syntax.
• Stage 2: Cold-Start SFT with Reasoning Traces: Using a curated dataset of Pareto-proximal designs (optimal trade-offs) augmented with Chain-of-Thought (CoT) reasoning traces generated by commercial reasoning models, the model learns structured hardware reasoning and how to select optimal pragma configurations.
• Stage 3: Group Relative Policy Optimization (GRPO): The model is refined using GRPO with a four-component reward signal:
  1. Reasoning Format: Ensures structured output with CoT traces.
  2. Compilation: Checks for syntactic correctness (GCC compilation).
  3. Functional Correctness: Verifies semantic correctness against testbenches.
  4. Proxy QoR Reward: A comparative reward derived from a Siamese network that predicts pairwise dominance.

3. Comparative Proxy Reward Model

• Architecture: A Bradley-Terry Siamese architecture that takes two HLS designs as input and outputs a probability of one dominating the other.
• Training: Trained on pairwise comparisons from Design Space Exploration (DSE) results, distinguishing between full Pareto dominance (better on all 5 metrics) and latency superiority.
• Uncertainty-Aware Switching: To prevent "reward hacking" (where the policy exploits blind spots in the proxy model), the system employs Monte Carlo (MC) dropout during inference to estimate uncertainty.
  • If a candidate design has high uncertainty, the system triggers real Vitis HLS synthesis to obtain the ground-truth QoR.
  • These real synthesis results are fed back to online update the proxy reward model, continuously expanding its coverage and reliability.
  • This mechanism ensures the proxy remains accurate while keeping the synthesis trigger rate low (target <5% at convergence).

4. Data Curation

The framework utilizes a Pareto-Proximal Quality-Diversity Sampling strategy. Instead of using only strict Pareto-optimal designs (which are too few), it includes designs near the Pareto front, ranked by their Euclidean distance to the front, ensuring the model learns fine-grained resource-latency trade-offs.

Key Contributions

1. Relative Reward Insight: Identification that HLS RL requires only relative QoR signals, not absolute synthesis results, enabling the use of a comparative proxy model.
2. Uncertainty-Aware Proxy System: A novel mechanism combining a Siamese comparative reward model with MC dropout uncertainty estimation and online updates via selective real synthesis. This prevents reward hacking and ensures self-improvement.
3. HLS-Seek Framework: A complete NL-to-HLS pipeline achieving state-of-the-art performance with only 7B parameters, utilizing a three-stage training strategy (Diversity SFT $\to$ Reasoning Cold-Start SFT $\to$ Proxy Reward GRPO).

Experimental Results

Evaluated on the HLS-eval benchmark and a ForgeHLS test set:

• Syntax and Functional Correctness:

  • Achieved 81.5% pass@1 syntax correctness and 81.4% pass@5 functional correctness on HLS-eval.
  • Outperformed frontier models (including GPT-5.1, DeepSeek-V3.2, and Qwen3-235B) and specialized HLS baselines (SAGE-HLS, Gai et al.) despite having significantly fewer parameters (7B vs. 235B+ or larger code models).
  • Specifically, it surpassed GPT-5.1 by +2.3 percentage points in functional pass@5 on HLS-eval.

• Quality of Results (QoR):

  • Achieved the lowest latency on 16 out of 30 kernels.
  • Pareto-dominates HLS-specific baselines (SAGE-HLS and Gai et al.) on 9 kernels (5 vs. SAGE-HLS, 4 vs. Gai et al.) with only 1 reverse win.
  • Demonstrated the ability to perform algorithmic code restructuring (e.g., scalar expansion and loop fission) rather than just pragma insertion, leading to significant latency reductions (e.g., 4.1x reduction in GEMVER).

• Training Efficiency:

  • Achieved an 8.5x speedup in training compared to real-reward RL.
  • In a 73-hour wall-clock budget, HLS-Seek completed 12,500 training steps, whereas a real-reward approach completed only 1,460 steps.
  • The proxy reward mechanism maintained high accuracy (99.53% Pareto-dominance accuracy) while reducing synthesis overhead to below 5% at convergence.

Significance and Claims

The paper claims that HLS-Seek demonstrates that domain-specific RL with learned QoR rewards is a viable path toward fully automated, hardware-efficient HLS code generation. By decoupling the need for absolute synthesis results from the RL training loop, the authors overcome the scalability bottleneck of synthesis-in-the-loop approaches.

The work highlights that:

1. QoR-awareness is achievable without expensive synthesis-in-the-loop by leveraging relative comparisons.
2. Small models (7B) can outperform massive frontier models in specialized hardware generation tasks when trained with appropriate QoR-aware signals and reasoning traces.
3. Algorithmic restructuring is a key capability learned through this process, going beyond simple pragma tuning to fundamentally optimize code structure for hardware parallelism.

The authors conclude that their uncertainty-aware proxy reward system successfully balances the trade-off between training speed and reward fidelity, enabling the generation of hardware designs that are not only functionally correct but also optimized for latency and resource usage.