🤖 AI

ORFS-agent: Tool-Using Agents for Chip Design Optimization

This paper introduces ORFS-agent, a modular, model-agnostic LLM-based iterative optimization agent that outperforms standard Bayesian optimization in open-source chip design flows by achieving superior performance metrics with fewer iterations and enabling flexible, interpretable multi-objective tuning through natural language objectives.

Original authors: Amur Ghose, Andrew B. Kahng, Sayak Kundu, Zhiang Wang

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amur Ghose, Andrew B. Kahng, Sayak Kundu, Zhiang Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to bake the perfect chocolate cake. You have a recipe, but it has thousands of tiny knobs you can turn: how much flour, exactly how long to mix, the precise temperature of the oven, the humidity in the room, and even the brand of chocolate chips. If you tweak just one of these slightly, the cake could turn out perfect, or it could be a dry, burnt disaster.

In the world of computer chips, engineers face this exact problem. Designing a chip is like baking a cake, but with millions of ingredients and rules. There are thousands of settings (parameters) that control how the chip is built. Getting these settings right is crucial for the chip to be fast, use little power, and fit on a small piece of silicon.

The Problem: The "Black Box" of Chip Design
Traditionally, engineers have used a method called "Bayesian Optimization" to find the best settings. Think of this as a very smart, but slightly blind, taste-tester. It tries a setting, tastes the cake, and says, "Okay, try a little more sugar next time." It's good, but it doesn't really understand the recipe. It just follows a mathematical pattern. It also needs a lot of tries (iterations) to get it right, which takes a lot of time and money.

The Solution: ORFS-agent (The "Thinking" Chef)
The authors of this paper introduced ORFS-agent. Instead of a blind taste-tester, imagine a Master Chef who can read the entire cookbook, understand the chemistry of baking, and talk to you in plain English.

This "Chef" is powered by a Large Language Model (LLM)—the same kind of AI that can write stories or answer questions. But here, the AI is given a special job:

It reads the results: It looks at the logs from the chip design process (like reading the cake's texture and taste).
It uses tools: It can call upon a "toolbox" of mathematical helpers (like a calculator or a graphing tool) to analyze the data.
It thinks: It doesn't just guess. It reasons through the data, asking itself, "Why did the chip run too hot? Maybe I need to slow down the clock speed and add more padding."
It adjusts: It changes the settings for the next round of "baking."

How It Works in Practice
The researchers tested this AI Chef on real chip designs using two different "brains" (AI models):

Sonnet 4.6: A powerful, commercial AI model.
Kimi K2.5: A powerful, open-source AI model that anyone can run on their own computers.

They asked the AI to optimize the chip for two main goals: making the wires inside the chip as short as possible (to save space) and making the clock speed as fast as possible (to make it run quicker).

The Results: Faster, Cheaper, and Smarter
The paper claims some impressive results:

Fewer Tries: The AI Chef found better settings using 40% fewer attempts than the traditional method. It's like finding the perfect cake recipe in half the time.
Better Cakes: The final chips were slightly better (about 1% to 2.7% improvement) in terms of speed, power, and size compared to the old method.
Open Source Friendly: The open-source AI (Kimi) performed almost exactly as well as the expensive commercial one. This means companies don't have to pay huge fees to use this technology; they can run it privately.
Listening to Instructions: The AI can understand natural language. If an engineer says, "I care more about speed than size, but don't let the size get more than 2% worse," the AI understands and follows that rule. Traditional methods struggle with this kind of flexible instruction.

The "Search" Tool: A Double-Edged Sword
The researchers also gave the AI a "search engine" tool, allowing it to look up documentation or research papers to help it understand the knobs better.

Early on: This helped the AI get up to speed quickly, like a chef reading a quick tip before starting.
Later on: Once the AI had learned the specific chip, searching for more info actually slowed it down or made it slightly worse. It's like a chef who keeps reading the book while the cake is in the oven, forgetting to actually check the cake.

Why This Matters
The most important part of this paper is that the ORFS-agent is a "plug-and-play" system. It doesn't need to be retrained or fine-tuned for every new chip or every new AI model. If a better AI comes out tomorrow, you can just swap the "brain" in the agent, and it immediately becomes smarter. It's like having a robot chef that can instantly upgrade its brain without needing a new body or a new recipe book.

In short, this paper shows that AI agents that can "think," use tools, and follow natural language instructions can design computer chips more efficiently and effectively than traditional mathematical methods, saving time and money in the process.

1. Problem Statement

Integrated Circuit (IC) design involves complex Electronic Design Automation (EDA) flows (from Register-Transfer Level to physical layout) governed by thousands of hyperparameters. Small adjustments in these parameters can significantly impact Power, Performance, and Area (PPA).

The Challenge: Traditional optimization methods, such as Bayesian Optimization (BO), are efficient but often lack domain knowledge, require explicitly defined objective functions, and struggle with high-dimensional, combinatorial search spaces.
The Gap: While Large Language Models (LLMs) possess broad "world knowledge" and reasoning capabilities, previous attempts to apply them to EDA often relied on fine-tuning specific models on domain data. This approach incurs high costs, requires maintenance when new foundation models are released, and limits flexibility.
Goal: Develop a model-agnostic, LLM-driven agent that can autonomously tune EDA parameters using natural language objectives, leveraging reasoning and tool use without requiring retraining.

2. Methodology: ORFS-agent

The authors introduce ORFS-agent, an iterative optimization framework integrated with OpenROAD-flow-scripts (ORFS), an open-source hardware design flow.

Core Architecture

Model-Agnostic Design: The agent acts as a wrapper around any frontier LLM (tested with Anthropic's Sonnet 3.5, Sonnet 4.6, and Moonshot's Kimi K2.5). It requires zero fine-tuning, allowing immediate adoption of newer, more capable models.
Iterative Loop: The agent operates in a loop consisting of Run, Read, Collate, Inspect, Optimize, Agglomerate, and Alter.
1. Run: Executes $K$ parallel OpenROAD jobs with distinct parameter sets.
2. Read/Collate: Gathers JSON logs and metrics (e.g., wirelength, timing slack) from partial or completed runs.
3. Inspect: Analyzes data distributions and correlations.
4. Optimize: Proposes new parameter sets for the next iteration.
5. Agglomerate: Selects the best $K$ candidates from a larger pool (e.g., maximizing diversity or quality).
6. Alter: Updates the configuration files (config.mk and SDC) for the next round.

Tool-Use Capabilities

Unlike simple prompt-based agents, ORFS-agent utilizes a function-calling interface with four categories of tools:

INSPECT: Statistical analysis tools (PCA, LOF, distribution inspection) to understand data manifolds without tokenizing raw logs.
OPTIMIZE: Algorithms to propose new points, including Gaussian Processes (GP), Expected Improvement (EI), and Latin Hypercube Sampling.
AGGLOM: Selection tools to filter candidate sets (e.g., EntropySelect, GraphSelect) to maintain diversity.
RETRIEVAL: Optional web search (Brave) and scholarly lookup (OpenAlex) to bootstrap parameter semantics or find documentation early in the optimization process.

Optimization Strategy

Natural Language Objectives: Users define goals via natural language (e.g., "minimize wirelength while keeping clock period under 1ns"). The agent translates these into weighted loss functions.
Surrogate Handling: The agent handles partial data (e.g., using Clock Tree Synthesis wirelength as a surrogate for final routed wirelength if the full flow times out).
Thinking Models: The framework leverages "thinking" models (like Sonnet 4.6 and Kimi K2.5) that generate internal reasoning traces. The authors note that while these traces are useful for debugging, they are not fed back verbatim to avoid context bloat; instead, concise decision summaries are retained.

3. Key Contributions

Framework Integration: Successfully embedded an LLM agent into the OpenROAD flow, enabling autonomous, iterative tuning of physical design parameters.
Model Agnosticism: Demonstrated that the agent works seamlessly across different LLM backends (proprietary and open-weight) without retraining, significantly reducing deployment costs and latency.
Tool-Augmented Reasoning: Showed that combining LLM reasoning with external tools (Bayesian optimization libraries, statistical analysis, and retrieval) outperforms pure black-box optimization.
Comprehensive Evaluation: Validated the system on six benchmarks (IBEX, AES, JPEG) across two technology nodes (SKY130HD and ASAP7), comparing against standard Bayesian optimization (OR-AutoTuner) and previous LLM baselines.

4. Experimental Results

The experiments were conducted on Google Cloud Platform using 25 parallel runs per iteration.

Performance vs. OR-AutoTuner (Bayesian Optimization)

Efficiency: ORFS-agent achieved superior or comparable results to OR-AutoTuner while using 40% fewer iterations (600 iterations for ORFS-agent vs. 1000 for OR-AutoTuner in 12-parameter settings).
Metrics Improvement:
- Sonnet 4.6: Improved geometric-mean normalized Wirelength ($WL$) by 1.0%, Effective Clock Period ($ECP$) by 1.3%, and co-optimization objectives by 2.7% over OR-AutoTuner.
- Kimi K2.5 (Open-Weight): Achieved nearly identical results to Sonnet 4.6 (within 0.24%), proving that open-weight models can be viable for private, cost-effective deployment.

Impact of Model Backends

Thinking Models: Moving from Sonnet 3.5 to Sonnet 4.6 (a "thinking" model) yielded significant gains: 7.5% improvement in $WL$, 3.1% in $ECP$, and 4.0% in co-optimization.
Reasoning: The "thinking" models demonstrated better adherence to constraints and more stable multi-objective trade-offs.

Retrieval Analysis

Early Convergence: Retrieval tools (WebSearch/ScholarlyLookup) accelerated early convergence (at 200 iterations), improving the geometric-mean co-optimization objective by ~0.35%.
Final Endpoint: Surprisingly, retrieval degraded the final 600-iteration endpoints by ~0.6%. The authors conclude that retrieval acts as "scaffolding" for initial parameter understanding but introduces noise or context bloat in later, fine-tuning stages.

Robustness and Constraints

Constrained Optimization: ORFS-agent successfully handled natural language constraints (e.g., "do not degrade power by more than 2%"), outperforming OR-AutoTuner which struggled with such soft constraints.
Statistical Significance: Extensive ablation studies (obfuscation, noise injection, multiple runs) confirmed that the results are robust and not due to data leakage or memorization.

5. Significance and Future Directions

Paradigm Shift: The paper demonstrates that model-agnostic, tool-using agents are a superior alternative to fine-tuning for dynamic, high-stakes engineering tasks like chip design. This allows the community to instantly leverage the latest LLM breakthroughs without retraining costs.
Interpretability: By logging reasoning summaries and decision trajectories, the agent provides an interpretable view of the optimization process, unlike "black-box" BO methods.
Cost Efficiency: The use of open-weight models (Kimi K2.5) and reduced iteration counts makes high-quality chip optimization significantly more accessible and affordable.
Future Work: The authors plan to enable dynamic tool generation (allowing the LLM to create its own tools), integrate visual feedback for multi-modal reasoning, and scale the framework to larger, more complex circuits.

In summary, ORFS-agent represents a significant step forward in AI-driven EDA, proving that LLMs equipped with reasoning and tool-use capabilities can outperform traditional optimization algorithms in complex, real-world hardware design scenarios while offering greater flexibility and interpretability.