Can an Actor-Critic Optimization Framework Improve Analog Design Optimization?

1. Problem Statement

Analog circuit design optimization is a notoriously slow and labor-intensive process. Key challenges include:

• High Simulation Cost: Evaluating a single design point requires expensive SPICE simulations.
• Complex Search Space: High-quality solutions occupy a narrow region within a vast, non-linear design space defined by coupled constraints (gain, bandwidth, phase margin, power).
• Limitations of Existing Methods:
  • Black-Box Optimizers: Traditional methods like Bayesian Optimization (BO) often lack the "judgment" human designers use to steer searches away from infeasible regions.
  • LLM Limitations: Recent Large Language Model (LLM) approaches often act as single agents that generate candidates or text. They struggle with long-context retention, can produce physically invalid designs, and lack a mechanism to critically evaluate their own proposals before expensive simulations are run.
• The Core Gap: There is a need for an optimization loop that mimics the human design review process: proposing a search direction, auditing it for feasibility, and refining the search based on evidence, rather than treating optimization as a blind sequence of samples.

2. Methodology: Actor-Critic Optimization Framework (ACOF)

The authors propose ACOF, a closed-loop framework that separates the roles of proposal and evaluation using an Actor-Critic structure adapted for analog sizing.

A. Framework Architecture

The process operates in iterative rounds, alternating between four stages: Proposal $\rightarrow$ Audit $\rightarrow$ Search $\rightarrow$ Reflection.

1. The Actor (Proposer):

   • Role: Does not output a single design point. Instead, it proposes a search region ($\tilde{\Omega}_n$) defined by ranges for all tunable parameters (e.g., transistor widths, bias currents).
   • Input: It receives a "round summary" ($e_{n-1}$) containing the best designs found so far, representative samples, and the critic's previous feedback.
   • Mechanism: Driven by an LLM (e.g., Qwen2.5, GPT-4o), it reasons about trade-offs to suggest where the next search should focus.

2. The Critic (Auditor):

   • Role: Acts as a gatekeeper and validator. It reviews the Actor's proposed region ($\tilde{\Omega}_n$) before any simulation occurs.
   • Function:
     • Validation: Ensures the proposed ranges respect Process Design Kit (PDK) bounds and hard circuit constraints.
     • Correction: If a proposal is malformed, too narrow, or too loose, the Critic repairs it to create an approved region ($\Omega^+_n$).
     • Stabilization: It prevents the search from drifting into physically impossible or unproductive areas, effectively "stabilizing" the learning process.

3. Search (Bayesian Optimization):

   • Once the Critic approves the region $\Omega^+_n$, a standard BO algorithm (using Gaussian Processes) samples candidate designs ($x_n$) only within this constrained region.
   • These candidates are evaluated using a SPICE simulator (Ngspice) to compute the Figure of Merit (FoM).

4. Reflection (Feedback Loop):

   • The results of the simulation round are compressed into a summary ($e_n$).
   • This summary includes numerical data (best FoM, valid rates) and interpretive memos from the Critic (e.g., "tighten the range for $R_6$ to align with higher-performing cases").
   • This feedback guides the Actor for the next round.

B. Key Distinction from Prior Art

Unlike "Single-LLM" approaches where the model generates candidates directly, or "Pure-BO" which searches the whole space blindly, ACOF introduces an explicit judgment layer. The Critic ensures that the search remains anchored to the feasible structure of the circuit, reducing the number of wasted simulations on invalid designs.

3. Key Contributions

• Novel Framework: Introduction of ACOF, the first framework to explicitly separate the "proposal" and "audit" roles in analog sizing using an Actor-Critic paradigm.
• Interpretability & Stability: The framework makes the search process transparent. The Critic's memos provide explicit reasoning for why search ranges are adjusted, mimicking human design review.
• Sample Efficiency: By filtering out infeasible regions before simulation, the framework significantly reduces the number of expensive SPICE runs required to find high-quality solutions.
• Model Agnosticism: The framework is agnostic to the specific LLM used, demonstrating that the structural separation of roles is more critical than the specific underlying model.

4. Experimental Results

The authors evaluated ACOF against Single-LLM (LLM + BO without a critic) and Pure-BO (BO without LLM) across four benchmark circuits (Two-stage amplifiers, Folded-cascode amplifiers) on 130nm and 180nm technology nodes.

Key Metrics & Findings:

• Design Quality (FoM):
  • ACOF improved the Top-10 Figure of Merit by an average of 38.9% over the strongest baseline.
  • Peak gains reached 70.5% on individual circuits.
  • ACOF consistently found designs that better balanced competing metrics (Gain, UGBW, Phase Margin, Power) compared to baselines which often optimized one metric at the expense of others.
• Regret Reduction:
  • ACOF reduced "regret" (the gap between the best found design and the ideal) by an average of 24.7%, indicating it reached high-quality solutions faster.
• Reliability:
  • Simulator Validity: ACOF maintained near-perfect validity (97–100%), whereas Pure-BO dropped to as low as 26.9% on complex circuits.
  • Physical Feasibility: ACOF produced a significantly higher fraction of physically meaningful designs (e.g., 85.6% vs. 19.3% for Pure-BO on the 21-parameter benchmark).
• Exploration vs. Exploitation:
  • UMAP visualizations showed that ACOF successfully explored distinct, high-performing "pockets" in the design space that baselines missed.
  • Unlike Pure-BO, which scattered samples widely (often into invalid regions), ACOF focused exploration on "workable territory."

5. Significance and Conclusion

This paper demonstrates that automated analog design can be made more robust and efficient by incorporating a "critic" mechanism.

• Paradigm Shift: It moves beyond treating LLMs as mere text generators or candidate generators. Instead, it utilizes LLMs as reasoning agents within a structured, multi-agent loop where one agent proposes and another judges.
• Practical Impact: By reducing the number of failed simulations and guiding the search toward feasible regions, ACOF offers a path toward scalable, automated analog sizing that aligns with how experienced human engineers reason about trade-offs.
• Generalizability: The success of ACOF across different LLMs (Qwen, GPT-4o, GPT-5) and circuit complexities suggests that the Actor-Critic structural pattern is a fundamental improvement for constrained optimization problems in engineering, not just a specific model trick.

In summary, ACOF bridges the gap between the intuition of human designers and the computational power of automated optimization, resulting in faster convergence to high-quality, physically feasible analog circuit designs.