Finding the Needle in a Haystack: Test-Time Analog Circuit Representation Adaptation for Bayesian Optimization

Technical Summary: Test-Time Analog Representation Adaptation for Bayesian Optimization (TTARO)

Problem Statement
Analog circuit topology design is a challenging optimization problem characterized by a discrete, structured search space and expensive evaluation costs (simulation, sizing, constraint checking). Bayesian Optimization (BO) is a sample-efficient framework often used to navigate this space. However, existing representation-based BO methods typically rely on fixed circuit embeddings generated by pre-trained encoders (e.g., graph neural networks) before the optimization begins.

The core issue identified is a mismatch between representation learning and optimization objectives. Encoders are usually trained to reconstruct circuit structure or capture topology semantics, not to organize circuits according to the specific Figure of Merit (FoM) being optimized. Consequently, the latent space geometry may not reflect performance similarity; high-performing circuits may be scattered, while structurally similar but low-performing circuits may be clustered. Furthermore, methods like Deep Kernel Learning (DKL) learn a feature transformation only from an initial set of evaluated designs and freeze it, failing to incorporate new FoM evidence accumulated during the search.

Methodology: TTARO
The paper introduces Test-Time Analog Representation Adaptation for Bayesian Optimization (TTARO), an online deep-kernel BO framework designed to align the representation space with the optimization objective dynamically.

• Framework Overview: TTARO starts with a finite bank of candidate circuit topologies mapped to fixed initial representations ($h_i$) by a pre-trained encoder.
• Online Adaptation: Unlike fixed-representation BO or standard DKL, TTARO treats the feature transformation as a learnable parameter that evolves throughout the BO process. At each iteration $t$:
  1. The system observes the FoM labels ($y_i$) of circuits evaluated so far.
  2. It jointly trains a nonlinear feature map (a two-layer MLP) and a Gaussian Process (GP) surrogate using the current dataset $D_t$.
  3. The feature map transforms the fixed initial representations into an adapted latent space ($z_i^{(t)}$) that is specifically aligned with the observed FoM distribution.
  4. The GP surrogate, defined over these adapted representations, computes acquisition scores (using EI, UCB, or TS) to select the next candidate.
  5. The new evaluation is added to the dataset, and the process repeats, retraining the feature map and surrogate with the expanded data.
• Surrogate Structure: The framework supports both linear and Radial Basis Function (RBF) kernels. The adaptation of the feature map $\phi_{\theta_t}$ directly modifies the covariance structure of the GP kernel, ensuring that circuits with similar FoMs are placed closer together in the adapted space as more data becomes available.

Key Contributions

1. Novel Framework: The introduction of TTARO, the first framework to explore online, objective-aware adaptation of pretrained circuit embeddings specifically for analog circuit topology search.
2. Generalizability: Demonstration that TTARO is compatible with various acquisition functions (Expected Improvement, Upper Confidence Bound, Thompson Sampling) and kernel types (Linear, RBF).
3. Comprehensive Evaluation: A large-scale study involving 160 principal configurations across two public benchmarks (Ckt-Bench-101 and Ckt-Bench-301), eight encoder/benchmark pairings, and multiple kernel/acquisition settings, evaluated over 20 random seeds each.

Results
The paper reports extensive experimental results comparing TTARO against:

• Fixed GP: BO over fixed pretrained embeddings.

• DKL: Deep Kernel Learning where the transformation is learned once from initial data and frozen.

• GP-Oracle: An upper-bound baseline where the feature map is trained on all available FoM labels before BO begins (non-deployable but informative).

• Performance Gains: Across 40 encoder/kernel/acquisition settings, TTARO reduced the Regret Area Under the Curve (AUC) by an average of 15.2% relative to Fixed GP and 20.7% relative to DKL.

• Specific Benchmarks:

  • On Ckt-Bench-101, TTARO reduced average regret AUC by 19.6% compared to GP.
  • On Ckt-Bench-301, the reduction was 12.2% compared to GP.
  • In specific configurations (e.g., D-VAE encoder with Linear/EI on Ckt-Bench-101), regret AUC reductions reached as high as 46.7%.

• Comparison with DKL: While DKL showed mixed results (sometimes degrading performance on larger, heterogeneous search spaces like Ckt-Bench-301), TTARO consistently avoided this degradation by continuously updating the representation.

• Geometry Alignment: Visualizations (UMAP) and KNN metrics indicate that TTARO successfully reorganizes the latent space, concentrating high-performing circuits into coherent regions, thereby improving the surrogate's ability to generalize.

Significance and Claims
The paper claims that the quality of the surrogate model in representation-based BO is critically dependent on the alignment between the kernel geometry and the optimization objective. While fixed representations and one-time learned transformations (DKL) are static, TTARO demonstrates that continually incorporating newly observed FoM labels into representation learning allows the search space to adapt to the objective. This leads to more effective acquisition decisions and improved sample efficiency.

The authors position TTARO as a general solution for the "mismatch" problem in analog design automation, arguing that as circuit evaluations accumulate, the representation must evolve to reflect the true performance landscape. The work suggests that future implementations could leverage scalable GP techniques (e.g., structured kernel interpolation, sparse variational methods) to mitigate the computational overhead of retraining the feature map at every step, though the current study focuses on the efficacy of the adaptation mechanism itself.