📊 statistics

Generative Bayesian Optimization: Generative Models as Acquisition Functions

This paper introduces a novel Bayesian optimization framework that leverages generative models trained directly on utility values to serve as acquisition functions, thereby enabling efficient large-batch optimization in high-dimensional and combinatorial spaces without relying on traditional surrogate models.

Original authors: Rafael Oliveira, Daniel M. Steinberg, Edwin V. Bonilla

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rafael Oliveira, Daniel M. Steinberg, Edwin V. Bonilla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the absolute best recipe for a cake, but you have a very strict rule: you can only taste the cake a limited number of times, and every time you taste it, the flavor might be slightly different due to random noise (like a shaky hand or a slightly different oven temperature). This is the core problem of Bayesian Optimization (BO): finding the "best" thing in a huge, messy world with very few tries.

Traditionally, to solve this, scientists use a two-step "middleman" approach. First, they build a map (a statistical model) of the cake world based on what they've tasted so far. Then, they use that map to guess where the next best spot to taste might be. Finally, they have to solve a difficult math puzzle to find the exact peak on that map.

The Problem with the Old Way:
When the "cake" is actually a complex protein design or a long text sequence, the map becomes so huge and complicated that the "middleman" step gets slow, expensive, and prone to errors. It's like trying to draw a detailed map of an entire continent just to find the best spot for a single picnic.

The New Solution: GenBO (Generative Bayesian Optimization)
This paper introduces a new strategy called GenBO. Instead of building a map and then searching it, GenBO teaches a "creative generator" (a type of AI that creates new things) to directly learn how to make the best candidates.

Here is how it works, using simple analogies:

1. The "Taste-Test" Teacher

Imagine you have a class of students (the AI model) who are trying to learn how to bake the perfect cake.

The Old Way: You give the students a textbook (the map) describing the cakes. They read the book, try to understand the theory, and then guess what to bake.
The GenBO Way: You skip the textbook entirely. You just give the students a list of cakes they baked and a simple score: "Good," "Okay," or "Bad." You tell them, "Make more cakes that taste like the 'Good' ones."

2. Learning from "Preferences" (The DPO Connection)

The paper borrows a clever trick from how we train Large Language Models (like the one you are talking to now). Usually, to teach an AI to be helpful, we show it two answers: one the human liked and one they didn't. The AI learns to prefer the "liked" one.

GenBO does something similar for optimization:

It takes two candidate solutions (e.g., two different protein sequences).
It checks which one performed better (had a higher "utility" score).
It tells the AI: "You should generate more things that look like the winner, and fewer things that look like the loser."
The AI learns this preference directly, without needing a complex map in between.

3. The "Density" Trick

Think of the AI as a machine that spits out candidates.

In the old method, the machine tries to find the single highest peak on a mountain.
In GenBO, the machine learns to sprinkle its seeds (candidates) more heavily in the valleys where the "good" cakes are found. It doesn't need to find the exact peak; it just needs to know where the "good" area is so it can drop a whole batch of seeds there at once.

Why is this a Big Deal?

The paper claims three main advantages:

Speed and Scale: Because it skips the "map-making" step, it can handle massive batches. Imagine needing to test 1,000 cake recipes at once. The old way struggles with this; GenBO just generates 1,000 variations that are likely to be good and sends them off.
Simplicity: It uses a single model to do the whole job. The old way used two models (one to map, one to generate), which meant errors in the first model would mess up the second. GenBO removes that middleman.
Handling Complexity: It works great on "combinatorial" problems—things where you are mixing and matching parts (like letters in a word or amino acids in a protein) rather than just adjusting a simple dial.

The Results

The authors tested this on two types of puzzles:

Text Optimization: Trying to find a 5-letter word that is closest to "ALOHA."
Protein Design: Trying to find the best sequence of amino acids to make a protein stable or to increase its surface area.

In these tests, GenBO performed just as well (and sometimes better) than the complex, multi-step methods, but it did it three times faster because it didn't waste time building the intermediate map.

The Bottom Line

The paper argues that you don't need a complex "surrogate" map to find the best solution. Instead, you can train a generative AI to directly "feel" what a good solution looks like by simply looking at which past attempts worked best. It's a shift from "mapping the territory" to "learning the vibe of the best spots," allowing for faster and larger-scale discovery in complex fields like biology and engineering.

Technical Summary: Generative Bayesian Optimization

Problem Statement

Bayesian Optimization (BO) is a standard approach for solving complex black-box optimization problems by utilizing probabilistic surrogate models, such as Gaussian Processes (GPs), to maximize an expected utility criterion known as an acquisition function. While effective, traditional BO faces significant scalability challenges in batch settings, particularly when dealing with high-dimensional and combinatorial design spaces (e.g., protein design).

Existing batch BO methods often rely on incrementally building candidate sets by sampling "fantasy" observations, a process that becomes computationally intractable as batch sizes grow. Alternatively, approaches that train generative models as proposal distributions typically employ a two-stage process: first fitting a surrogate (regression or classification) model to approximate the objective or acquisition function, and then training a generative model on top of it. This two-stage pipeline compounds approximation errors and increases computational costs. Furthermore, many existing generative approaches are specific to certain acquisition functions (e.g., Probability of Improvement) or require explicit reward models that introduce additional sources of error.

Methodology

The paper proposes Generative Bayesian Optimization (GenBO), a unified framework that turns generative models directly into candidate solution samplers for batch BO without intermediate surrogate models. The core insight is to train the generative model to approximate a target distribution whose density is proportional to the BO acquisition function (expected utility), effectively performing direct inference over the optimum location $x^*$ rather than the objective function $f$ .

The framework offers two primary strategies for training the generative model $q$ using noisy utility values derived from observations:

Preference-Based Learning (DPO-inspired):
Inspired by Direct Preference Optimization (DPO) for Large Language Models, this approach formulates a pairwise contrastive loss. Instead of training an intermediate reward model, the generative model is trained directly on pairs of inputs $(x_{i,1}, x_{i,2})$ and their corresponding utility values $(u_{i,1}, u_{i,2})$ .
- Robust Preference Loss (rPL): To handle the noise inherent in black-box evaluations (where the sign of utility differences may be flipped), the authors employ a robust version of the DPO loss. This loss eliminates the need for an explicit reward model and a partition function, directly optimizing the generative model to approximate the target distribution $p^*_u(x) \propto p_0(x) \exp(\frac{1}{\eta} E[u(y)|x])$ .
Divergence-Based Learning:
This approach trains the generative model to match the target distribution directly using divergence minimization.
- Forward KL: The model minimizes the Kullback-Leibler (KL) divergence between the proposal and the target. The authors derive an unbiased training objective using importance sampling weights.
- Balanced Forward KL: To address the issue where standard Forward KL fails to penalize regions with zero utility (e.g., below an improvement threshold), a balanced variant is introduced. This adds a soft penalty term derived from Bregman divergences, ensuring the model does not maintain high probability density in low-utility regions.

The framework is generalizable to any acquisition function expressible as an expected utility (e.g., Probability of Improvement, Expected Improvement, Soft Expected Improvement) and supports various utility functions including mean and softplus variants.

Key Contributions

Single-Model Framework: GenBO eliminates the need for intermediate regression or classification surrogates, reducing approximation errors and computational overhead by training a single generative model directly from noisy observations.
General Utility Encoding: The method encodes general utility functions directly into the training objectives of generative models, extending beyond preference-based feedback to general reward signals and loss functions.
Theoretical Convergence: The paper provides theoretical analysis showing that under mild regularity assumptions (e.g., models lying in a Reproducing Kernel Hilbert Space), the sequence of generative models follows distributions that asymptotically approximate the optimal target distribution. The analysis establishes that the approximation error concentrates similarly to kernel methods.
Scalability: The approach is designed to scale to large batches and high-dimensional combinatorial spaces where traditional BO and two-stage generative methods struggle.

Experimental Results

The authors evaluated GenBO on challenging sequence optimization tasks, including text optimization (minimizing edit distance to "ALOHA") and protein design (optimizing stability and solvent accessible surface area, as well as synthetic Ehrlich functions).

Performance: GenBO variants, particularly those using the Robust Preference Loss (rPL) and Balanced Forward KL (bfKL), achieved competitive or superior performance compared to strong baselines including CbAS, VSD (Variational Search Distributions), and LaMBO-2.
Efficiency: GenBO demonstrated significant runtime improvements, being approximately 3 times faster than VSD in text optimization tasks due to the elimination of the intermediate surrogate fitting step.
Robustness: In protein design tasks, GenBO variants showed rapid convergence on the SASA task, while handling the more challenging stability task comparably to other methods. The experiments highlighted that GenBO is relatively less sensitive to threshold annealing schemes compared to classifier-guided baselines.
Batch Scaling: The method successfully handled large batch sizes ( $B=128$ ) in high-dimensional protein sequence spaces ( $M=64$ ), demonstrating monotonic improvements with increased batch size.

Significance and Claims

The paper claims that GenBO offers a simpler and more scalable alternative to multi-stage guided generation methods for black-box optimization. By removing the intermediate surrogate model, GenBO reduces the pipeline to learning a single generative model, thereby cutting down computational resources and eliminating a source of approximation errors.

The authors position GenBO as a "general recipe" that unifies likelihood-free BO with modern generative modeling techniques. They emphasize that while the framework currently focuses on acquisition functions expressible as expected utilities, the theoretical underpinnings allow for extensions to other scoring rules and model types (e.g., diffusion models, flow matching). The work suggests that direct inference over the optimum location, driven by utility-based pseudo-likelihood factors, is a viable and efficient path for optimizing complex, high-dimensional design spaces.