📊 statistics

Bayesian Optimization for Function-Valued Responses under Min-Max Criteria

This paper proposes Min-Max Functional Bayesian Optimization (MM-FBO), a framework that optimizes expensive black-box functions with functional responses by using functional principal component analysis and Gaussian processes to directly minimize the maximum error across the functional domain.

Original authors: Pouya Ahadi, Reza Marzban, Ali Adibi, Kamran Paynabar

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pouya Ahadi, Reza Marzban, Ali Adibi, Kamran Paynabar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a professional chef tasked with creating a new recipe for a signature sauce.

Most people would judge your sauce by one single number: "How good is it on average?" This is what most current AI optimization methods do. They look for a "scalar" value—a single score. If your sauce is delicious most of the time but occasionally tastes like pure salt, an "average" score might still look good.

But what if you are cooking for a very picky food critic? That critic doesn't care about the average; they care about the worst moment. If even one spoonful is terrible, the whole meal is a failure. This is what this paper calls "Min-Max Optimization." They aren't looking for the best average performance; they are looking to minimize the worst-case mistake.

Here is how the researchers solved this problem using a new method they call MM-FBO.

1. The Problem: The "Wavy Line" Challenge

In many scientific fields, the "answer" isn't just a single number; it’s a curve.

In Photonics (Light Science): You don't just want a material that reflects light well at one specific color; you want it to reflect light perfectly across the entire rainbow.
In Chemistry: You don't just want a chemical reaction to happen; you want the concentration of the chemical to follow a specific "shape" over time.

If you only optimize for the "average" shape, you might get a curve that is mostly correct but has a massive, ugly "dip" or "spike" somewhere. In science, that dip could mean a device fails or a chemical reaction explodes.

2. The Solution: The "Musical Orchestra" Approach

How do you manage a complex, wavy line? The researchers used a mathematical trick called a Karhunen–Loève expansion.

Think of a complex musical symphony. A symphony is a single, complicated "wave" of sound, but it is actually made up of many different instruments playing different parts: the violins, the cellos, the flutes, and the drums.

Instead of trying to model the entire complicated "symphony" (the whole curve) at once—which is mathematically exhausting—the researchers broke the curve down into its "instruments" (individual modes). They modeled each "instrument" separately using Gaussian Processes (a type of smart AI that is very good at guessing patterns and measuring its own uncertainty).

3. The Strategy: The "Smart Explorer"

Once they had their "orchestra" of models, they needed a way to decide where to "test" the next design. They created a special Acquisition Function.

Imagine you are exploring a dark, mountainous landscape to find the smoothest path. You have two instincts:

Exploitation (The Perfectionist): "I think the path is smooth over there; let's go check that spot again to be sure."
Exploration (The Adventurer): "I have no idea what's behind that giant fog bank; let's go see if there's a huge cliff there."

The researchers' formula perfectly balances these two. It looks for the spot where the predicted worst-case error is lowest (Perfectionist) but also subtracts a bonus for spots where the uncertainty is high (Adventurer). This prevents the AI from getting "stuck" thinking a path is safe just because it hasn't looked closely enough at the dangerous parts.

4. Does it work?

They tested this on two very difficult real-world problems:

Designing Metasurfaces: Creating tiny structures that can manipulate light. Their method found designs that worked across the whole spectrum of light much faster and more reliably than old methods.
Chemical Infiltration: Predicting how chemicals soak into materials. Again, their "Min-Max" approach ensured the chemical process stayed on track without unexpected, dangerous deviations.

Summary in a Nutshell

Old Way: "Find a design that is good most of the time." (The Average Approach)
New Way (MM-FBO): "Find a design where even the worst moment is as good as possible." (The Robust Approach)

By breaking complex curves into simple "musical parts" and balancing "perfectionism" with "adventure," they've created a tool that helps scientists build much more reliable and high-performing technology.

Technical Summary: Min-Max Functional Bayesian Optimization (MM-FBO)

1. Problem Statement

Standard Bayesian Optimization (BO) is designed to optimize scalar-valued black-box functions. However, in many scientific and engineering domains—such as nanophotonics, materials science, and chemical modeling—the output is functional (e.g., a spectrum over wavelengths or a concentration profile over time).

Existing approaches often "scalarize" these functional outputs by minimizing the integrated deviation (average error) from a target. The authors argue that this is insufficient for robust design, as a solution might perform well on average but fail catastrophically at specific points in the functional domain. The core problem addressed is the functional min-max optimization problem:
$\min_{x \in X} \max_{\lambda \in \Lambda} [f(x, \lambda) - f^*(\lambda)]^2$
where the goal is to find a design $x$ that minimizes the worst-case squared deviation from a target function $f^*(\lambda)$ across the entire domain $\Lambda$ .

2. Methodology

The authors propose Min-Max Functional Bayesian Optimization (MM-FBO), a framework that avoids expensive Monte Carlo simulations by using analytical closed-form derivations. The methodology follows five key steps:

Functional Representation (Karhunen–Loève Expansion): To handle the infinite-dimensional nature of functions, the response surface $f(x, \lambda)$ is modeled as a Gaussian Process (GP) with a separable covariance kernel. Using Mercer’s Theorem, the authors decompose the function into a countable sum of independent coefficient processes $\alpha_m(x)$ and orthonormal eigenfunctions $\phi_m(\lambda)$ . This reduces the problem from modeling a surface to modeling $M$ independent scalar GPs.
Posterior Inference: After observing functional responses at $n$ design points, the authors project these observations onto the basis functions to obtain coefficient data. They then perform independent GP regression for each coefficient mode to reconstruct the posterior mean $\mu_f(x, \lambda)$ and variance $\sigma^2_f(x, \lambda)$ of the full functional response.
Squared Deviation Distribution: Because the posterior of $f$ is Gaussian, the squared deviation $d(x, \lambda) = [f(x, \lambda) - f^*(\lambda)]^2$ follows a scaled noncentral chi-squared distribution. The authors derive the exact closed-form expressions for its first two moments (mean $\mu_d$ and variance $\sigma^2_d$ ) without needing sampling.
Tractable Acquisition Function: The authors design a novel acquisition function $\alpha(x)$ $α (x)$ that balances two competing needs:
1. Worst-case Exploitation: $\sup_{\lambda \in \Lambda} \mu_d(x, \lambda)$ (minimizing the predicted maximum expected deviation).
2. Integrated Exploration: $\int_{\Lambda} \sigma_d(x, \lambda) d\lambda$ (seeking regions where the deviation process is highly uncertain).
Numerical Implementation: The continuous framework is discretized using a grid-based approach for basis extraction and quadrature for integration, ensuring the method is computationally efficient and scalable.

3. Key Contributions

New Formulation: A formalization of BO specifically for min-max functional objectives, distinguishing it from integrated-error methods.
Analytical Tractability: The derivation of closed-form moments for the squared deviation, allowing for an acquisition function that does not rely on computationally expensive Monte Carlo approximations.
Theoretical Guarantees: The paper provides rigorous proofs for:
- Truncation error bounds (how much information is lost by using $M$ modes).
- Posterior reconstruction accuracy.
- Consistency: A proof that the acquisition function and its minimizers converge to the true min-max solution as the surrogate becomes accurate.
Adaptive Strategy: An adaptive trade-off parameter $\kappa$ that manages the exploration-exploitation balance throughout the optimization process.

4. Results

The authors validated MM-FBO through synthetic benchmarks and two complex physics-based case studies:

Synthetic Benchmarks: Tested on ODEs, nonlinear predator-prey models, and heat diffusion PDEs. MM-FBO consistently achieved lower regret, faster convergence, and higher stability (tighter interquartile ranges) than standard BO (which optimized a scalarized version of the objective) and static space-filling designs.
Case Study 1 (Metasurface Design): In designing photonic structures for broadband scattering, MM-FBO successfully identified geometric parameters that produced a uniform scattering spectrum. It significantly outperformed baselines in "Time-to-Threshold" (TT) metrics, reaching target accuracy much faster.
Case Study 2 (Vapor-Phase Infiltration): In modeling chemical infiltration in polymers, the method effectively managed the complex reaction-diffusion dynamics, achieving much lower final regret and better sample efficiency.

5. Significance

This work is significant because it provides a principled mathematical bridge between Functional Data Analysis (FDA) and Bayesian Optimization. By moving from "average-case" to "worst-case" optimization, it enables the design of highly robust systems in high-stakes engineering fields (like nanophotonics and materials science) where local failures in a functional profile (e.g., a sudden drop in light transmission at a specific wavelength) are unacceptable. The method is computationally efficient enough to be used in real-world, expensive simulation loops.