Technical Summary: Elicitation-Augmented Bayesian Optimization (EA-BO)

Problem Statement

Bayesian Optimization (BO) is a sample-efficient framework for optimizing expensive black-box functions. While Human-in-the-Loop (HITL) BO methods aim to improve cost-efficiency by leveraging domain expertise, existing approaches typically assume experts can explicitly quantify their knowledge (e.g., by specifying prior distributions or pinpointing query locations). However, human expertise is often tacit and qualitative; humans are generally more reliable at ranking candidates than assigning calibrated numerical values.

This paper addresses the challenge of incorporating tacit expert knowledge into BO via pairwise comparisons (ranking two designs) rather than direct quantification. The core problem is to develop a method that:

1. Combines direct function evaluations (expensive, high-fidelity) with pairwise comparisons (cheap, lower-fidelity, noisy).
2. Handles both single- and multi-objective optimization under a known utility function.
3. Dynamically allocates a fixed budget between these two information sources based on their relative costs and informativeness.

Methodology

1. Surrogate Model: Mixed-Likelihood Sparse Variational GP

The authors propose a Gaussian Process (GP) surrogate model that treats both direct observations and pairwise comparisons as noisy evidence about the objective function $f$.

• Direct Observations: Modeled with a standard Gaussian likelihood: $y = f(x) + \epsilon_{eval}$.
• Pairwise Comparisons: Modeled using a probit link function. A binary judgment $d \in \{0, 1\}$ indicates whether $x$ is ranked higher than $x'$ based on the utility difference $\Delta = U(f(x)) - U(f(x'))$. The likelihood is $p(d=1) = \Phi\left(\frac{\Delta}{\sqrt{2}\sigma_{comp}}\right)$.
• Inference: The model employs Sparse Variational Inference (SVGP) to approximate the posterior. The Evidence Lower Bound (ELBO) is decomposable into separate terms for observations and comparisons.
  • The observation term admits a closed-form Gaussian solution.
  • The comparison term involves an expectation of $\log \Phi(\cdot)$. For linear utility functions, the utility difference is Gaussian, allowing the expectation to be computed via one-dimensional Gauss–Hermite quadrature. Nonlinear utilities are handled via moment matching.

2. Acquisition Function: Cost-Aware Value of Information (VoI)

The method adopts a myopic, one-step lookahead strategy to select the next action. The action space includes both direct evaluations ($x \in \mathcal{X}$) and pairwise queries ($(x, x') \in \mathcal{X}^2$).

• Objective: Maximize the cost-normalized Value of Information: $\frac{\alpha(a; D)}{c(a)}$, where $\alpha$ is the expected utility gain and $c(a)$ is the cost.
• Direct Evaluation VoI ($\alpha_{eval}$): Reduces to the Knowledge Gradient for linear utilities. Optimized using a "one-shot" reformulation with fantasy points.
• Pairwise Comparison VoI ($\alpha_{comp}$): The authors derive a closed-form fantasy posterior update under the probit likelihood for multi-output objectives with known utility. This extends the pairwise knowledge gradient (Wu and Gardner, 2026) to the multi-output case. The VoI is calculated exactly as a discrete expectation over the binary outcome, avoiding Monte Carlo approximation for the comparison step.
• Optimization: Both acquisition functions are optimized jointly with candidate inputs using the one-shot reformulation and Adam optimizer.

Key Contributions

1. Mixed-Likelihood Sparse Variational GP: A principled method for updating a GP posterior by coupling a Gaussian observation likelihood with a probit comparison likelihood, yielding a decomposable ELBO with tractable terms.
2. Cost-Aware Acquisition Rule: Derivation of a closed-form posterior mean update under probit comparisons for multi-output objectives, enabling a cost-normalized VoI acquisition function that balances direct evaluations and pairwise queries without manual scheduling.
3. Unified Framework: A single algorithm (EA-BO) that handles both single- and multi-objective optimization uniformly by treating pairwise rankings as evidence about the observable objective function values.

Experimental Results

The method, named Elicitation-Augmented Bayesian Optimization (EA-BO), was evaluated on standard single- and multi-objective benchmarks (Branin, Hartmann6, BraninCurrin, VLMOP2) with linear and Chebyshev utility functions.

• Performance vs. Single-Source: EA-BO approaches the convex hull of the trajectories of individual information sources.
  • When pairwise comparisons are cheap (e.g., 5x cheaper than evaluations), EA-BO allocates the majority of the early budget to comparisons, achieving fast initial convergence comparable to comparison-only methods (KG-Comp).
  • As the posterior sharpens or comparisons become noisy, EA-BO shifts toward direct evaluations, recovering the asymptotic accuracy of evaluation-only BO (KG-Eval).
• Performance vs. HITL Baselines: EA-BO substantially outperforms CoExBO (Adachi et al., 2024), the closest existing HITL method using pairwise comparisons. Unlike CoExBO, which uses comparisons to guide candidate selection via an auxiliary model, EA-BO incorporates them directly as likelihood evidence.
• Sensitivity:
  • Cost Ratio: EA-BO automatically abandons comparisons when they are more expensive than evaluations and increases allocation as the cost ratio grows.
  • Noise: The method front-loads comparisons when they are informative. As comparison noise increases, the algorithm progressively shifts to direct evaluations. Interestingly, at moderate noise levels, a non-monotonic dip in performance occurs because the myopic VoI still assigns positive value to noisy comparisons before abandoning them entirely at high noise levels.

Significance and Claims

The paper claims that EA-BO offers a robust, adaptive framework for leveraging tacit human expertise in BO. Its primary significance lies in its ability to automatically balance between cheap, noisy qualitative data (rankings) and expensive, high-fidelity quantitative data (evaluations).

The authors state that the method:

• Improves sample-efficiency significantly when pairwise queries are cheaper than evaluations.
• Recovers the performance of standard BO when pairwise queries are too costly or noisy to be useful.
• Avoids the need for manual scheduling or heuristic switching between information sources.
• Provides a principled extension of pairwise knowledge gradients to multi-objective settings with known utilities.

The paper acknowledges limitations, specifically the assumption that expert judgments are governed by the true utility function $U \circ f$ (ignoring potential deviations in the expert's mental model) and the requirement that the utility function $U$ is known prior to optimization. Future work is proposed to address learnable utility functions and learnable correlations between expert mental models and the true objective.