Maximin Robust Bayesian Experimental Design

1. Problem Statement

Bayesian Experimental Design (BED) aims to select experiments that maximize the Expected Information Gain (EIG), typically measured by Shannon's mutual information between latent parameters ($\theta$) and observations ($x$). However, standard BED is highly brittle under model misspecification. If the assumed likelihood model $p(x|\theta, \xi)$ differs from the true data-generating process, the standard EIG can be misleading, favoring designs that appear informative under the wrong model but yield poor results in reality.

Existing approaches to robustness often treat inference (updating beliefs) and data generation (design selection) in isolation or use ad-hoc modifications (e.g., Gibbs posteriors) without a unified optimization perspective. The paper addresses the need for a framework that jointly handles robustness in both the posterior update and the data generation process under a principled decision-theoretic formulation.

2. Methodology

The authors propose a Maximin Robust Framework formulated as a two-player zero-sum game between an experimenter and an adversarial nature.

A. The Maximin Game Formulation

• The Players: The experimenter chooses a stochastic design policy $\pi$ (a distribution over designs $\xi$). Nature chooses an adversarial data-generating process $q$ from an ambiguity set $\mathcal{Q}_\rho$.
• The Constraint: The ambiguity set is defined via a Kullback-Leibler (KL) divergence budget $\rho$. Nature can perturb the nominal model $p(\theta, x|\xi)$, but the average KL divergence between the perturbed model and the nominal model must not exceed $\rho$.
• The Objective: The experimenter seeks to maximize the worst-case utility:
  $$V^* = \sup_{\pi} \inf_{q \in \mathcal{Q}_\rho} \mathbb{E}_{\pi} [U(\xi, q)]$$
  where $U$ is the information gain.

B. Derivation of the Robust Objective

By solving the inner minimization problem (finding the worst-case $q$), the authors derive that the robust expected information gain corresponds to Sibson's $\alpha$-mutual information ($I^S_\alpha$).

• Key Result: The robust objective is:
  $$\Gamma_\beta(\xi) = I^S_\alpha(\theta; x)(\xi) = D_\alpha \left( p(\theta, x|\xi) \parallel p(\theta) p_\alpha(x|\xi) \right)$$
  where $D_\alpha$ is the Rényi divergence of order $\alpha$, and $p_\alpha(x|\xi)$ is a tilted marginal distribution.
• Role of $\alpha$: The parameter $\alpha \in (0, 1)$ is the dual variable associated with the ambiguity radius $\rho$. It acts as a measure of the experimenter's confidence in the model:
  • $\alpha \to 1$: Recovers standard Shannon mutual information (nominal, well-specified case).
  • $\alpha \to 0$: Represents maximal adversarial influence, where information gain vanishes.

C. Robust Belief Update

The solution to the game implies a specific update rule for the experimenter's beliefs. Under the worst-case scenario, the posterior is not the standard Bayesian posterior but an $\alpha$-tilted posterior:
$$q^*(\theta | x, \xi) \propto p(\theta) p(x | \theta, \xi)^\alpha$$
This effectively down-weights the likelihood, providing a natural mechanism for robustness against model misspecification (similar to power-likelihood methods).

D. Optimization via PAC-Bayes

Since $I^S_\alpha$ involves nested expectations that are intractable in closed form for general models, it must be estimated using Nested Monte Carlo (NMC) estimators. These estimators are inherently biased and noisy, making standard deterministic optimization unreliable.

To address this, the authors adopt a PAC-Bayes framework:

1. Stochastic Policies: Instead of optimizing a single deterministic design $\xi$, they optimize a distribution over designs $\pi$.
2. Lower Bound: They derive a high-probability lower bound on the true robust objective:
   $$\mathbb{E}_\pi [I^S_\alpha] \geq \mathbb{E}_\pi [\tilde{I}^S_\alpha] - \text{Penalty}(\pi, \pi_0) - \text{Error Terms}$$
   where $\tilde{I}^S_\alpha$ is the biased NMC estimator, and the penalty term involves the KL divergence between the learned policy $\pi$ and a prior policy $\pi_0$.
3. Optimization: The optimal policy is a Gibbs distribution: $\pi^*(\xi) \propto \pi_0(\xi) \exp(\lambda \tilde{I}^S_\alpha(\xi))$, which can be optimized using stochastic policy search methods (e.g., mirror descent).

3. Key Contributions

1. Unified Robust Framework: The paper unifies robust inference and robust design selection into a single max-min game, showing that they naturally lead to the same mathematical structure (Rényi divergence and $\alpha$-tilted posteriors).
2. Theoretical Characterization: It rigorously derives that Sibson's $\alpha$-mutual information is the correct objective for robust BED under KL-constrained misspecification, replacing the standard Shannon mutual information.
3. Bias-Aware Optimization: It introduces a PAC-Bayes approach specifically tailored for robust experimental design. This allows for the optimization of designs even when the objective function is estimated via biased, nested Monte Carlo estimators, providing rigorous finite-sample guarantees.
4. Interpretation of $\alpha$: It provides a decision-theoretic interpretation of the Rényi order $\alpha$ as a direct consequence of the adversarial budget $\rho$, rather than a heuristic hyperparameter.

4. Results

The paper validates the framework through numerical experiments on Linear Regression and A/B Testing (both continuous and discrete settings).

• Robustness vs. Nominal Performance:
  • Nominal designs (optimizing Shannon MI) appear to yield higher information gains but are overconfident. In coverage analysis, nominal posteriors frequently fail to contain the true parameter (under-coverage).
  • Robust designs (optimizing $I^S_\alpha$) produce conservative posteriors. While the nominal gain might be lower, the actual coverage of the credible sets is significantly better, ensuring the true parameter is captured more reliably.
• Impact of $\alpha$: As $\alpha$ decreases (increasing robustness), the optimal designs shift to be less sensitive to model errors. The information gain decreases, reflecting the "cost" of robustness, but the reliability of the inference increases.
• PAC-Bayes vs. Naive Optimization:
  • When optimizing using the noisy NMC estimator, naive deterministic optimizers (e.g., gradient descent on the noisy estimate) suffer from high variance and suboptimal designs due to estimator bias.
  • The PAC-Bayes stochastic policy consistently achieves lower regret and designs closer to the theoretical optimum, demonstrating the necessity of the PAC-Bayes framework for handling estimation uncertainty.

5. Significance

This work provides a principled solution to the brittleness of Bayesian experimental design. By framing the problem as a game against nature, it moves beyond ad-hoc fixes to a mathematically grounded approach where robustness is explicitly traded off against information gain via the parameter $\alpha$.

The introduction of PAC-Bayes stochastic policies is particularly significant for the broader field of machine learning and statistics, as it offers a rigorous way to optimize objectives that are only available through biased, nested estimators—a common scenario in complex scientific simulations where analytical solutions are impossible. The framework ensures that experimental designs remain effective even when the underlying models are imperfect, a critical requirement for real-world applications in epidemiology, robotics, and physics.