Technical Summary: Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights

1. Problem Definition

This paper addresses a specific variant of the Active Learning (AL) problem where the goal is to learn an unknown function $f$ with low prediction error under a target distribution $P_f$ that is self-induced by the function itself. Unlike standard AL (which assumes a uniform or known target distribution) or Distributionally Robust AL (which assumes an ambiguity set of possible distributions), this setting posits that the target distribution follows a Boltzmann form:
$$P_f(x) = Z_f^{-1} \exp(\lambda f(x) + b(x))$$
where $\lambda \neq 0$ is a known parameter, $b(x)$ is a known bias function, and $Z_f$ is the partition function.

Key Challenges:

1. Circular Dependence: The target distribution $P_f$ depends on the unknown function $f$, creating a circular dependency where the sampling strategy depends on the very function it seeks to learn.
2. Intractable Partition Function: The normalization constant $Z_f = \int \exp(\lambda f(x) + b(x)) dx$ is generally intractable to compute, especially as $f$ is unknown and evolving.
3. Unknown Target: The distribution is not fully specified in advance; it emerges from the learning process.

This setting naturally arises in scientific domains such as Potential Energy Surface (PES) modeling in computational chemistry (where molecular dynamics samples from a Boltzmann distribution weighted by energy) and molecular drug discovery (where interest concentrates on high-scoring compounds).

2. Methodology

The authors propose SIDAL (Active Learning under Self-Induced Distribution) and introduce two Gaussian Process (GP)-based acquisition functions: AB-SID-iVAR and TS-SID-iVAR.

2.1. Theoretical Foundation

The approach relies on bounding the expected Mean Squared Error (MSE) under the unknown $P_f$ by relating it to a computable surrogate distribution $P_{\mu_t}$ (based on the GP posterior mean $\mu_t$).

• High-Probability Bound: By decomposing the log-density ratio between $P_f$ and $P_{\mu_t}$, the authors derive an upper bound involving the partition function ratio. This leads to a bound with an exponential prefactor dependent on the maximum prediction error.
• Average-Case Bound: By taking the expectation over the GP posterior $f|D_t$, the authors utilize Jensen's inequality and Moment Generating Function (MGF) identities for Gaussian variables. This allows the partition function ratio to be absorbed, yielding a tighter bound without the need to estimate $Z_f$ explicitly.
  $$\mathbb{E}_{f|D_t}[\mathbb{E}_{P_f}[(f - \mu_T)^2]] \leq C \cdot \mathbb{E}_{P_{\mu_t}}[\sigma_T^2(x)]$$
  where $\sigma_T^2(x)$ is the predictive variance.

2.2. Acquisition Functions

Both proposed methods aim to minimize the integrated posterior variance over a tractable surrogate of the self-induced distribution.

1. AB-SID-iVAR (Approximate Bayesian SID):

   • Approximates the intractable expectation $\mathbb{E}_{f|D_{t-1}}[P_f]$ in closed form using a zero-order Taylor expansion.
   • The surrogate unnormalized density is: $\tilde{p}^u_{t-1}(x) = \exp(\lambda \mu_{t-1}(x) + \frac{\lambda^2}{2}\sigma_{t-1}^2(x) + b(x))$.
   • This formulation naturally incorporates posterior uncertainty ($\sigma^2$) into the weighting, preventing premature commitment to a single mode.

2. TS-SID-iVAR (Thompson Sampling SID):

   • Approximates the expectation via a single Monte Carlo sample (Thompson Sampling): $\tilde{p}^u_{t-1}(x) = \exp(\lambda \tilde{f}_{t-1}(x) + b(x))$, where $\tilde{f}_{t-1} \sim f|D_{t-1}$.
   • This offers a bias-variance tradeoff compared to the closed-form approximation.

2.3. Algorithmic Implementation

The algorithms select the next query point $x_t$ by minimizing the expected posterior variance over a potential set $\bar{X}_t$:
$$x_t = \arg \min_{x \in \bar{X}_t} \int_{x^* \in \mathcal{X}} \tilde{p}^u_{t-1}(x^*) \sigma_t^2(x^* | x_t = x) dx^*$$

• Constraint Set $\bar{X}_t$: To ensure exploration, the search is restricted to regions where the current variance exceeds a threshold (estimated via the surrogate).
• Continuous Domains: For continuous inputs, the integral and the constraint threshold are approximated using Sequential Monte Carlo (SMC) sampling from the surrogate distribution.

3. Key Contributions

1. Problem Formalization: The paper formally defines the SIDAL problem, distinguishing it from standard AL and DRAL by the dependency of the target distribution on the unknown function.
2. Novel Acquisition Functions: Introduction of AB-SID-iVAR and TS-SID-iVAR, which approximate the Bayesian target distribution in closed form (or via single-sample MC) without requiring partition function estimation. These methods are applicable to both discrete and continuous domains.
3. Theoretical Guarantees:
   • Convergence: The authors establish that the terminal prediction error vanishes with high probability as the number of queries $T \to \infty$.
   • Rates: They provide sublinear convergence rates: $O(T^{-1/2+\epsilon})$ with constant Monte Carlo samples, and $O(T^{-1+\epsilon})$ when the number of samples scales linearly with $T$.
   • Average-Case Analysis: A tighter average-case bound is provided, showing improved logarithmic factors compared to the high-probability bound.
   • Heuristic Analysis: The paper provides the first convergence analysis for existing heuristic methods (Generic Heuristic Active Learning, GHAL) within this framework.
4. Empirical Validation: Demonstrated consistent improvements over baselines (Random Sampling, Uncertainty Sampling, IMSE, EPIG, and GHAL) on synthetic benchmarks and real-world tasks.

4. Experimental Results

The methods were evaluated on:

• Synthetic Benchmarks: Functions in dimensions $d=1$ to $d=6$ (e.g., Gramacy, Branin, Hartmann, Ishigami).
  • Results: AB-SID-iVAR consistently achieved lower weighted MSE than baselines. The performance gap widened in higher dimensions and for multimodal targets (e.g., Branin), where standard uncertainty sampling failed by concentrating on high-uncertainty but low-probability regions.
• Potential Energy Surface (PES) Modeling: Tasks involving H2 on Cu, H on Cu clusters, Si crystals, and H2O on Pt.
  • Results: AB-SID-iVAR achieved the lowest weighted MSE across all systems, demonstrating robustness in physical chemistry applications where accurate prediction is only required in specific, low-energy configurations.
• Molecular Drug Discovery: A discrete pool-based task using GuacaMol scoring functions.
  • Results: The method significantly outperformed baselines in predicting properties for high-scoring (high Boltzmann weight) molecules. While baselines degraded to negative $R^2$ on the top-k% of compounds, AB-SID-iVAR maintained accuracy near zero, indicating it successfully focused sampling on the relevant chemical space.

Ablation Studies:

• Replacing the Bayesian surrogate with a naive plug-in ($\exp(\lambda \mu_t)$) caused failure on multimodal targets, confirming the necessity of incorporating posterior variance ($\sigma^2$) into the surrogate.
• Removing the constraint set $\bar{X}_t$ led to higher variance and less robust convergence.

5. Significance and Claims

The paper claims to fill a gap in the active learning literature by addressing scenarios where the target distribution is self-induced and follows a Boltzmann form, a common but previously unstudied setting in fields like computational chemistry and drug discovery.

• Theoretical Significance: It provides the first rigorous convergence guarantees for learning under self-induced distributions, showing that terminal error vanishes despite the circular dependency and intractable partition function.
• Practical Significance: The proposed methods (specifically AB-SID-iVAR) offer a principled, parameter-free (regarding hyperparameter tuning of the acquisition function) alternative to existing heuristics that often require careful tuning. The methods successfully balance exploration and concentration, ensuring accurate predictions in high-probability regions of the self-induced distribution without needing to know the distribution's support a priori.
• Limitations: The authors note that their theoretical bounds exhibit an exponential dependence on $|\lambda|$ due to the density-ratio argument. Computationally, the reliance on MCMC sampling for continuous domains introduces overhead, though sensitivity analysis suggests this can be mitigated with fewer particles.

In summary, the paper presents a theoretically grounded and empirically effective framework for active learning when the "importance" of data points is determined by the unknown function being learned, specifically in the context of Boltzmann-weighted distributions.