Confidence intervals for functionals in constrained inverse problems via data-adaptive sampling-based calibration

Technical Summary: Confidence Intervals for Functionals in Constrained Inverse Problems via Data-Adaptive Sampling-Based Calibration

Problem Statement
The paper addresses the challenge of frequentist uncertainty quantification (UQ) for ill-posed inverse problems where the forward model is rank-deficient, the parameter dimension exceeds the observation dimension, and the unknown parameters are subject to physical constraints (e.g., non-negativity). The specific goal is to construct a confidence interval $C_\alpha(y)$ for a one-dimensional quantity of interest (QoI), $\phi(x^*)$, that satisfies a uniform finite-sample coverage guarantee:
$$P_{x^*}(\phi(x^*) \in C_\alpha(y)) \geq 1 - \alpha$$
for all feasible $x^* \in X$.

Existing methods, particularly "One-at-a-Time Strict Bounds" (OSB), rely on optimization-based endpoint computations. While OSB is computationally feasible, its coverage guarantees are theoretically unproven and often fail in practice due to the difficulty of calibrating the test statistic threshold ($\psi^2_\alpha$) in the presence of constraints. The standard approach of using a simultaneous confidence set for the entire parameter space is overly conservative, leading to excessively wide intervals. Conversely, tailoring the interval to a specific QoI (the OSB approach) introduces a "Burrus Conjecture" problem where the coverage guarantee is generally invalid because the null hypothesis space is composite and potentially unbounded.

Methodology
The authors propose a framework that combines test inversion, data-adaptive constraint shrinking, and sampling-based calibration. The core methodology proceeds in three stages:

1. Data-Adaptive Constraint Shrinking (Berger–Boos Set):
   To handle the unboundedness and complexity of the composite null hypothesis, the authors introduce a data-adaptive subset of the constraint set, $B_\eta(y)$, inspired by the Berger–Boos construction for nuisance parameters.

   • They define a confidence set for the observations, $\Gamma_\eta(y) = \{y' : \|y - y'\|^2 \leq \chi^2_{n,\eta}\}$.
   • The Berger–Boos set is the pre-image of this region intersected with the constraints: $B_\eta(y) = \{x \in X : \|y - f(x)\|^2 \leq \chi^2_{n,\eta}\}$.
   • This set contains the true parameter $x^*$ with probability at least $1-\eta$. The remaining miscoverage budget ($\alpha - \eta$) is allocated to calibrate the likelihood ratio test (LLR) at a level $\gamma \leq \alpha - \eta$.

2. Quantile Estimation via Sampling:
   Instead of attempting to compute the maximum quantile of the LLR statistic over the entire (potentially unbounded) space, the method estimates the quantile function over the bounded set $B_\eta(y)$.

   • Sampling: The authors employ specific samplers to draw design points uniformly from $B_\eta(y)$. For low-dimensional full-rank cases, they use the Voelker–Gosmann–Stewart (VGS) sampler. For high-dimensional or rank-deficient cases, they utilize a custom Polytope sampler based on the Vaidya walk.
   • Calibration: Two algorithms are proposed to estimate the quantile surface:
     • Algorithm 3.1 (Direct Estimation): Samples the LLR statistic $N$ times at each design point to estimate the empirical quantile.
     • Algorithm 3.2 (Quantile Regression): Samples the LLR statistic once per design point and uses quantile regression (specifically Gradient Boosting with pinball loss) to learn the conditional quantile surface $q_\gamma(x)$. This approach trades repeated optimization for a smoothness assumption.

3. Interval Construction:
   Using the estimated quantiles, the authors define four distinct interval constructions based on two taxonomies:

   • Global vs. Sliced: Global methods maximize the quantile over the entire $B_\eta(y)$, while Sliced methods maximize it over level sets of the QoI ($\Phi_\mu \cap B_\eta$).
   • Inverted vs. Optimized: Inverted methods accept functional values $\mu$ where the observed LLR is below the estimated quantile. Optimized methods define endpoints by optimizing $\phi(x)$ over a region defined by the estimated quantile threshold.

Key Contributions

• Theoretical Framework: The paper extends the test-inversion framework of [2] to constrained, ill-posed problems by integrating the Berger–Boos set. This allows for the control of Type I error over a bounded, data-adaptive region rather than the entire unbounded parameter space.
• Novel Integration: While Berger–Boos sets and quantile regression exist in literature, this work is the first to integrate them specifically for producing constraint-aware confidence intervals for functionals in high-dimensional, potentially unbounded inverse problems.
• Algorithmic Implementation: The authors provide practical algorithms for sampling from the complex geometry of the Berger–Boos set (which is an intersection of a polytope and an ellipsoid) and for estimating the necessary quantile surfaces using machine learning techniques.
• Theoretical Justification: The paper proves that the proposed interval constructions converge in probability to the true confidence sets as the number of samples increases, provided the quantile function is continuous and the sampling covers the set effectively.

Results
Numerical experiments were conducted on synthetic Gaussian models and a realistic high-energy physics unfolding problem (80-dimensional parameter space, rank-deficient forward model).

• Coverage: In scenarios where the state-of-the-art OSB intervals fail to achieve nominal coverage (specifically in 3D constrained Gaussian and adversarial high-dimensional settings), the proposed methods consistently achieve the target coverage.
• Interval Length:
  • In cases where OSB achieves nominal coverage, the proposed "Sliced" intervals are competitive, often providing shorter expected lengths (e.g., ~11–18% reduction in the smooth high-dimensional deconvolution experiment).
  • In cases where OSB fails, the proposed methods achieve coverage while maintaining shorter expected lengths than OSB.
  • The "Global" constructions tend to be more conservative (wider) than the "Sliced" ones, reflecting the trade-off between robustness and tightness.
• Sensitivity: Experiments varying the parameter $\eta$ (the size of the Berger–Boos set) suggest that a small positive $\eta$ (e.g., 0.01) offers a beneficial trade-off, shrinking the optimization domain enough to improve length without excessively inflating the quantile threshold.

Significance and Claims
The paper claims that its approach offers a flexible, theoretically grounded framework for UQ in constrained inverse problems that avoids the pitfalls of regularization bias inherent in Bayesian or deterministic regularization methods. By shifting the complexity from proving coverage for unbounded sets to sampling within a data-adaptive bounded set, the method produces intervals that are both computationally feasible and less conservative than simultaneous bounds.

The authors modestly note that the choice of the parameter $\eta$ is currently heuristic (based on small pre-specified values) and that developing an automated, optimal selection rule for $\eta$ that preserves finite-sample guarantees is a direction for future work. Similarly, while the method handles high-dimensional problems effectively in the experiments, extending it to even higher dimensions requires further development of sampling techniques to address the curse of dimensionality. The work does not claim to solve all inverse problems but provides a specific, rigorous pathway for functional uncertainty quantification where constraints and rank-deficiency are present.