Spotlight, priorsketching and Bayesian approximation error paradigms

1. Problem Statement

The paper addresses a fundamental challenge in large-scale computational inverse problems: the trade-off between model accuracy and computational cost.

• The Dilemma: High-fidelity forward models (often based on PDEs) require dense discretization and precise knowledge of nuisance parameters (e.g., domain geometry, material properties). Solving inverse problems with these models is computationally prohibitive due to the "curse of dimensionality."
• The Consequence of Approximation: Replacing detailed models with computationally cheaper surrogate models (e.g., coarse discretization or fixed nuisance parameters) introduces model discrepancy (approximation error).
• The Failure Mode: Standard inverse solvers often treat this discrepancy as observation noise. However, approximation errors typically have non-zero means and strong correlations. Ignoring them leads to severe artifacts, blurring, and "clutter" in the reconstructed solutions, rendering them unreliable.

The authors aim to bridge two distinct methodologies for handling these errors: the Bayesian Approximation Error (BAE) framework and Spotlight Inversion (linear algebraic clutter removal).

2. Methodology

The paper develops a theoretical and practical connection between two approaches to model reduction:

A. Bayesian Approximation Error (BAE) Paradigm

• Core Concept: Instead of treating the error between an accurate model $f^*$ and a surrogate model $f$ as simple noise, BAE models the error $M = f^*(X) - f(Z)$ as a random variable with a specific statistical distribution.
• Implementation:
  1. Generate samples of the approximation error by drawing from the prior distributions of the unknowns and nuisance parameters, then computing the difference between accurate and surrogate model outputs.
  2. Estimate the mean ($\mu$) and covariance ($C_M$) of these errors.
  3. Incorporate these statistics into the likelihood function. The inverse problem is solved by minimizing a cost function that accounts for the error covariance, effectively "whitening" the data mismatch.
• Limitation: Requires the computation and inversion of large covariance matrices, which can be expensive.

B. Spotlight Inversion

• Core Concept: A linear algebraic approach where the "nuisance" part of the data (clutter) is eliminated via orthogonal projection.
• Implementation:
  1. Decompose the observation model into a term of interest and a clutter term (nuisance parameters).
  2. Construct an orthogonal projector $P^\perp$ onto the subspace orthogonal to the range of the clutter matrix.
  3. Apply this projector to the data and the forward operator, effectively removing the nuisance contribution before solving the inverse problem.
• Limitation: In underdetermined problems or when the signal and clutter subspaces are nearly aligned, simple projection can destroy signal information or fail to suppress clutter completely.

C. The Synthesis: Priorsketching and Theoretical Connection

The authors demonstrate that these two methods are closely related but not equivalent:

• Clutter as Approximation Error: The authors show that the "clutter" in spotlight inversion can be interpreted as the approximation error in the BAE framework.
• Priorsketching: A novel concept introduced in the paper. Instead of using standard random matrices for sketching (dimensionality reduction), the authors propose drawing the sketching matrix from the prior distribution of the nuisance parameters.
  • This allows the construction of a low-rank approximation of the error covariance matrix ($C_M \approx A_2 \Omega \Omega^T A_2^T$) without explicitly forming the full covariance.
  • The singular vectors of this "sketched" matrix define the projection subspace for the spotlight inversion.
• Theoretical Limit: The paper proves that Spotlight Inversion is a limit case of the BAE approach.
  • In BAE, data mismatch in the error subspace is weighted by the inverse of the eigenvalues of the error covariance.
  • In Spotlight Inversion, the weights for the dominant error modes are effectively set to zero (infinite penalty), completely discarding data fidelity in those directions.

3. Key Contributions

1. Unification of Paradigms: The paper establishes a rigorous mathematical link between the probabilistic BAE approach and the deterministic Spotlight Inversion, showing they are two sides of the same coin regarding model reduction.
2. Introduction of Priorsketching: The authors introduce "priorsketching," a method to generate sketching matrices based on prior beliefs about nuisance parameters. This ensures that the dimensionality reduction focuses on the subspace where the error actually lives, rather than a generic random subspace.
3. Computational Efficiency: By utilizing priorsketching, the authors demonstrate that one can achieve the benefits of BAE (error compensation) using the computational simplicity of Spotlight Inversion (orthogonal projection), avoiding the need to invert large, dense covariance matrices.
4. Handling Non-Gaussian Priors: The methodology is extended to handle non-Gaussian priors (e.g., positivity constraints) via logit transformations and sigmoid functions, which are common in physical imaging problems.

4. Results (Computed Examples)

The authors validate their theory using two distinct imaging problems:

Example 1: X-ray Tomography (Linear)

• Setup: A 2D fanbeam tomography problem where the region of interest (ROI) is kept at high resolution, while the rest of the domain is coarsened to reduce dimensionality.
• Findings:
  • Naïve Approach: Solving the coarse model without error compensation resulted in severe blurring and boundary halos in the ROI.
  • BAE Approach: Using the full BAE covariance correction significantly reduced artifacts and recovered the ROI details accurately.
  • Spotlight/Priorsketching Approach: Using the projection method based on priorsketching (using only 250 error samples) produced results nearly identical to the full BAE solution but with significantly lower computational overhead (no covariance inversion required).

Example 2: Electrical Impedance Tomography (Non-Linear)

• Setup: An EIT problem with unknown domain geometry (boundary shape uncertainty). The goal is to reconstruct conductivity despite the mismatch between the true domain shape and the reference mesh used for computation.
• Findings:
  • Naïve Approach: Ignoring geometry uncertainty led to strong artifacts mimicking the shape of the boundary error.
  • Spotlight Approach: Using only five random realizations of the geometry to estimate the error subspace and applying the projection technique successfully removed the geometric artifacts.
  • Significance: This demonstrated that the method is robust even with very few samples of the approximation error, making it highly efficient for complex, non-linear forward models where generating error samples is expensive.

5. Significance and Conclusion

The paper makes a significant contribution to the field of computational inverse problems by:

• Bridging Theory and Practice: It provides a theoretical justification for using simple linear algebraic projections (Spotlight) to solve problems that are theoretically framed in complex Bayesian terms (BAE).
• Scalability: The "priorsketching" approach offers a scalable solution for large-scale problems where computing and storing full error covariance matrices is impossible.
• Robustness: The methods are shown to be effective even with limited samples of the approximation error, which is crucial for problems involving expensive PDE solvers.
• Future Direction: The authors suggest that this framework opens the door to further connections with randomized linear algebra (e.g., Hessian sketching) and more sophisticated BAE models that account for the interdependency between the approximation error and the quantity of interest.

In summary, the paper argues that by viewing model reduction errors through the lens of "clutter" and utilizing priorsketching, one can achieve the accuracy of Bayesian error compensation with the computational efficiency of linear projection methods.