Sparse-grids-like surrogate models enhanced with gradient information

Technical Summary: Sparse-grids-like surrogate models enhanced with gradient information

Problem Statement
The paper addresses the challenge of constructing efficient surrogate models for Quantities of Interest (QoI) derived from parametric non-linear partial differential equations (PDEs). While sparse-grid interpolation is a well-established technique for approximating functions over high-dimensional parameter domains, it traditionally relies solely on function evaluations. The authors aim to extend this methodology to incorporate gradient information (derivatives of the QoI with respect to PDE parameters).

A central motivation is that, for many PDE-based problems, computing the gradient $\nabla_\Theta Q$ at a specific parameter point $\theta$ is computationally comparable to computing the QoI $Q(\theta)$ itself. Specifically, once the solution $u_\theta$ is found (e.g., via Newton's method), the gradient can be obtained by solving a linear system with the same operator but different right-hand sides (or via adjoint methods), often at a cost similar to a single additional function evaluation. The goal is to leverage this "cheap" gradient information to improve surrogate accuracy.

However, the paper identifies a significant theoretical hurdle: incorporating gradient information into sparse-grid constructions without careful consideration leads to a "derivative-induced curse of dimensionality." Standard tensor-product osculatory interpolation (matching function values and first derivatives) requires evaluating all $2^N$ mixed partial derivatives at each grid point, which is infeasible for PDE applications where only unidirectional derivatives are available. Furthermore, a previous attempt to solve this by nesting Smolyak algorithms (proposed in [11]) is shown to be theoretically flawed, failing to provide interpolatory properties or consistent convergence.

Methodology
The authors propose a hybrid approach termed Gradient-Enhanced Least-squares on Sparse grids (GELS). This method decouples the sampling strategy from the approximation strategy:

1. Sparse-Grid Sampling: The Smolyak algorithm is used to define the set of collocation points (the sparse grid) in the parameter domain and to identify a suitable polynomial space. This addresses the "sampling curse of dimensionality" by avoiding full Cartesian grids.
2. Gradient-Informed Least Squares: Instead of using the sparse-grid points for direct interpolation (which fails when mixing function and gradient data in the standard combination technique), the method formulates a weighted least-squares problem. The surrogate model is constructed as the minimizer of the sum of squared errors for both function values and gradient components over the sparse grid points:
   $$G_w Q := \arg\min_{v \in P_{F_w}} \sum_{\tau_j \in T_{I_{sum}(w)}} \left[ \|Q(\tau_j) - v(\tau_j)\|^2 + \sum_{n=1}^N \omega_n \|\partial_{\theta_n}Q(\tau_j) - \partial_{\theta_n}v(\tau_j)\|^2 \right]$$
   where $P_{F_w}$ is the polynomial space derived from the sparse-grid combination of Lagrange and osculatory operators, and $\omega_n$ are weights balancing the magnitude of function and gradient errors.

Key Contributions

1. Critique of Existing Approaches: The paper provides a rigorous analysis demonstrating why the nested Smolyak approach proposed in [11] is faulty. It shows that the resulting operator does not possess the necessary telescopic structure to guarantee convergence and, in numerical tests, can yield approximations with higher error than their individual component operators.
2. The GELS Framework: The authors introduce the GELS method, which successfully combines the efficiency of sparse-grid sampling with the accuracy benefits of gradient information without requiring mixed derivatives.
3. Analysis of Cost and Accuracy Trade-offs: The work establishes that the efficacy of GELS is not universal but depends critically on two factors:
   • Computational Cost: The relative cost ($\lambda$) of computing a partial derivative compared to the QoI.
   • Data Accuracy: The relative precision of the gradient evaluations compared to the function evaluations. The paper demonstrates that if gradient data is significantly noisier or less accurate than function data (a common scenario in PDE discretizations), GELS may stagnate or perform worse than standard sparse grids.

Numerical Results
The authors validate GELS through three sets of experiments:

• Benchmark Functions: Tests on closed-form functions with varying regularity and dimensions ($N=2$ to $11$). Results show that GELS outperforms standard sparse grids (SG) for smooth functions when gradient costs are low, but the advantage diminishes for less regular functions or higher dimensions.
• Noise Robustness: Experiments with artificially noisy data reveal that GELS is sensitive to the relative accuracy of gradients. If gradient noise exceeds function noise, the convergence of GELS stagnates at the noise level, potentially underperforming standard SG.
• Parametric PDEs: Two non-linear Darcy flow problems (p-Laplace equation) are tested:
  • Test 1 (Well-behaved): A problem with a piecewise-constant permeability where the solution remains regular. Here, numerical gradients are as accurate as the QoI, and GELS converges faster than SG.
  • Test 2 (Degenerate): A problem with moving interfaces and degenerate diffusion. Here, the numerical gradients suffer from lower regularity and higher error than the QoI. Consequently, GELS stagnates, while SG continues to converge smoothly.

Significance and Claims
The paper modestly claims to provide a viable mechanism for incorporating gradient information into sparse-grid surrogate modeling, correcting a flawed theoretical foundation in the literature. It emphasizes that the "convenience" of this technique is conditional. The authors conclude that GELS is advantageous only when:

1. The cost of computing gradients is sufficiently low relative to function evaluations.
2. The numerical accuracy of the computed gradients is comparable to that of the QoI itself.

The work highlights that simply adding gradient data is insufficient; the quality of the derivative information relative to the function value is a decisive factor in the success of the surrogate model. The authors suggest that further theoretical analysis is needed to fully understand the approximation power of GELS under varying regularity and noise conditions.