Adaptive anisotropic composite quadratures for residual minimisation in neural PDE approximations

1. Problem Statement

The paper addresses a critical limitation in Physics-Informed Neural Networks (PINNs) and other residual-minimisation methods for solving Partial Differential Equations (PDEs). In these methods, a neural network (NN) is trained to minimize a loss function defined by the integral of the PDE residual over the domain.

• The Core Issue: The quality of the numerical quadrature (the method used to approximate these integrals) is often treated as a secondary concern. Standard approaches use fixed uniform grids, Monte Carlo (MC), or Quasi-Monte Carlo (QMC) sampling.
• The Consequence: If the quadrature is too coarse or poorly distributed, the optimizer may minimize the discrete loss (based on the quadrature points) while the continuous residual remains large. This leads to overfitting to the specific quadrature points, causing a mismatch between training performance and the true solution accuracy.
• The Challenge: Neural networks are global, highly nonlinear approximators. Constructing a fixed quadrature rule accurate enough for all possible NN realizations during training is computationally infeasible. Therefore, the quadrature must adapt dynamically to the evolving solution.

2. Methodology

The authors propose a three-pronged methodology combining theoretical error analysis, an adaptive quadrature algorithm, and a training strategy.

A. Theoretical Framework: Nonlinear Strang-Type Estimate

The authors first establish an abstract error framework separating three error sources:

1. Approximation Error: The inherent error of the NN architecture.
2. Optimization Error: The gap between the global minimum and the optimizer's output.
3. Quadrature Error: The error introduced by approximating integrals.

They derive a nonlinear Strang-type estimate showing that the relative quadrature error ($\varepsilon_{rel}$) acts as a multiplicative factor on the best-approximation error. Crucially, if $\varepsilon_{rel}$ is not controlled, it can distort the loss landscape, amplifying approximation errors and leading to instability. This justifies the need to control quadrature error relative to the loss magnitude, rather than just minimizing absolute error.

B. Adaptive Anisotropic Composite Quadrature

To control the quadrature error, the authors propose an anisotropic $h$-adaptive composite quadrature strategy:

• Primal vs. Reference: For every cell in the integration domain, they use a "primal" quadrature rule (for training) and a richer "reference" rule (for error estimation).
• Error Estimation: The local error is estimated by the difference between the primal and reference integrals ($\delta_K = |Q'_K - Q_K|$).
• Anisotropic Refinement: Instead of isotropic splitting (dividing a cell into equal sub-cells), the algorithm uses bisection. It calculates "fourth differences" (symmetric finite differences) along coordinate axes to determine the direction of highest irregularity in the integrand. The cell is bisected only along the axis that promises the greatest error reduction. This mitigates the curse of dimensionality.
• Positive Weights: The method explicitly uses quadrature rules with positive weights (e.g., Gauss-Legendre or Witherden-Vincent). This is vital because negative weights can cause the loss to become unbounded from below, leading to numerical instabilities in NN training.

C. Refresh-Based Training Strategy

To balance computational cost and accuracy, the authors introduce a refresh-based training loop:

1. Monitoring: During training, an online error indicator ($\eta$) compares the primal and reference losses.
2. Trigger: When $\eta$ exceeds a user-defined threshold ($\tau$), the current quadrature is deemed unreliable for the current NN state.
3. Rebuild: The adaptive quadrature algorithm (Algorithm 1) is re-run from a base mesh to generate a new, refined partition tailored to the current NN realization.
4. Resume: Training continues with the new quadrature. This ensures the quadrature evolves with the solution, preventing the "stagnation" of error control.

3. Key Contributions

1. Theoretical Justification: Derivation of a nonlinear Strang-type estimate proving that controlling relative quadrature perturbations is essential for the stability and accuracy of residual minimisation in NNs.
2. Anisotropic Adaptive Algorithm: Development of a bisection-based, anisotropic $h$-adaptive quadrature that efficiently targets high-gradient regions without the computational explosion of isotropic refinement.
3. Training-Aware Adaptivity: A novel "refresh" methodology that dynamically rebuilds the quadrature only when necessary, linking the quadrature control directly to the optimization process.
4. Positive Weight Enforcement: A practical constraint ensuring quadrature weights remain positive to maintain loss stability, avoiding the pitfalls of nested rules (like Gauss-Kronrod) that often introduce negative weights.

4. Numerical Results

The authors tested their approach on a wide range of benchmark problems (1D to 3D, linear to nonlinear, steady to time-dependent) and compared it against Uniform, Monte Carlo (MC), Latin Hypercube (LHC), and Halton sampling.

• Function Approximation (2D Arc-Tan): The adaptive strategy (AQ) achieved superior $L^2$ and $H^1$ errors compared to all non-adaptive methods. Crucially, only AQ maintained a monotonic decrease in both training and reference losses, indicating no overfitting. Uniform grids suffered from Runge phenomena.
• 1D Advection-Diffusion (Boundary Layers): For high Péclet numbers (sharp boundary layers), AQ successfully concentrated points at the boundary layer with as few as 20 partitions. Uniform and MC methods failed to resolve the layer or overfitted.
• Viscous Burgers & KdV Equations (Shock/Solitons): AQ captured shock waves and soliton interactions with high fidelity. It achieved an order-of-magnitude lower error than QMC (Halton) and uniform grids while using significantly fewer quadrature points (e.g., ~166 partitions vs. 10,000 MC points in Burgers).
• Cahn-Hilliard & Navier-Stokes (Complex Physics):
  • Cahn-Hilliard: AQ resolved phase separation with high accuracy, outperforming methods that stagnated or overfitted.
  • Navier-Stokes (Re=3200): AQ solved the lid-driven cavity problem with a relative $L^2$ error of $3.71 \times 10^{-2}$, improving upon state-of-the-art results ($4.21 \times 10^{-2}$) using a smaller network and 75% fewer training iterations.
  • Stokes (Moffatt Vortices): AQ captured the complex hierarchy of vortices near corners, a task where non-adaptive methods failed to resolve fine scales.
• Parametric & 3D Problems: In parametric and 3D convection-diffusion problems, non-adaptive methods showed early stagnation and growing gaps between training and reference losses. AQ maintained robust generalization across the parameter space and 3D domain.

5. Significance

This work fundamentally shifts the paradigm of training Neural PDE solvers by treating numerical quadrature as a dynamic, adaptive component rather than a static preprocessing step.

• Efficiency: It demonstrates that high accuracy can be achieved with significantly fewer data points (quadrature nodes) by focusing computational effort where the solution is complex (anisotropic refinement).
• Reliability: By explicitly controlling the relative error between training and reference losses, the method provides a robust indicator of solution quality, preventing the common issue of "deceptive convergence" where the training loss decreases but the true error does not.
• Scalability: The anisotropic approach makes high-dimensional problems (3D and parametric) more tractable by avoiding the exponential growth of points associated with isotropic refinement.
• Generalizability: The strategy is applicable to a broad class of PDEs (elliptic, hyperbolic, parabolic, nonlinear) and domain geometries, offering a unified framework for improving the accuracy and stability of scientific machine learning.