Neural network approximation in discrete dual norms with adaptive test spaces

Technical Summary: Neural Network Approximation in Discrete Dual Norms with Adaptive Test Spaces

Problem Statement
The paper addresses the challenge of training Robust Variational Physics-Informed Neural Networks (RVPINNs) for partial differential equations (PDEs) exhibiting localized singularities, steep gradients, or interface layers. In RVPINNs, the loss function is defined via the discrete Riesz representative of the variational residual within a finite-dimensional test space. While this formulation guarantees that the loss is a robust estimator of the true error (up to a remainder term), a fixed, coarse test space often fails to resolve the continuous Riesz representative during training. Consequently, the neural network may converge to a solution determined by the discrete test space rather than the true continuous solution. Conversely, using a sufficiently fine test space from the outset is computationally inefficient, particularly in complex scenarios. The core problem is to develop an adaptive strategy that enriches the test space only where necessary to control the discrepancy between discrete and continuous Riesz representatives without incurring prohibitive computational costs.

Methodology
The authors propose a framework that integrates adaptive test-space refinement into the RVPINN training loop. The methodology relies on the following components:

1. Theoretical Framework:

   • The study establishes two types of error upper bounds for the true error $\|u - u_\theta\|_U$: one based on a residual remainder term and another based on the error between the discrete and continuous Riesz representatives ($\|\phi(u_\theta) - \phi_k(u_\theta)\|_V$).
   • The authors prove the equivalence of these two bounds, demonstrating that controlling the discrepancy between the discrete and continuous Riesz representatives is sufficient to ensure robustness.
   • Two adaptive strategies are analyzed: an idealized strategy (assuming infinite training iterations per mesh) and a sequential strategy (assuming the Riesz discrepancy does not increase after training). Both strategies provide theoretical convergence guarantees under specific conditions.

2. Computable Refinement Indicator:

   • To avoid the non-computable nature of the exact Riesz discrepancy, the authors introduce a practical refinement indicator based on a two-level approach.
   • This indicator compares the discrete Riesz representative computed in the current test space $V_k$ against one computed in an enriched test space $\hat{V}_k$ (e.g., using higher-order polynomials or a finer subgrid).
   • Under a standard saturation assumption, the authors prove that this indicator is a reliable and efficient estimator for the true discrepancy.

3. Adaptive Algorithm:

   • A practical algorithm (Algorithm 6.1) is developed following the SOLVE–ESTIMATE–MARK–REFINE paradigm.
   • ESTIMATE: Compute the local discrepancy indicator between the coarse and enriched test spaces.
   • MARK: Use the Dörfler marking criterion to select elements contributing most to the error.
   • REFINE: Refine the marked elements to create a new mesh.
   • SOLVE: Train the neural network on the refined mesh until the loss (square root of the discrete dual norm) meets a prescribed tolerance.
   • This cycle repeats, reducing the tolerance threshold geometrically ($\epsilon_k = \delta \epsilon_{k-1}$) at each outer iteration.

Key Contributions

• Error Analysis: The paper establishes upper bounds for the RVPINN approximation error and proves their equivalence, clarifying the role of the Riesz discrepancy in training robustness.
• Adaptive Strategies: The authors derive theoretical convergence results for both idealized and sequential adaptive strategies within the RVPINN framework.
• Refinement Indicator: A computable refinement indicator is proposed and proven to be a reliable and efficient error estimator for the discrepancy between discrete and continuous Riesz representatives under the saturation assumption.
• Practical Implementation: A complete adaptive algorithm is presented and validated numerically.

Numerical Results
The proposed method is tested on three benchmark elliptic Dirichlet problems:

1. Smooth Solution: A Poisson problem on a square domain. The adaptive RVPINN achieved an empirical convergence rate of approximately 3 against the test-space dimension, significantly outperforming the standard piecewise-linear FEM rate of 0.5.
2. Kink Solution: An elliptic interface problem with a discontinuous coefficient. The adaptive refinement successfully concentrated mesh elements along the circular interface where the solution lacked smoothness, achieving a convergence rate of approximately 0.5, competitive with adaptive FEM.
3. Singular Solution: A Poisson problem on an L-shaped domain with a corner singularity. The adaptive RVPINN achieved a convergence rate of approximately 1.5, outperforming the adaptive FEM rate of 0.5. The method maintained a strong correlation between the loss function and the true error throughout training, even in the presence of singularities.

In all cases, the adaptive RVPINN approach demonstrated smaller errors than reference adaptive FEM when compared against the dimension of the discrete test space.

Significance and Claims
The paper claims that the proposed adaptive test-space strategy effectively accelerates convergence for RVPINNs in complex problems without requiring the inclusion of extra control terms in the loss function (which would increase training costs). By monitoring and minimizing the discrepancy between discrete and continuous Riesz representatives, the method ensures that the loss function remains a robust estimator of the true error. The authors assert that their approach leverages the high expressivity of neural network architectures to achieve convergence rates superior to standard adaptive FEM in terms of test-space dimension, particularly for problems with singularities. The work provides a theoretical foundation and a practical algorithm for making RVPINNs robust in scenarios where fixed test spaces fail.