Enriching continuous Lagrange finite element approximation spaces using neural networks

1. Problem Statement

The paper addresses the challenge of solving parametric partial differential equations (PDEs), specifically linear elliptic problems, with high accuracy and reduced computational cost.

• Limitations of Standard FEM: While the Finite Element Method (FEM) provides rigorous error estimates and convergence guarantees, it becomes computationally expensive for problems with strong local variations (e.g., sharp gradients, boundary layers) or high-dimensional parametric spaces, often requiring extremely fine meshes.
• Limitations of PINNs: Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative that is dimension-insensitive and fast for parametric problems. However, they often lack the precision and convergence guarantees of classical numerical methods, particularly regarding the accuracy of high-order derivatives.
• Goal: The authors propose a hybrid framework that combines the speed and flexibility of PINNs with the accuracy and theoretical guarantees of FEM. The objective is to use a neural network prediction as a "prior" to enrich the FEM approximation space, allowing for accurate solutions on coarse meshes.

2. Methodology

The core contribution is the modification of the classical continuous Lagrange finite element space by incorporating a prior solution $u_\theta$ (predicted by a PINN). The authors propose two distinct enrichment strategies:

A. Additive Enrichment

The trial space is modified by adding the prior to the standard finite element space:
$$V_h^+ = \{ u_h^+ = u_\theta + p_h^+ \mid p_h^+ \in V_h^0 \}$$
where $V_h^0$ is the standard space of continuous Lagrange polynomials vanishing on the boundary.

• Formulation: The problem is solved for the correction term $p_h^+$ using a Petrov-Galerkin approach. The source term is modified to account for the residual of the prior:
  $$a(p_h^+, v_h) = l(v_h) - a(u_\theta, v_h)$$
• Boundary Conditions: If the PINN satisfies boundary conditions exactly (e.g., via a level-set function), the correction term satisfies homogeneous boundary conditions.

B. Multiplicative Enrichment

The trial space is modified by multiplying the prior with a correction factor. To ensure the prior is non-zero, the problem is "lifted" by a constant $M$ such that $u_M = u + M > 0$.
$$V_h^\times = \{ u_{h,M}^\times = u_{\theta,M} \cdot p_h^\times \mid p_h^\times \in 1 + V_h^0 \}$$
where $u_{\theta,M} = u_\theta + M$.

• Formulation: The correction $p_h^\times$ is solved in a modified variational form where the test functions are weighted by the prior.
• Convergence: Theoretical analysis shows that as $M \to \infty$, the multiplicative approach converges to the additive approach.

C. Prior Construction (PINN Training)

To ensure the enriched FEM works effectively, the prior must accurately approximate not just the solution but also its derivatives (up to order $k+1$, where $k$ is the polynomial degree of the FEM). The paper employs several advanced PINN training techniques:

• Exact Boundary Conditions: Using level-set functions to enforce Dirichlet conditions exactly, removing the need for boundary loss terms.
• Sobolev Training: Adding a loss term based on the derivatives of the PDE residual to force the network to learn higher-order derivatives accurately.
• Fourier Features: Using sinusoidal embeddings to overcome spectral bias and capture high-frequency solutions.
• Architecture Adaptation: For non-smooth problems (e.g., transmission problems), using specialized architectures (e.g., ReCoNNs) that incorporate known singularities.

3. Key Contributions

1. Theoretical Error Estimates:

   • The authors prove a priori error estimates for both additive and multiplicative enrichments.
   • Key Result: The error of the enriched method is bounded by the error of the prior. Specifically, the error constant depends on the ratio of the $(k+1)$-th derivative of the error between the prior and the true solution.
   • Implication: If the prior is a good approximation (especially of derivatives), the enriched FEM on a coarse mesh achieves the same accuracy as a standard FEM on a much finer mesh, without degrading the convergence order.

2. Two Enrichment Strategies:

   • The paper rigorously compares additive and multiplicative approaches. The additive approach is shown to be more robust and easier to implement, while the multiplicative approach is effective in specific cases (e.g., convection-dominated flows with sharp gradients) but sensitive to the choice of the lifting constant $M$.

3. Parametric Efficiency:

   • The framework is designed for parametric problems. The PINN is trained offline on a set of parameters. The online phase involves solving a single coarse FEM problem for a specific parameter, using the pre-trained PINN as a prior. This drastically reduces the cost for solving multiple parameter instances.

4. Numerical Validation:

   • Extensive experiments in 1D, 2D, and 3D (Poisson, elliptic systems, transmission problems, anisotropic diffusion) validate the theory.
   • The method demonstrates significant gains in accuracy for a given mesh size, or conversely, allows for much coarser meshes to achieve a target accuracy.

4. Results

• Accuracy Gains:

  • In 2D low-frequency Poisson problems, the additive approach with $P1$ elements on a coarse mesh achieved the same accuracy as standard FEM with $P1$ elements on a mesh 16 times finer.
  • In high-frequency cases, the gains were even more pronounced (up to 28 times coarser mesh) because standard FEM struggles with oscillations on coarse grids, while the PINN prior captures the oscillatory nature.
  • For 3D problems, the enriched approach reduced the online computation time by a factor of ~1471 for a target error of $10^{-3}$ compared to standard FEM.

• Parametric Speed-up:

  • When solving a set of 100 parameter instances, the enriched method became more efficient than standard FEM after solving just 19 instances (including the offline PINN training cost). This makes the method highly suitable for real-time or many-query scenarios.

• Derivative Importance:

  • Experiments comparing data-driven priors vs. physics-informed priors (PINNs) confirmed that Sobolev training (learning derivatives) is critical. Data-driven networks often fail to capture high-order derivatives, leading to poor performance in the enriched FEM, whereas PINNs excel here.

• Non-Smooth Problems:

  • For a 1D transmission problem with discontinuous coefficients, a standard PINN failed to capture the solution jump. However, using an enriched architecture (incorporating the singularity) as the prior restored the method's effectiveness, proving the framework's flexibility.

5. Significance and Impact

• Bridging the Gap: This work successfully bridges the gap between the mesh-free, data-hungry nature of deep learning and the rigorous, mesh-based reliability of FEM. It provides a "best of both worlds" solution: the speed of neural networks for parametric exploration and the certified accuracy of finite elements.
• Computational Efficiency: By enabling the use of coarse meshes, the method significantly reduces the degrees of freedom (DoFs) required for high-accuracy solutions. This is particularly transformative for 3D and high-dimensional parametric problems where standard FEM is often prohibitive.
• Robustness: The additive approach is shown to be robust across various geometries (squares, annuli, cubes), boundary conditions (Dirichlet, Robin, mixed), and physical regimes (convection-dominated, anisotropic).
• Future Directions: The paper suggests that this framework can be extended to non-linear equations, used for adaptive mesh refinement (via a posteriori error estimators), and potentially combined with Neural Operators for even better generalization.

In conclusion, the paper presents a mathematically grounded and numerically validated framework that uses neural networks not as a replacement for FEM, but as a powerful prior to enhance FEM, achieving significant computational savings while maintaining rigorous error bounds.