Locally Subspace-Informed Neural Operators for Efficient Multiscale PDE Solving

1. Problem Statement

The paper addresses the challenge of solving heterogeneous Partial Differential Equations (PDEs) characterized by high-contrast coefficients across multiple scales (e.g., porous media flow, diffusion in complex materials).

• The Trade-off:
  • Generalized Multiscale Finite Element Method (GMsFEM): A robust numerical method that captures fine-scale features by constructing localized spectral basis functions on coarse grids. However, it suffers from high computational costs during the "offline stage" because it requires solving expensive local eigenvalue problems for every new coefficient field.
  • Neural Operators (NOs): Data-driven models (like FNOs, DeepONets) that learn the solution operator directly. While fast, standard NOs often struggle with high-contrast heterogeneities, require massive datasets, and lack robustness when the forcing terms (right-hand side) change (out-of-distribution failure).
• Goal: Develop a hybrid framework that retains the robustness and accuracy of GMsFEM while achieving the computational speed of Neural Operators, specifically by accelerating the expensive offline basis generation.

2. Methodology: GMsFEM-NO

The proposed GMsFEM-NO framework replaces the traditional offline spectral solver of GMsFEM with a Neural Operator that predicts the subspace of multiscale basis functions.

A. Core Architecture

1. Local Domain Decomposition: The computational domain is divided into local subdomains ($\omega_i$) based on a coarse grid. These subdomains vary in geometry (full, half, corner in 2D; full, half, quarter, corner in 3D).
2. Specialized Neural Operators: Instead of one global network, the authors train separate NOs (specifically Factorized Fourier Neural Operators, F-FNO) for each geometric type of local domain.
3. Input/Output:
   • Input: The heterogeneous coefficient field $\kappa(x)$ restricted to a local domain.
   • Output: The predicted subspace spanned by the first $N_{bf}$ multiscale basis functions.
4. Preprocessing: Local domains are normalized (rotated) to ensure consistent input structures for the NOs based on their geometric type.

B. Key Innovation: Subspace-Informed Loss Functions

A critical contribution is the design of loss functions that learn the subspace rather than individual basis functions. This addresses the issue that basis functions are defined only up to a sign and rotation.

• Subspace Alignment Loss (SAL): Measures the geometric alignment between the predicted subspace and the target subspace using the Frobenius norm of the product of their orthonormalized bases ($Q_R$ and $Q_{\hat{R}}$).
  $$L_{SAL} = \mathbb{E}_i \left[ N_{bf} - \|Q_{R_i}^\top Q_{\hat{R}_i}\|_F^2 \right]$$
  This minimizes the Grassmannian distance between the true and predicted subspaces.
• Projection Regularization (PR): An additional term ($L_{SAL-PR}$) that enforces consistency in how vectors are projected onto the subspace, ensuring the learned basis captures specific functional behaviors, not just geometric overlap.
• Comparison: These are shown to outperform the standard Relative Basis Function $L_2$ loss ($L_{RBFL2}$), which struggles with sign invariance and subspace coherence.

C. Inference Pipeline

1. The trained NOs predict the basis functions for each local domain.
2. These bases are assembled into a global restriction matrix $\hat{R}$.
3. The fine-scale system is projected onto the coarse space: $\hat{A}_0 = \hat{R} A \hat{R}^\top$.
4. The coarse-scale system is solved, and the solution is reconstructed.

3. Key Contributions

1. Novel Hybrid Framework: Introduces GMsFEM-NO, which decouples the expensive offline basis generation from the online solve, accelerating the process by >60x.
2. Subspace-Aware Learning: Proposes the Subspace Alignment Loss (SAL), which learns the entire relevant solution subspace. This is more data-efficient and robust than learning individual functions or full PDE solutions.
3. Resolution Invariance: Demonstrates that models trained on coarse grids can be effectively applied to finer grids without retraining, a significant advantage for scalability.
4. Forcing Term Independence: Unlike standalone NOs that learn the mapping from coefficient to solution (which depends on the forcing term $f$), GMsFEM-NO learns the mapping from coefficient to basis. This allows the solver to handle arbitrary forcing terms without retraining, preserving the generalization capabilities of GMsFEM.
5. Robustness: Achieves superior performance on high-contrast problems compared to state-of-the-art NOs (F-FNO, GNOT, Transolver++), particularly in out-of-distribution scenarios.

4. Experimental Results

The method was validated on 2D and 3D benchmarks:

• Linear Elliptic Diffusion Equation: With high-contrast coefficients (range [1, 9600]).
• Nonlinear Steady-State Richards Equation: Modeling water movement in unsaturated porous media.

Key Findings:

• Accuracy: GMsFEM-NO achieves solution errors ($L_2$ and $H_1$ norms) nearly identical to standard GMsFEM (often within 0.1% difference) and significantly better than standalone NOs.
  • Example: On the Richards equation with complex forcing, GMsFEM-NO reduced error by 2.8x compared to the best standalone NO.
• Speed: Basis generation time was reduced by >60x (e.g., from ~10,500 seconds to ~1.33 seconds for a $100^3$ grid).
• Data Efficiency: GMsFEM-NO requires roughly half the training samples of standalone NOs to achieve similar accuracy. It remains stable even with small datasets (200 samples), whereas standalone NOs degrade significantly.
• Out-of-Distribution (OOD) Performance: Standalone NOs failed catastrophically (errors >100%) when tested on forcing terms different from training data. GMsFEM-NO maintained low errors because the basis functions are independent of the forcing term.
• Scalability: Models trained on $100 \times 100$ grids successfully solved problems on $500 \times 500$ grids with minimal accuracy loss.

5. Significance and Impact

• Paradigm Shift: The paper establishes a new paradigm where Neural Operators are used not to replace the numerical solver, but to accelerate the construction of the solver's mathematical structure (the basis functions).
• Practical Applicability: By retaining the rigorous numerical foundation of GMsFEM, the method ensures physical consistency and stability, addressing the "black box" reliability concerns of pure deep learning approaches.
• Efficiency: The massive speedup in the offline stage makes high-fidelity multiscale simulations feasible for real-world applications (e.g., reservoir simulation, material science) where repeated solves with varying parameters are required.
• Future Directions: The authors note current limitations regarding structured grids and plan to extend the framework to unstructured meshes and more complex PDEs, leveraging geometry-aware NOs.

In summary, GMsFEM-NO successfully bridges the gap between the accuracy of traditional numerical methods and the speed of machine learning, offering a robust, scalable, and highly efficient solver for multiscale PDEs.