Born-Series-Inspired Residual Metric for Learning-based Preconditioners

1. Problem Statement

The paper addresses the challenge of solving indefinite Partial Differential Equations (PDEs), specifically the high-frequency heterogeneous Helmholtz equation, using learning-based solvers.

• The Core Issue: Standard learning-based PDE solvers (e.g., Neural Operators) typically optimize an unpreconditioned Euclidean residual norm ($\|r\|_2$). For indefinite operators like the Helmholtz equation, this metric creates a "stiff" optimization landscape where near-resonant spectral components dominate the loss, making both learning and iterative correction inefficient.
• The Gap: While classical preconditioning (e.g., Shifted-Laplacian) successfully reshapes the residual geometry to improve convergence, learning-based methods often ignore this geometric structure, leading to suboptimal performance in highly ill-conditioned or heterogeneous media.

2. Methodology

The authors propose a framework that unifies the Born Series (integral equation theory) with Shifted-Laplacian preconditioning to derive a new training metric and solver architecture.

A. Theoretical Foundation: The Born-Preconditioner Identity

The paper establishes a key algebraic identity linking the Born Series and left preconditioning.

• Let $A$ be the Helmholtz operator and $L_\eta = -\Delta - (k_0^2 + i\eta)$ be a damped reference operator (Shifted Laplacian) with inverse $G_\eta = L_\eta^{-1}$.
• Defining the perturbation $V_\eta = L_\eta - A$, the authors prove:
  $$I - G_\eta V_\eta = G_\eta A = L_\eta^{-1} A$$
• Implication: The integral residual in the Born Series is algebraically equivalent to the left-preconditioned residual. This suggests that the natural metric for measuring residuals is not Euclidean, but the Riesz-map metric induced by the Green operator:
  $$R_\eta = G_\eta^* G_\eta, \quad \|r\|_{R_\eta} = \|G_\eta r\|_2$$

B. Neural Preconditioned Born Series (NPBS)

The authors introduce NPBS, a learned iterative solver that generalizes the Convergent Born Series (CBS).

• Mechanism: Instead of using a scalar relaxation parameter ($\gamma$) as in classical CBS, NPBS employs a neural operator ($M_\theta$) to map the preconditioned residual to a correction.
• Iteration Step:
  $$u_{m+1} = u_m + M_\theta(G_\eta(V_\eta u_m + f) - u_m)$$
  This operates in preconditioned coordinates, where the spectrum of the operator is clustered near 1 (if $\|G_\eta V_\eta\| < 1$), facilitating faster convergence.
• Efficiency: The implementation leverages Fast Fourier Transforms (FFT) or Discrete Sine/Cosine Transforms (DST/DCT) to apply $G_\eta$ in $O(N \log N)$ time.

C. Metric-Matched Training Objectives

To ensure the neural network learns the correct mapping for the NPBS iteration, the authors propose two specific loss functions, moving beyond the standard Euclidean loss:

1. $L_{\ell_2}^{bs}$ (Bridge Loss): Trains the network to approximate the inverse in preconditioned coordinates but measures error in Euclidean space.
2. $L_{R_\eta}^{bs}$ (Born-Series-Inspired Loss): The primary contribution. It measures the residual in the preconditioned metric $R_\eta$:
   $$L_{R_\eta}^{bs}(\theta) = \mathbb{E} \left[ \frac{\|(I - G_\eta V_\eta) M_\theta(G_\eta r) - G_\eta r\|_2^2}{\|G_\eta r\|_2^2} \right]$$
   This aligns the training gradient with the inference geometry, ensuring the network learns to correct errors in the same metric used during the iterative solve.

3. Key Contributions

1. NPBS Solver: A novel architecture that replaces scalar corrections in the Born Series with a residual-driven neural operator, compatible with various neural operator backbones (e.g., FNO).
2. Residual Metric Theory: Derivation of the $R_\eta$ metric based on the identity between Born Series and Shifted-Laplacian preconditioning, providing a theoretical justification for metric-matched training.
3. Metric-Matched Loss: Development of the $L_{R_\eta}^{bs}$ loss function, which significantly improves optimization stability for indefinite operators by reweighting spectral components according to the preconditioner.
4. Generalization: Demonstration that this framework extends beyond Helmholtz equations to Convection-Diffusion-Reaction (CDR) equations and linearized Newton systems for nonlinear PDEs.

4. Experimental Results

The framework was evaluated on heterogeneous Helmholtz problems (OpenFWI datasets), CDR equations, and nonlinear PDEs.

• Helmholtz Equation (Heterogeneous Media):

  • Performance: NPBS trained with $L_{R_\eta}^{bs}$ required 1.6x to 2.0x fewer iterations than direct Euclidean baselines.
  • Ill-Conditioning: The advantage grew as the problem became more difficult (higher wavenumbers, sharper velocity contrasts). For the most challenging "CurveFault-B" dataset at 12 points-per-wavelength, NPBS reduced iterations from 1150 (baseline) to 561.
  • Comparison: NPBS significantly outperformed classical CBS (scalar correction), reducing iterations by ~3x on difficult datasets.

• Convection-Diffusion-Reaction (CDR):

  • The method generalized effectively. NPBS + $L_{R_\eta}^{bs}$ converged in 59 iterations, compared to 4,533 iterations for the direct baseline, demonstrating massive efficiency gains even for transport-type equations.

• Nonlinear PDEs (Newton Linearization):

  • Applied to the inner linear systems of Newton's method for a non-convex problem. NPBS reduced inner-loop iterations by over 4x compared to the baseline. Interestingly, for these relatively well-conditioned linearized systems, the simpler $L_{\ell_2}^{bs}$ loss sometimes outperformed the full metric-matched loss, suggesting adaptability based on problem conditioning.

5. Significance and Conclusion

• Theoretical Insight: The paper reframes the design of learning-based preconditioners as a metric selection problem. It proves that optimizing in the wrong metric (Euclidean) for indefinite operators leads to suboptimal learning, whereas aligning the loss with the preconditioner's geometry yields robust convergence.
• Practical Impact: The proposed method offers a principled way to accelerate iterative solvers for high-frequency wave propagation and other ill-conditioned PDEs, significantly reducing the number of iterations required for convergence.
• Limitations & Future Work: While the method drastically reduces iteration counts, the authors note that the wall-clock time is currently dominated by the inference overhead of the neural network. Future work must focus on designing computationally efficient neural architectures to realize end-to-end speedups. The framework is also poised for extension to 3D problems and time-dependent evolution equations.