The Coercivity Gap in Neural PDE Solvers: Parameter Escape and Functional Convergence

Technical Summary: The Coercivity Gap in Neural PDE Solvers

1. Problem Formulation and the Coercivity Gap
The paper addresses a fundamental disconnect in variational partial differential equation (PDE) solvers based on neural-network ansatzes, such as Physics-Informed Neural Networks (PINNs) and the Deep Ritz method. In these settings, the PDE energy is minimized over a nonlinear family of realizations rather than a linear trial space. The authors identify and formalize the coercivity gap: a phenomenon where the PDE energy is coercive on the physical state space, but this coercivity does not control the parameters of the neural representation. Consequently, minimizing sequences may exhibit parameter escape (diverging weights or centers) while the corresponding realized states remain bounded and converge strongly in the natural energy norm.

The authors distinguish this from standard approximation theory issues by emphasizing that parameter divergence is not necessarily indicative of PDE approximation failure. Instead, it reflects an ill-conditioned or non-unique parametrization where the target state lies on the boundary of the realization class, requiring parameters to escape to infinity to approach it.

2. Methodology and Abstract Framework
The analysis proceeds in two stages: an abstract Hilbert-space formulation and a concrete model using fixed-variance Gaussian radial-basis functions.

• Abstract Setting: Let $X$ be the physical state space (e.g., $H^1(\mathbb{R}^d)$) and $E: X \to \mathbb{R}$ be a convex, coercive, and strongly convex energy functional. Let $\Phi_P: \Theta_P \to X$ be a realization map from parameter space $\Theta_P$ to the state space, generating a realization class $A_P = \Phi_P(\Theta_P)$. The authors prove that if $E$ is $\lambda$-strongly convex, then near-minimal energy errors imply strong convergence of the realized states in $X$, regardless of whether the parameters remain bounded. Specifically, if $v_P \in A_P$ satisfies $E(v_P) \le \inf_{v \in A_P} E(v) + \epsilon_P$, then $\|v_P - u\|_X^2 \le \frac{2}{\lambda}(\alpha_P + \epsilon_P)$, where $u$ is the exact minimizer and $\alpha_P$ is the best approximation error.
• Gaussian Elliptic Model: To demonstrate the gap concretely, the authors consider the whole-space elliptic equation $-\Delta u + u = f$ in $\mathbb{R}^d$. The realization class $M_P$ consists of finite mixtures of fixed-variance Gaussians:
  $$\Phi_P(\Theta)(x) = \sum_{j=1}^P w_j G(x - x_j), \quad G(x) = (4\pi)^{-d/2}e^{-|x|^2/4}.$$
  The energy $J(v) = \frac{1}{2}\|v\|_{H^1}^2 - \langle f, v \rangle$ is 1-strongly convex, satisfying the identity $J(v) - J(u) = \frac{1}{2}\|v - u\|_{H^1}^2$.

3. Key Contributions and Results

• Collision Mechanism and Nonattainment: The core mechanism driving the coercivity gap is the collision of active neurons. The authors construct a sequence of two-neuron realizations $q_h(x) = \frac{G(x+he_1) - G(x-he_1)}{2h}$, which represents a difference quotient. As $h \to 0$, the centers collide and the weights diverge ($\|w\| \sim h^{-1}$), yet the realized states converge strongly in $H^1(\mathbb{R}^d)$ to the Gaussian derivative $\partial_1 G$. Since $\partial_1 G$ does not belong to any finite-dimensional realization class $M_P$ (proven via Fourier analysis showing $\partial_1 G$ has unbounded Fourier transform relative to the bounded exponential decay of Gaussian translates), the realization class $M_P$ is non-closed.
• Nonattainment of Infimum: For the specific forcing term $f = (-\Delta + I)\partial_1 G$, the exact solution is $u = \partial_1 G$. The authors prove that for any $P \ge 2$, the reduced infimum $I_P = \inf_{v \in M_P} J(v)$ equals the exact minimum $I = J(u)$, but this infimum is not attained in $M_P$. Consequently, every parameter-space minimizing sequence must escape to infinity ($\|\Theta_k\| \to \infty$).
• Quantitative State Convergence: Despite parameter escape, the states converge. The authors derive explicit approximation rates. By combining Hermite spectral truncation with finite-difference realization of Gaussian jets, they establish that for $u \in H^1_1(\mathbb{R}^d) \cap H^2(\mathbb{R}^d)$ (where $H^1_1$ denotes a weighted Sobolev space), the best approximation error satisfies:
  $$\inf_{w \in M_P} \|u - w\|_{H^1} \le \frac{C_d}{\sqrt{\log(2+P)}} (\|u\|_{H^1_1} + \|u\|_{H^2}).$$
  This yields a logarithmic convergence rate $O((\log P)^{-1/2})$ for the state error, independent of parameter boundedness.
• Measure-Valued Relaxation: The authors analyze a convex relaxation where finite sums are replaced by signed Radon measures $\mu$. They show that while the relaxed problem attains the correct limiting state, the representing measures may still diverge in total variation norm. Thus, convex relaxation does not inherently resolve the lack of coercivity in the representing parameters unless a total-variation penalty (Tikhonov regularization) is explicitly added.
• Residual Minimization and HYCO: The coercivity gap extends to residual-based methods (PINNs). If the residual is measured in a stability norm (e.g., $H^{-1}$ for second-order elliptic problems), small residuals imply state convergence, but not parameter compactness. The authors also formulate a hybrid physics-data (HYCO) framework in Hilbert spaces compatible with PDE stability, noting that while the hybrid state problem is well-posed and strongly convex, the finite-dimensional parameterization may still exhibit escape.

4. Significance and Claims
The paper claims to isolate a general mechanism where variational neural PDE solvers are coercive in the realized state but non-coercive in the parameters. Its significance lies in:

1. Mathematical Clarification: Providing a rigorous explanation for the common computational observation that neural PDE solvers can produce accurate physical fields even when their weights or centers are unbounded or hard to interpret.
2. Separation of Concerns: Distinguishing between state stability (controlled by PDE energy/residual norms) and parameter compactness (dependent on architecture and regularization). The authors argue that parameter escape should be viewed as a conditioning or identifiability issue rather than a failure of the PDE approximation.
3. Model Mechanism: Using the fixed-variance Gaussian collision as a transparent model to demonstrate that non-closedness of realization classes leads to nonattainment and parameter escape, even when the target state is arbitrarily well-approximated in the energy norm.

The authors explicitly state that this is not a claim of architectural optimality for Gaussians, but rather a demonstration of a mechanism that persists across various architectures whenever the realization map has non-compact fibers or a non-closed image. The work does not address algorithmic training dynamics (e.g., gradient descent convergence) but focuses on the variational properties of global or near-global minimizing sequences.