Guaranteeing Conservation of Integrals with Projection in Physics-Informed Neural Networks

Technical Summary: Guaranteeing Conservation of Integrals with Projection in Physics-Informed Neural Networks

Problem Statement
Physics-Informed Neural Networks (PINNs) utilize soft constraints to enforce the structure of partial differential equations (PDEs) and boundary conditions during training. While this approach offers flexibility, it permits learned solutions to violate fundamental physical laws, specifically conservation laws. Standard soft constraints penalize deviations from conservation but do not guarantee that the solution strictly obeys them. The authors argue that for PINNs to be trustworthy in modeling real-world physical systems, they must satisfy conservation laws as hard constraints, ensuring that integral quantities (such as total momentum or energy) remain constant or evolve exactly according to the governing physics, regardless of data limitations or training dynamics.

Methodology
The paper proposes a novel framework, PINN-Proj, which guarantees the conservation of integral quantities by projecting the neural network's output into a functional space where conservation is strictly satisfied. This is achieved through a differentiable projection layer applied after the network's forward pass.

1. Projection Formulation: The method treats the conservation of an integral quantity as a constrained optimization problem. The goal is to find a prediction vector $y$ that minimizes the distance to the raw neural network output $\bar{u}_\theta$ while satisfying a specific integral constraint (linear or quadratic).

   • Linear Integral: For a conserved quantity defined as $c(t) = \int u \, dx$, the authors derive a closed-form projection using Lagrange multipliers. The solution shifts the raw output by a constant term to satisfy the linear sum constraint.
   • Quadratic Integral: For a quantity defined as $c(t) = \int u^2 \, dx$, the method solves a non-linear constrained optimization problem. The resulting projection scales the raw output vector to satisfy the quadratic sum constraint.
   • Combined Constraints: The framework extends to simultaneously satisfy both linear and quadratic constraints by solving a joint optimization problem, yielding a specific projection formula that adjusts the raw output to lie on the intersection of the two constraint manifolds.

2. Handling Non-Conserved Systems: The method is generalized to systems with sources, sinks, or boundary fluxes (non-conserved systems). In these cases, the target integral $c(t)$ is not constant. The authors propose estimating missing $c(t)$ values via linear interpolation using second-order accurate gradients from defined time steps, allowing the projection to adapt to time-varying conservation targets.

3. Dimensionality: The approach is extended to 2D systems by integrating over both spatial dimensions to compute the conserved quantity, maintaining the same projection logic.

Key Contributions

• Novel Projection Method: The authors introduce a differentiable projection technique that guarantees the exact conservation of linear and quadratic integrals, both separately and jointly, within a PINN architecture.
• Analytical Derivations: The paper provides explicit mathematical derivations for the projection formulae by solving constrained non-linear optimization problems for both linear and quadratic cases.
• Hard Constraint Implementation: Unlike prior work that relies on soft penalties or iterative updates, this method incorporates conservation directly as a hard constraint layer, ensuring the output strictly adheres to physical laws.
• Generalizability: The framework is demonstrated to work on both conserved and non-conserved systems, as well as 1D and 2D PDEs.

Experimental Results
The authors evaluated PINN-Proj against a standard PINN and a PINN with a soft constraint on conservation (PINN-SC) across five PDE datasets: Reaction-Diffusion, 1D/2D Advection, Wave, and Korteweg-De Vries (KdV) equations.

• Conservation Accuracy: PINN-Proj significantly outperformed both baselines in conserving integral quantities. It reduced the error in conservation ($Error_c$) by three to four orders of magnitude compared to PINN-SC. In many cases, the conservation error reached approximately $10^{-6}$, approaching machine precision limits (though limited by discretization and solver errors in the ground truth data).
• Solution Accuracy: While the primary goal was conservation, PINN-Proj also marginally improved the accuracy of the PDE solution prediction ($Error_u$) in 11 out of 15 test configurations compared to the baselines.
• Convergence and Conditioning: Spectral analysis of the Hessian matrix of the loss function revealed that PINN-Proj generally improved the conditioning of the loss landscape. The eigenvalues were more tightly clustered around zero compared to standard PINNs, suggesting that the projection constrains the search space and guides gradient descent more effectively. Consequently, PINN-Proj required fewer training epochs to converge than both PINN and PINN-SC, despite the increased computational cost per epoch.
• Computational Cost: The projection method increased training time (due to evaluating the network over the full domain and additional summation calculations) by an average of 6.5x for 1D and 33x for 2D systems compared to standard PINNs. However, the reduction in epochs partially offset this cost.

Significance and Claims
The paper claims that this method provides a general framework to guarantee the conservation of any integral quantity in a PINN, provided a tractable differentiable solution to the corresponding constrained optimization problem exists. The authors emphasize that the "guarantee" ensures that the only sources of error in conservation are data noise and machine precision, rather than the methodological failure to enforce the law.

The significance lies in bridging the gap between the flexibility of soft-constrained PINNs and the strict physical fidelity required for safety-critical applications (e.g., chemical reactors, inverse design). By demonstrating that hard constraints can be implemented via differentiable projection layers without sacrificing convergence speed or solution accuracy, the work suggests a path toward more physically trustworthy neural solvers. The authors note that while the method improves conditioning and accuracy, the benefits are most pronounced in complex PDEs where the loss landscape is otherwise ill-conditioned.