Physics-Informed Neural Networks for Solving Derivative-Constrained PDEs

This paper introduces Derivative-Constrained PINNs (DC-PINNs), a general framework that enhances Physics-Informed Neural Networks by embedding general nonlinear constraints on states and derivatives via automatic differentiation and self-adaptive loss balancing, thereby stabilizing training and ensuring physically admissible solutions for problems requiring derivative-based relations beyond standard PDE residuals.

The Gist

Imagine you are trying to teach a robot to predict how heat spreads through a metal rod, how stock prices move, or how water flows around a boat. You give the robot a set of rules (the laws of physics, written as complex math equations) and ask it to learn the pattern.

This is what Physics-Informed Neural Networks (PINNs) do. They are like students who are given a textbook (the physics equations) and a few test questions (data points). They try to learn the answer by minimizing their mistakes on the test.

The Problem:
Sometimes, just following the textbook isn't enough. In the real world, there are "common sense" rules that the textbook doesn't explicitly state but are absolutely critical.

• Heat: Heat always flows from hot to cold; it never spontaneously concentrates in one spot.
• Stocks: A stock price shouldn't be able to go up and down in a way that guarantees a free lunch (arbitrage).
• Water: Water can't just disappear or appear out of thin air; the amount flowing in must equal the amount flowing out.

If you only teach the robot the main textbook equations, it might come up with a mathematically "correct" answer that is physically impossible—like a stock price that creates infinite money or water that flows uphill. The robot gets the math right but the reality wrong.

The Solution: DC-PINNs (Derivative-Constrained PINNs)
The authors of this paper, Kentaro, Carolyn, and Paolo, created a new method called DC-PINNs. Think of this as upgrading the robot's teacher from a strict textbook reader to a wise coach who understands the spirit of the game.

Here is how they did it, using some everyday analogies:

1. The "Guardrails" Analogy

Imagine driving a car (the neural network) down a winding mountain road (the solution to the equation).

• Standard PINNs: The car has a GPS (the physics equations) telling it where to go. But if the GPS is slightly off, the car might drive off the cliff because it's only trying to follow the GPS coordinates.
• DC-PINNs: The authors added guardrails to the road. These guardrails represent the "derivative constraints."
  • Constraint: "The car must never go faster than the speed limit."
  • Constraint: "The car must never drive backward."
  • Constraint: "The road must always slope downward if it's a downhill section."

Even if the GPS (the main equation) gets confused, the guardrails force the car to stay on the safe, physically possible path. In math terms, these guardrails ensure that the rate of change (derivatives) of the solution makes sense (e.g., heat doesn't get hotter as it spreads).

2. The "Tug-of-War" Coach (Self-Adaptive Balancing)

One of the hardest parts of training these robots is balancing the different rules.

• Rule A: "Match the data points."
• Rule B: "Follow the physics equations."
• Rule C: "Don't break the guardrails."

Sometimes, Rule C is very hard to satisfy, and the robot gets stuck. If you tell the robot "Rule C is super important!" too loudly, it might ignore the data. If you say it's "not important," it breaks the laws of physics.

The authors introduced a Self-Adaptive Coach. Imagine a coach who watches the robot struggle.

• If the robot is failing to stay within the guardrails, the coach whispers, "Hey, pay a little more attention to the guardrails!"
• If the robot is ignoring the data points, the coach says, "Look at the data!"
• The coach automatically adjusts the volume of each instruction so the robot doesn't get overwhelmed by one loud rule. This removes the need for a human to guess the perfect settings (hyperparameters) beforehand.

3. Real-World Tests

The team tested their new "Guardrail Robot" on three difficult scenarios:

• The Heat Diffusion Test: They simulated heat spreading through a beam. Standard robots made the heat "wiggle" or go up and down in impossible ways. The DC-PINN robot kept the heat flowing smoothly and naturally, respecting the rule that heat always dissipates.
• The Stock Market Test: In finance, you can't have a "free lunch" (arbitrage). Standard robots sometimes created stock price charts that looked weird and allowed for impossible profits. The DC-PINN robot created smooth, logical price curves that respected the "no free lunch" rule.
• The Water Flow Test: They simulated water swirling around a cylinder (like a boat piling up water). Standard robots sometimes made the water compress or vanish. The DC-PINN robot ensured the water remained incompressible (it doesn't squish), creating a realistic flow pattern with swirling vortices.

The Bottom Line

DC-PINNs are a smarter way to teach computers to solve physics problems. Instead of just memorizing the equations, they are taught to respect the fundamental "common sense" rules of the universe (like conservation of energy or money).

By adding these "guardrails" and a "smart coach" to balance the rules, the AI produces solutions that are not just mathematically close, but physically real. It's the difference between a student who memorizes the answers and a student who truly understands the subject.

Technical Summary

1. Problem Statement

While Physics-Informed Neural Networks (PINNs) have successfully recast Partial Differential Equation (PDE) solving as an optimization problem by minimizing PDE residuals, they often fail to enforce derivative-based constraints that are fundamental to physical realism.

• The Gap: Many real-world applications require solutions to satisfy inequality constraints on derivatives (e.g., monotonicity, convexity, bounds, incompressibility, no-arbitrage conditions). Standard PINNs treat these as secondary or ignore them, leading to solutions that are mathematically feasible (low PDE residual) but physically implausible (e.g., negative volatility, non-monotonic price surfaces, or non-physical fluid wakes).
• The Challenge: Incorporating these constraints introduces a multi-objective optimization problem. Balancing the PDE residual, boundary conditions, and derivative inequality penalties is difficult due to differing magnitudes and sensitivities, often requiring delicate manual hyperparameter tuning and leading to unstable training or slow convergence.

2. Methodology: Derivative-Constrained PINNs (DC-PINNs)

The authors propose DC-PINNs, a general framework that treats constrained PDE solving as an optimization guided by a minimum principle where the physics resides in the constraints themselves.

A. Problem Formulation

The goal is to find a mapping $u_\theta$ approximating the solution $u$ that minimizes a loss function subject to:

1. PDE Residuals: $f(x, Du_\theta) = 0$ in the domain.
2. Boundary/Initial Conditions: $b(x, Du_\theta) = 0$.
3. Derivative Inequalities: $h(x, D_h u_\theta) \leq 0$ (e.g., $\nabla u \geq 0$, $\nabla^2 u \geq 0$).

B. Loss Function Design

The total loss is a weighted sum of four components:
$$\mathcal{L} = \lambda_0 \hat{\mathcal{L}}_0 + \lambda_f \hat{\mathcal{L}}_f + \lambda_b \hat{\mathcal{L}}_b + \lambda_h \hat{\mathcal{L}}_h$$

• Inequality Penalty ($\hat{\mathcal{L}}_h$): Uses a one-sided hinge loss $[h(x, D_h u_\theta)]_+^2$ (where $[z]_+ = \max(0, z)$). This yields zero gradients within the feasible region and a linear signal upon violation, ensuring numerical stability when combined with automatic differentiation (AD).
• Scaling Factors:
  • Sample-wise weights ($m_\chi$): Diagonal matrices applied to individual residual components to handle scaling differences between physical quantities.
  • Category-wise weights ($\lambda_\chi$): Scalar coefficients balancing the importance of data, PDE, boundary, and constraint terms.

C. Self-Adaptive Loss Balancing

To eliminate manual hyperparameter tuning, DC-PINNs employ two automated balancing mechanisms updated via gradient ascent during training:

1. Individual Loss Balancing: Updates sample-wise weights ($m$) to amplify gradients for components with larger violations.
2. Categorised Loss Balancing: Updates category weights ($\lambda$) based on the mean absolute gradient magnitude of each loss term. This ensures that sparse or stiff inequality constraints (which often have zero gradients in feasible regions) are not ignored during optimization.

D. Baselines for Comparison

The paper compares DC-PINNs against:

• Standard PINNs: No derivative constraints.
• PINNs+Ineq (Fixed): Adds inequality penalties with fixed global weights.
• Hard-Constraint Formulations:
  • hPINN (pen.): Penalty method with increasing weights.
  • hPINN (AL) & AL-PINNs: Augmented Lagrangian methods with dual variables.

3. Key Contributions

1. General Framework: A unified approach to embedding general nonlinear derivative constraints (bounds, monotonicity, convexity) directly into the PINN loss function.
2. Self-Adaptive Balancing: A novel mechanism that automatically tunes loss weights, reducing reliance on problem-specific architecture and manual hyperparameter tuning.
3. Stabilization: Explicitly encoding derivative constraints stabilizes training and steers optimization toward physically admissible minima, even when the PDE residual is small.
4. Comprehensive Benchmarking: Extensive validation across three distinct domains: Thermodynamics (Heat Equation), Quantitative Finance (Volatility Surface), and Fluid Dynamics (Navier-Stokes).

4. Experimental Results

A. 1D Heat Equation (Thermodynamics)

• Task: Solve heat diffusion with constraints on concavity ($\partial_{xx} u \leq 0$) and time decay ($\partial_t u \leq 0$).
• Results: DC-PINNs achieved the lowest RMSE for both the solution and its derivatives. Standard PINNs exhibited noisy oscillations and boundary bias. Hard-constraint baselines (AL-PINNs) were "over-smoothed," losing temporal variation. DC-PINNs maintained spatial smoothness and correct temporal decay.

B. Volatility Surface Calibration (Finance)

• Task: Calibrate a local volatility surface satisfying no-arbitrage conditions (monotonicity and convexity of option prices).
• Results: DC-PINNs produced smooth, arbitrage-free surfaces consistent with the ground truth. Standard and fixed-penalty PINNs violated constraints near strike boundaries. Hard-constraint methods showed optimization instability and poor data fitting. DC-PINNs successfully balanced data fidelity with physical admissibility.

C. Navier-Stokes Equations (Fluid Dynamics)

• Task: Simulate flow past a cylinder (von Kármán vortex street) with incompressibility ($\nabla \cdot V = 0$) and pressure gradient bounds.
• Results: While hard-constraint baselines (AL-PINNs) sometimes achieved lower raw data error, DC-PINNs significantly reduced constraint violations (divergence and pressure bounds) and improved stability. The method successfully recovered the viscosity parameter ($\mu_2$) and reconstructed coherent vortex shedding patterns.

D. Computational Efficiency

• DC-PINNs incur a 1.5x to 2x training time overhead compared to standard PINNs due to additional derivative evaluations and adaptive weight updates.
• However, this cost is offset by systematic gains in physical consistency and reduced oscillation during training. The method is most beneficial when derivative constraints dominate the ill-posedness of the problem.

5. Significance and Conclusion

The paper demonstrates that explicitly encoding derivative constraints is crucial for solving PDEs where physical principles (like energy minimization or no-arbitrage) are as fundamental as the governing equations.

• Reliability: DC-PINNs provide reliable solutions grounded in minimum principles, preventing the generation of "mathematically feasible but physically implausible" results.
• Automation: The self-adaptive balancing removes the burden of delicate hyperparameter tuning, making constrained PDE solving more accessible and robust.
• Scalability: The framework scales to higher dimensions (tested up to 128D heat equation) and complex multi-field systems without architectural changes, relying on automatic differentiation for mixed derivatives.

In summary, DC-PINNs represent a significant step forward in making Physics-Informed Deep Learning a viable tool for complex, constrained real-world applications where strict adherence to physical laws is non-negotiable.