🤖 machine learning

A derivative-fidelity failure mode in physics-informed neural networks: strengthened benchmark evidence from function-value training

This paper identifies and validates a "derivative-fidelity failure mode" in physics-informed neural networks, demonstrating that models trained solely on function values can achieve visually accurate approximations while producing significantly erroneous second derivatives, particularly in high-curvature regions.

Original authors: Koji Koyamada

Published 2026-09-15✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Koji Koyamada

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern computing, scientists are increasingly turning to a type of artificial intelligence called a physics-informed neural network to solve complex problems involving how things move, flow, or change over time. These systems act as powerful mathematical surrogates, learning to mimic the behavior of physical laws not just by memorizing data points, but by being trained to satisfy the underlying equations that govern reality. The appeal is profound: if a computer can learn the shape of a curve and simultaneously respect the rules of physics, it could revolutionize how engineers design bridges, how meteorologists predict storms, or how doctors model blood flow. The standard approach assumes that if the computer gets the overall shape of a solution right, it has successfully captured the physics. However, this assumption relies on a hidden premise: that getting the shape right automatically means getting the rate of change right.

A recent study challenges this premise, revealing a subtle but critical flaw in how these systems are currently validated. The research focuses on a specific failure mode where a neural network appears to have perfectly learned a function, yet fails completely when asked to calculate how that function is changing. In the language of physics, a network might predict the position of an object with high precision, but its calculation of the object's acceleration—the second derivative of its position—could be wildly inaccurate. This distinction matters because many physical laws depend entirely on these rates of change. If the acceleration is wrong, the forces driving the system are wrong, even if the position looks correct. The study demonstrates that this is not a rare glitch but a fundamental limitation that can hide in plain sight, leading researchers to believe they have a working simulation when the underlying physics are actually broken.

To investigate this, the researchers set up a controlled experiment using simple, one-dimensional mathematical curves that are easy to visualize. They chose two specific targets: a smooth, repeating wave and a curve that grows exponentially, becoming steeper and steeper as it moves to the right. They trained a standard neural network to match these curves using only the values of the points themselves, without ever explicitly teaching the network what the slopes or curvatures should be. This mimics the standard training process for many physics problems, where the network is rewarded for hitting the right numbers but is not directly corrected on how fast those numbers are changing. Once the training was complete, the researchers used a standard computer tool to automatically calculate the second derivative of the network's output and compared it against the true mathematical answer.

The results were striking. For the repeating wave, the network produced a line that overlapped the true curve almost perfectly, making it look like a flawless success. However, when the researchers looked at the second derivative, which represents the curvature of the line, they found significant errors. In specific local regions, the error in the curvature was ten to one hundred times larger than the error in the position itself. The network had learned to draw the line correctly but had failed to learn how that line was bending. The situation was even more dramatic with the exponential curve. While the network's prediction of the curve's height remained visually accurate, the calculation of its curvature collapsed near the right-hand boundary. In this high-curvature region, the error in the second derivative grew by orders of magnitude, reaching values as high as ten, while the error in the height remained tiny. This proved that a visually convincing solution can coexist with a physically broken one.

The researchers then tested whether this problem could be fixed by simply giving the computer more data or by changing the internal settings of the network. They increased the number of training points from a few dozen to over five hundred, but the large errors in the curvature near the boundary persisted. They tried different mathematical functions inside the network, known as activation functions, to see if one was better suited for this task. While some choices reduced the error slightly, none eliminated the fundamental gap between the accuracy of the position and the accuracy of the curvature. The study also showed that standard measures of success, which often look at the average error across the whole domain, were hiding the severity of the problem. When the researchers looked at the worst-case error in specific small regions, the failures were severe and localized, particularly where the curve was changing most rapidly.

This work suggests that the current method of trusting a simulation because the output looks right is insufficient. The study does not claim that these networks are useless, but rather that they have a blind spot. A network can be excellent at interpolation, filling in the gaps between known points, yet fail to capture the higher-order dynamics required for physical laws. The researchers conclude that for these systems to be reliable, scientists must stop treating the calculation of derivatives as a secondary check. Instead, they must actively monitor the errors in these rates of change, especially in regions of high curvature or near boundaries. Just as a map can show the correct location of a city but fail to show the steepness of the roads leading to it, a physics-informed neural network can show the correct state of a system while missing the forces that drive it. The path forward requires a new diagnostic protocol that separates the accuracy of the value from the reliability of the physics, ensuring that the simulation is not just a pretty picture, but a true representation of reality.

Technical Summary: A Derivative-Fidelity Failure Mode in Physics-Informed Neural Networks

Problem Statement
Physics-informed neural networks (PINNs) rely on automatic differentiation to enforce differential equation residuals, assuming that a neural network surrogate trained to minimize residuals will accurately capture both function values and their derivatives. However, a growing body of literature indicates that PINNs can exhibit failure modes where solutions match boundary or data values but violate underlying physical structures. This paper identifies a specific, elementary failure mode: derivative fidelity. The core problem is that a neural network may achieve high accuracy in approximating target function values while simultaneously producing significantly inaccurate approximations of the target derivatives. Since PINN residuals depend on these derivatives, a visually accurate solution field can mask a physically unreliable residual field, leading to distorted physical structures (e.g., streamlines or magnetic fields) even when conventional error metrics appear satisfactory.

Methodology
To isolate the relationship between function-value accuracy and derivative accuracy, the author conducted a controlled study using one-dimensional benchmark functions trained exclusively on function values, without imposing differential equation constraints.

Benchmarks: Two smooth target functions were selected: $f(x) = \sin(x)$ on $[-2\pi, 2\pi]$ and $f(x) = \exp(x)$ on $[-2, 2]$ . Their analytical second derivatives ( $f''(x) = -\sin(x)$ and $f''(x) = \exp(x)$ ) served as ground truth.
Network Architecture: A fully connected multilayer perceptron (MLP) with three hidden layers and a width of 64 was employed.
Training Protocol: The networks were trained using a value-only mean-squared-error (MSE) loss. The second derivatives were not fitted directly but were obtained via automatic differentiation (AD) post-training, mirroring the mechanism used in standard PINN residual evaluation.
Strengthened Validation: To ensure the findings were not artifacts of a single training run, the study included four comparative analyses:
1. Training-point density sweep: Varying $n_{train}$ from 32 to 512.
2. Activation function comparison: Testing tanh, sine, and GELU activations.
3. Endpoint-dense evaluation: A specific grid density increase near the high-curvature boundary of the $\exp(x)$ case.
4. Metric diversity: Evaluating both Root Mean Square Error (RMSE) and maximum absolute error to distinguish average quality from localized failures.

Key Results
The experiments demonstrated a consistent decoupling between function-value accuracy and second-derivative fidelity:

Baseline Discrepancy: For both $\sin(x)$ and $\exp(x)$ , the learned functions visually overlapped the exact curves with low error ( $\approx 10^{-3}$ ). However, the second-derivative errors were substantially larger, reaching orders of magnitude higher ( $10^{-2}$ to $10^{-1}$ for $\sin(x)$ ; up to $10^1$ for $\exp(x)$ ).
Localization of Failure: The derivative errors were not uniform. For $\exp(x)$ , the second derivative "collapsed" near the right boundary ( $x \approx 2$ ), where curvature is highest. The maximum absolute error in this region was approximately 10, despite the function value error remaining small.
Ineffectiveness of Data Density: Increasing the number of training points did not monotonically reduce second-derivative errors. For $\exp(x)$ , the second-derivative RMSE remained on the order of 1, and maximum errors remained near $10^1$ even with 512 training points.
Activation Sensitivity: The choice of activation function significantly influenced derivative fidelity. For $\sin(x)$ , the sine activation yielded the lowest second-derivative RMSE, while tanh produced the largest distortion. Crucially, the activation most favorable for function values was not necessarily the same as that for derivative fidelity.
Metric Divergence: Maximum absolute error metrics revealed severe localized failures that were obscured by average RMSE metrics or visual inspection of the function plots.

Significance and Claims
The paper posits that derivative fidelity is a distinct failure mode that must be diagnosed separately from value accuracy. The author claims that:

Value accuracy is not a surrogate for derivative reliability: A low value-based training loss and visually plausible solution fields do not guarantee that the differentiated output preserves the underlying physical structure.
Hidden Failure Mechanism: This gap explains why PINNs might produce solutions that satisfy boundary conditions but fail to preserve physical topologies (e.g., distorted streamlines), as the PDE residuals themselves become unreliable in regions of high curvature or near boundaries.
Diagnostic Protocol: The study proposes a diagnostic protocol for PINN validation that separates value accuracy from derivative and residual accuracy. It argues that derivative-error profiles, endpoint/high-curvature diagnostics, and maximum absolute error metrics should be treated as "first-class diagnostics" rather than secondary checks.

The author explicitly frames this work as a foundational study isolating the mechanism of structure-preservation failures. They do not claim that derivative fidelity is impossible to achieve, but rather that finite, value-only training with standard architectures can leave derivative errors large enough to compromise simulation reliability. Future work suggested by the author includes extending these diagnostics to full PDE benchmarks (e.g., Poisson or heat equations) and exploring derivative-augmented losses.

More like this