🔬 applied physics

Inverted model selection in physics-informed neural networks: when a lower residual selects a worse solution

This paper demonstrates that the standard practice of selecting Physics-Informed Neural Network (PINN) solutions based solely on minimizing aggregate equation residuals is fundamentally flawed because it frequently favors structurally invalid solutions over exact ones, necessitating a new lexicographic admissibility gate that prioritizes structural identities and boundary conditions before residual comparison.

Original authors: Rabiu Musah

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rabiu Musah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern science, researchers often rely on artificial intelligence to solve complex equations that describe how the physical universe behaves. These equations, known as partial differential equations, govern everything from the flow of blood through veins to the movement of air over a wing. For decades, scientists have used traditional computer methods to solve them, but a newer approach called a Physics-Informed Neural Network has gained popularity. This method trains a computer program to learn the solution by checking how well it satisfies the governing equations, rather than by memorizing a pre-calculated answer. The program is given a set of rules and a target score: the lower the score, the better the solution. It is a system built on the assumption that if a program produces a lower score, it has found a more accurate representation of reality.

However, a recent study challenges this fundamental assumption. The research reveals that in many cases, the computer program can achieve a lower score by breaking the very rules it was supposed to follow. The author, Rabiu Musah, investigated this phenomenon by setting up a series of controlled tests where two versions of the same problem were solved side by side. In one version, a crucial physical rule was hard-wired into the program's structure, meaning the solution could not violate it. In the other version, the same rule was added as a penalty, a soft suggestion that the program could ignore if it helped lower the overall score. The results were startling: the version that ignored the rule frequently produced a lower score than the version that obeyed it. When researchers ranked the solutions based on this score, they consistently chose the one that was physically impossible, rejecting the correct solution in favor of a flawed one.

The study focused on a specific type of fluid flow problem involving swirling motion, known as vorticity, within a circular area. The researchers created a reference solution using a trusted, traditional method to serve as the ground truth. They then trained dozens of neural networks to solve the same problem. Some networks were designed to strictly enforce the relationship between the swirling motion and the fluid's pressure, while others were allowed to violate this relationship if it helped them minimize their error score. The findings showed that the networks allowed to violate the rule often achieved a lower error score, sometimes by a significant margin, even though their solutions were wildly incorrect. In fact, across a large set of experiments involving different types of networks and random starting conditions, this misleading ranking happened in 83% of the cases. The program that looked better on paper was actually violating the core physics of the problem by a massive amount, while the program that followed the rules was ranked as inferior.

This issue is not limited to a single type of network or a specific problem. The researcher tested six different families of neural network architectures, including some designed to learn complex operators rather than simple fields. In nearly every case, the pattern held true: the version that treated the physical law as a flexible penalty rather than a rigid rule tended to produce a lower error score while failing to satisfy the law. The study also identified a specific type of failure where the network learned to mimic the shape of the solution based on a hidden bias in its design, rather than learning from the physics itself. Even when the source of the physical force was removed from the equation, the network continued to produce a structured solution that looked correct, passing all standard checks. This suggests that the network was simply copying a template rather than understanding the underlying science.

To address this, the author proposes a new way of evaluating these solutions. Instead of relying on a single number to declare a winner, the evaluation process must first check if the solution satisfies several independent conditions. These checks include verifying that the solution respects the boundary of the area, that it obeys the fundamental structural relationship between the variables, and that it satisfies a specific integral condition that must be zero for any valid physical solution. Only after a solution passes all these gates is it allowed to compete based on its error score. This approach ensures that a solution is physically valid before it is judged on how well it fits the equation. The study demonstrates that without these checks, the standard method of ranking solutions by their error score is likely to select the wrong answer, leading scientists to believe they have found a solution when they have actually found a sophisticated illusion.

The research does not claim that these neural networks are useless, but rather that the current method of judging them is flawed. The phenomenon occurs because the mathematical space of possible solutions is vast, and allowing a program to break a rule gives it more freedom to lower its score, even if that freedom leads it away from the truth. The study shows that this is not a rare glitch but a common occurrence that depends on the specific problem being solved. While the researcher could not predict exactly which problems would trigger this behavior, the evidence is clear that relying solely on a low error score is dangerous. The solution is to treat the physical laws not as suggestions to be minimized, but as non-negotiable constraints that must be met before any other measure of success is considered. By adding these rigorous checks, scientists can ensure that the artificial intelligence they use is not just producing a low number, but is actually describing the physical world correctly.

Technical Summary: Inverted Model Selection in Physics-Informed Neural Networks

Problem Statement
The paper addresses a critical assumption in the evaluation of Physics-Informed Neural Networks (PINNs): that a smaller aggregate residual (training loss) reliably indicates a better solution. In standard computational physics, solvers are verified by ensuring they satisfy system constraints (e.g., incompressibility, conservation laws) to a stated tolerance before accuracy is assessed. However, PINN literature often relies on a single scalar residual norm. The author argues that for constrained Partial Differential Equation (PDE) systems, minimizing a composite objective function (equation residual + penalty for constraint violation) can lead to "inverted model selection." In this scenario, a solver that violates a defining structural identity by a large margin may achieve a lower equation residual than a solver that satisfies the identity exactly, leading to the erroneous selection of the inferior solution.

Methodology
The study employs a rigorous benchmarking framework to test this hypothesis across three constrained PDE systems:

S1: A steady forced vorticity transport problem on a unit disk.
S2: A mixed-form Poisson problem on a square domain.
S3: A steady viscous Burgers equation in mixed form.

For each system, the author compares "matched pairs" of solvers. These pairs differ only in how a defining structural identity is enforced:

Structural Variant: The identity is "hard-wired" (imposed by construction), ensuring the constraint is satisfied exactly ( $R_{str} \equiv 0$ ).
Penalised Variant: The identity is imposed as a soft penalty term in the loss function with a weight $\lambda_{str}$ .

The evaluation utilizes a Lexicographic Admissibility Gate rather than simple residual ranking. A candidate solution is deemed admissible only if it independently satisfies three checks on held-out data:

Structural Identity ( $R_{str}$ ): The magnitude of the constraint violation.
Boundary Trace ( $R_{bc}$ ): The error on the boundary.
Solvability Integral ( $R_{sol}$ ): An integral condition (e.g., vanishing of the advection term integral) that must hold for any admissible field, independent of the equation residual.

Only admissible candidates are ranked by their equation residual ( $R_{PDE}$ ). An "inversion" is recorded when the penalised variant achieves a lower $R_{PDE}$ than the structural variant but fails the admissibility gate due to constraint violation.

The experiments span 64 matched pairs across two systems (S2, S3) using four network variants (MLP with varying widths/activations) and eight random seeds. A broader study includes six architecture families (MLP, cPINN, XPINN, hp-VPINN, DeepONet, FNO) on S3. Additionally, the paper investigates "artefact classes," including cases where a prescribed-structure prior yields fields that pass all checks despite the physics being removed.

Key Results

High Frequency of Inversion: Across 64 matched pairs in the system study, the inversion occurred in 83% of cases (95% Wilson CI: 72–90%). The rate varied by system, from 72% (S2) to 94% (S3).
Architecture Independence: In a study of 48 pairs across six architecture families on S3, 46 pairs (96%) exhibited inversion. This indicates the phenomenon is problem-dependent rather than specific to the neural network approximator.
Magnitude of Violation: In the penalised variants that achieved lower residuals, the structural identity was frequently violated by several orders of magnitude (e.g., $R_{str} \approx 3.3$ when the tolerance was $10^{-6}$ ), while the equation residual was only marginally lower (e.g., a 9.37% reduction in S1).
Architectural Bias Detection: A specific artefact class (A4) demonstrated that a bounded template prior could generate fields that passed all admissibility checks even when the source term (physics) was removed. This suggests that standard single-run checks cannot distinguish between structure learned from physics and structure imposed by priors.

Key Contributions

Empirical Demonstration of Inversion: The paper provides systematic evidence that lower residuals do not guarantee better solutions in constrained PINN problems, challenging the standard practice of ranking models solely by loss.
Admissibility Gate: The author proposes a lexicographic evaluation protocol that prioritizes constraint satisfaction (structural identity, boundary trace, solvability) over equation residual. This gate effectively filters out solutions that trade constraint violation for residual minimization.
Diagnostic for Priors: The paper introduces a "physics-off" ablation test to detect architectural bias, showing that standard metrics fail to identify solutions where the network relies on a prior rather than the governing equations.
Theoretical Explanation: The phenomenon is explained via constrained optimization theory. The penalised formulation is a relaxation of the constrained problem; thus, its attainable minimum residual is theoretically bounded above by the structural formulation. For small penalty weights, the penalised solver exploits the enlarged feasible set to lower the equation residual at the cost of constraint violation.

Significance and Claims
The paper claims that the reliance on a single aggregate residual is hazardous because it cannot distinguish between a "good" solution and a "relaxed" solution that violates fundamental physical identities. The author asserts that this inversion is common (occurring in roughly 6 of 7 matched pairs) but not universal, suggesting it depends on specific problem geometries not yet fully understood.

The significance lies in shifting the verification paradigm for Scientific Machine Learning. The author argues that before discussing accuracy, PINN solvers must be shown to satisfy constraints. The proposed gate catches three classes of artefacts (soft-constraint violations, solvability violations, and architectural bias in specific contexts) but admits a limitation: it cannot detect bias introduced by structural priors without the specific "physics-off" ablation. The paper concludes that while hard-constraining identities removes this specific failure mode, it does not guarantee overall optimality, and when penalties are unavoidable, constraint residuals must be reported separately from equation residuals.

More like this