Learning Hidden Physics and System Parameters with Deep Operator Networks

This paper introduces two DeepONet-based frameworks, Deep Hidden Physics Operator (DHPO) and a parameter identification method, to efficiently discover hidden physical laws and infer system parameters from sparse, noisy observations across diverse partial differential equation families, overcoming the generalization and retraining limitations of existing data-driven approaches.

The Gist

Imagine you are a detective trying to solve a mystery, but you only have a few blurry photos of the crime scene and no idea what the criminal's weapon looked like. In the world of science and engineering, this is exactly the challenge researchers face when trying to understand complex systems like weather patterns, blood flow, or how materials break. They have some data (the blurry photos), but they don't know the exact rules (the physics) or the specific settings (the parameters) that govern the system.

This paper introduces two new "super-detectives" built using Artificial Intelligence to solve these mysteries. These detectives are based on a technology called Deep Operator Networks (DeepONet).

Here is a breakdown of how they work, using simple analogies:

The Problem: The "Black Box" Mystery

Usually, to understand a physical system, scientists need two things:

1. The Rules: The mathematical equations that describe how the system works (e.g., how heat spreads).
2. The Settings: Specific numbers inside those equations (e.g., how conductive the metal is).

The problem is that in the real world, we often have sparse data (measurements from just a few sensors) and noisy data (measurements with errors). Traditional AI methods are like students who memorize one specific test. If the test questions change slightly (a different initial condition or a new parameter), the student fails and has to study all over again.

The Solution: Two New Detective Tools

The authors created two complementary tools that act like a master detective who can learn the logic of a crime, not just memorize the details of one specific case.

Tool 1: The "Hidden Physics" Detective (DHPO)

The Analogy: Imagine you are watching a magic trick. You see the magician's hands move (the data), but you don't know the secret mechanism inside the box (the hidden physics).
How it works:

• Most AI tries to guess the final result. This tool, called DHPO (Deep Hidden Physics Operator), tries to guess the secret mechanism itself.
• It looks at the data and asks, "What invisible rule must be happening here to make the system behave this way?"
• Instead of just memorizing one specific scenario, it learns the general logic. If it learns how a fluid moves in a pipe today, it can instantly apply that logic to a different pipe tomorrow without needing to re-study.
• The Result: It can discover unknown parts of the laws of physics directly from messy, incomplete data.

Tool 2: The "Parameter Finder" Detective (Parameter Identification)

The Analogy: Imagine you have a car engine running, but you don't know the exact size of the pistons or the viscosity of the oil. You can only listen to the engine from a few windows (sparse sensors).
How it works:

• This tool uses a "pre-trained" AI that already knows how to reconstruct the whole picture from a few clues.
• It then works backward. It says, "If the engine sounds this way, the oil must be this thick."
• Unlike older methods that need a library of "perfect" examples to learn from, this tool uses the laws of physics as a guide. It knows the engine must follow certain rules, so it narrows down the possibilities until it finds the correct settings.
• The Result: It can pinpoint the exact settings (like viscosity or heat conductivity) even if the data is noisy or very limited.

Why This is a Big Deal (The "Aha!" Moment)

1. The "One-and-Done" Learning
Old methods are like a student who studies for one specific exam. If the exam changes, they fail.
These new tools are like a student who learns the subject. Once they learn the subject of "Fluid Dynamics," they can solve any fluid problem instantly without re-studying. This saves massive amounts of time and computing power.

2. Working with "Blurry Photos"
Real-world sensors are often broken or far apart.

• Old AI: "I can't solve this because I don't have enough data points."
• New AI: "I don't need every pixel. I know the laws of physics, so I can fill in the gaps and guess the rest."
  The paper shows these tools work even when data is noisy (like a photo with static) or sparse (like a photo with only a few dots).

3. Knowing What They Don't Know (Uncertainty)
Sometimes, the data is so sparse that there are multiple possible answers.

• Old AI: Gives you one number and hopes it's right.
• New AI: Says, "The answer is likely between X and Y, and here is the probability." It gives you a range of possibilities, which is crucial for safety in engineering (e.g., "The bridge is probably safe, but there's a 5% chance it's not").

Real-World Examples Tested

The researchers tested their detectives on four famous "mysteries":

• Reaction-Diffusion: How chemicals mix and react (like patterns on a leopard's skin).
• Burgers' Equation: How shockwaves move in fluids (like traffic jams or sonic booms).
• Heat Equation: How heat spreads through a weirdly shaped metal plate.
• Helmholtz Equation: How sound waves bounce around to find a hidden source (like finding a hidden speaker in a room).

In all cases, the new tools found the hidden rules and the correct settings with high accuracy, even when the data was imperfect.

The Bottom Line

This paper presents a shift from "memorizing data" to "learning the language of physics." By combining deep learning with the fundamental laws of nature, these new frameworks allow scientists to solve complex engineering problems faster, with less data, and with a better understanding of the uncertainty involved. It's like giving a detective a map of the city and the rules of traffic, rather than just a list of addresses.

Technical Summary

1. Problem Statement

The paper addresses two central challenges in computational science and engineering:

1. Hidden Physics Discovery: Identifying unknown governing physical laws (terms in Partial Differential Equations, PDEs) from sparse and noisy observational data.
2. System Parameter Identification: Inferring critical system parameters (e.g., viscosity, diffusivity, thermal conductivity) directly from sparse sensor measurements without requiring paired ground-truth parameter datasets.

Limitations of Existing Methods:

• SINDy (Sparse Identification of Non-linear Dynamics): Requires prior knowledge of candidate terms and is sensitive to noise; typically limited to single dynamical systems.
• Deep Hidden Physics Models (DHPM): Effective for single PDE instances but lacks generalization across different PDE families and cannot identify unknown system parameters from labeled datasets.
• Physics-Informed Neural Networks (PINNs): Require retraining for every new parameter variation, limiting computational efficiency and generalizability.
• Neural Inverse Operators (e.g., NIO, iFNO): Typically require fully paired datasets (measurements + ground truth parameters) generated via numerical simulations, which are often unavailable in real-world scenarios.

2. Methodology

The authors propose two complementary frameworks based on Deep Operator Networks (DeepONet), which learn mappings between infinite-dimensional function spaces, enabling operator-level generalization.

A. Deep Hidden Physics Operator (DHPO)

• Goal: Discover unknown PDE terms (the "hidden physics") directly from sparse data.
• Architecture:
  • DeepONet: Consists of a Branch Net (processing input source terms $f(x)$ at fixed sensor locations) and a Trunk Net (processing spatio-temporal coordinates $(x, t)$). The dot product of these outputs approximates the solution field $u(x, t)$.
  • Hidden Physics MLP: A separate neural network takes the solution $u$ and its spatial derivatives ($u_x, u_{xx}$, etc.) as inputs to approximate the unknown physics operator $N$.
• Training Strategy:
  • Minimizes a composite loss function comprising: Initial Conditions (IC), Boundary Conditions (BC), PDE Residuals, and Data Fidelity.
  • Key Innovation: Unlike DHPM, DHPO generalizes across multiple PDE families and system conditions simultaneously without retraining, leveraging physics-informed constraints to learn the mapping of unknown operators.

B. Physics-Informed Parameter Identification

• Goal: Infer unknown system parameters (e.g., $\nu, D, K$) from sparse sensor data when the PDE structure is known but parameters are not.
• Architecture: A two-stage framework:
  1. Forward Surrogate: A DeepONet is pre-trained to reconstruct the full solution field $u(x, t)$ from sparse sensor measurements.
  2. Inverse Surrogate: An MLP maps the sparse sensor data directly to the unknown parameter $\xi$.
• Training Strategy:
  • Deterministic: The networks are trained jointly using the PDE residual as the primary loss term, ensuring physical consistency.
  • Probabilistic: The MLP outputs the mean ($\mu_\xi$) and standard deviation ($\sigma_\xi$) of a Gaussian distribution. A reparameterization trick allows backpropagation through stochastic sampling, and a KL-divergence term regularizes the distribution against a prior. This captures epistemic uncertainty and parameter non-uniqueness inherent in inverse problems.

3. Key Contributions

1. Unified Framework: Combines operator learning with physics-informed modeling to solve both hidden physics discovery and parameter identification in a single, data-efficient paradigm.
2. Operator-Level Generalization: The models generalize across families of PDEs and varying initial/boundary conditions without the need for instance-specific retraining.
3. Data Efficiency: Achieves high accuracy using only sparse sensor measurements (no dense field observations or paired parameter labels required).
4. Uncertainty Quantification: Introduces a probabilistic extension that quantifies parameter ambiguity, providing a distribution of plausible parameters rather than a single point estimate.
5. Robustness: Demonstrates resilience to noise and data sparsity across diverse benchmark problems.

4. Results and Performance

The frameworks were tested on four benchmark problems: Reaction-Diffusion, Burgers' Equation, 2D Heat Equation, and 2D Helmholtz Equation.

• Accuracy:
  • Hidden Physics Discovery: Achieved relative solution errors on the order of $O(10^{-2})$.
  • Parameter Identification: Achieved absolute parameter estimation errors on the order of $O(10^{-3})$ (e.g., viscosity, diffusivity).
• Robustness:
  • Noise: Maintained accuracy with up to 5% Gaussian noise in sensor data.
  • Sparsity: Performed well even with extremely limited temporal data points (e.g., only $t=0$ and $t=1$).
• Probabilistic Validation:
  • Forward simulations using parameters sampled from the predicted distributions showed high agreement ($R^2 > 0.95$) with observed sensor data, confirming the framework correctly captures the manifold of valid parameters.
• Computational Efficiency:
  • Speedup: The operator learning framework offers a 80x to 540x speedup compared to traditional iterative optimization methods (which require repeated forward solver evaluations).
  • Inference: Once trained, parameter estimation takes milliseconds per query, enabling real-time applications.

5. Significance and Impact

• Paradigm Shift: Moves beyond "instance-level" learning (retraining for every new case) to "operator-level" learning, where a single model handles a family of problems.
• Practical Applicability: The ability to infer parameters from sparse, noisy sensor data without ground-truth labels makes this highly relevant for real-world engineering (e.g., material failure analysis, fluid dynamics, medical imaging).
• Handling Ill-Posedness: The probabilistic approach explicitly addresses the non-uniqueness of inverse problems, providing a rigorous way to quantify uncertainty rather than forcing a potentially misleading single solution.
• Future Directions: The authors note limitations regarding fixed sensor locations (inherited from standard DeepONet) and suggest future work integrating Set-based operators (SetONet) to handle variable sensor configurations.

In conclusion, this work provides a robust, unified, and computationally efficient framework for solving complex inverse problems in scientific machine learning, bridging the gap between data-driven modeling and physical laws.