🤖 machine learning

Learning functional components of PDEs from data using neural networks

This paper demonstrates a method for recovering unknown functional components of partial differential equations from data by embedding neural networks into the PDE framework, enabling accurate approximation of interaction kernels and potentials through standard parameter-fitting workflows while accounting for factors like data quantity, sampling density, and noise.

Original authors: Torkel E. Loman, Yurij Salmaniw, Antonio Leon Villares, Jose A. Carrillo, Ruth E. Baker

Published 2026-02-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Torkel E. Loman, Yurij Salmaniw, Antonio Leon Villares, Jose A. Carrillo, Ruth E. Baker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you only have the footprints left behind, not the person who made them.

In the world of science, Partial Differential Equations (PDEs) are like the complex laws of physics that describe how things move and change over time and space. Think of them as the "rulebook" for how a crowd of people moves, how heat spreads through a metal rod, or how bacteria swarm.

Usually, to use this rulebook to predict the future, you need to know every single detail of the rules. But often, nature is messy. We can see the result (the footprints, the crowd pattern, the temperature), but we don't know the hidden rules (the specific way people interact, the exact shape of the terrain, or the invisible forces at play). These hidden rules are often complex, wiggly lines (functions) rather than simple numbers.

This paper introduces a clever new way to reverse-engineer those hidden rules using Artificial Intelligence (Neural Networks).

The Core Idea: Teaching the Equation to Learn

Think of a standard PDE as a recipe for baking a cake.

The Ingredients: The variables (like flour, sugar, eggs).
The Process: The mixing and baking instructions (the math).
The Mystery: Usually, we know the recipe but don't know the exact amount of sugar or the specific type of flour (the parameters).

In this paper, the authors are dealing with a recipe where a whole ingredient is missing. Maybe the "sugar" isn't a fixed cup measurement, but a secret, wiggly curve that changes depending on where you are in the kitchen.

The Solution: They take a "black box" AI (a Neural Network) and stuff it directly inside the recipe.

The Setup: They give the AI a blank slate and tell it, "You are the missing ingredient. You need to figure out what shape you are."
The Training: They show the AI the final result (the footprints/cake).
The Guessing Game: The AI makes a guess about the shape of the missing ingredient. They run the recipe with that guess.
The Correction: If the result doesn't match the real footprints, the AI adjusts its shape slightly and tries again.
The Victory: Eventually, the AI "learns" the exact shape of the hidden rule that produced the footprints.

The Case Study: The "Swarm" Equation

To test this, the authors used a model called an Aggregation-Diffusion Equation. Imagine a swarm of bees or a school of fish.

Diffusion: They wander randomly (like a drunk person walking).
Aggregation: They are attracted to each other (like magnets).
The Mystery: We can see the final shape of the swarm (a tight ball, a long line, a ring). But we don't know exactly how they feel about each other. Do they like to be close? Do they get scared if they get too close? Is there a wind blowing them?

The authors asked: Can we look at the final shape of the swarm and figure out the invisible "social rules" (the interaction kernel) and the "wind" (the external potential) that created it?

What They Found

They ran thousands of experiments, acting like a scientist in a lab, and found some fascinating things:

1. It Works (Mostly):
If you give the AI a clear, perfect picture of the swarm, it can almost perfectly reconstruct the invisible social rules. It's like looking at a shadow and perfectly guessing the 3D object casting it.

2. The "Noise" Problem:
In the real world, data is messy. There's static, errors, and missing pieces (like trying to see footprints in the rain).

Good News: The AI can still figure out the rules even if the data is a bit noisy or sparse (like having only a few footprints).
Bad News: If the noise is too loud, the AI gets confused and starts guessing the wrong rules. It's like trying to hear a whisper in a rock concert; you might hear something, but you'll get the words wrong.

3. One vs. Many Footprints:
Sometimes, one single snapshot of the swarm isn't enough to solve the mystery.

The "Bifurcation" Analogy: Imagine a river splitting into two paths. If you only see the water at the split, you don't know which path it took. But if you see the water at the split and further downstream, you can figure out the terrain.
The paper shows that if you have multiple different snapshots of the swarm (e.g., a small group, a huge group, a group in a windstorm), the AI can solve the mystery much faster and more accurately. Different snapshots provide different clues.

4. The "Many Answers" Trap:
Sometimes, two completely different sets of rules can produce the exact same footprint.

Example: A crowd might form a circle because they are all attracted to the center, OR because they are all repelled by the edges. The result looks the same, but the rules are opposite.
The authors found that if you only have one snapshot, the AI might get stuck guessing the wrong rule. But if you give it multiple scenarios, it can usually tell the difference.

Why This Matters

This isn't just about math puzzles. This is a new tool for scientists everywhere.

Ecologists can look at animal populations and figure out the invisible landscape of food and danger without having to map the whole continent.
Engineers can look at a cracked bridge and figure out the hidden stress points without drilling holes in it.
Doctors might one day look at the spread of a disease and figure out the hidden transmission rules.

The Bottom Line

The authors have built a bridge between Machine Learning (which is great at finding patterns) and Physics (which is great at explaining why things happen).

Instead of just using AI to say "Here is a prediction," they used AI to say "Here is the law of nature that caused this prediction." They turned the AI from a fortune teller into a detective that can read the hidden rulebook of the universe, even when that rulebook is written in invisible ink.

1. Problem Statement

Partial Differential Equations (PDEs) are fundamental for modeling spatio-temporal phenomena across physics, biology, and engineering. However, many real-world systems contain unknown functional parameters (e.g., spatially varying interaction kernels, external potentials, or diffusivity maps) that are difficult or impossible to measure directly. While methods exist to recover scalar parameters from data (inverse problems), recovering full functional forms remains challenging.

The core problem addressed is: How can we recover unknown spatial functions (functional parameters) within a PDE directly from observed solution data (specifically steady states), using a framework that maintains mechanistic interpretability?

2. Methodology

The authors propose a workflow based on Universal Partial Differential Equations (UPDEs), extending the concept of Universal Differential Equations (UDEs) from ODEs to PDEs.

A. The UPDE Framework

Instead of treating unknown functions as fixed scalars, the authors replace them with Neural Networks (NNs) embedded directly into the PDE structure.

Case Study: They utilize a 1D nonlocal aggregation-diffusion equation:
$\partial_t u = \sigma \partial_{xx} u + \kappa \partial_x(u \partial_x[W * u]) + \partial_x(u \partial_x V)$
where $W(x)$ is the interaction kernel and $V(x)$ is the external potential.
Approximation: $W(x)$ and $V(x)$ are replaced by neural networks $NN_W(x; \theta_W)$ and $NN_V(x; \theta_V)$ .
Goal: Train the NN parameters ( $\theta$ ) and scalar parameters ( $\kappa$ ) to minimize the discrepancy between the model's steady state and observed data.

B. Loss Function and Optimization

A critical innovation is the choice of the loss function, which avoids differentiating noisy data:

Fixed-Point Residual: Instead of minimizing the PDE residual (which requires second derivatives and is sensitive to noise), the authors exploit the fact that steady states are fixed points of a nonlinear map $T$ .
$\text{Loss} = \| T(u) - u \|$
where $T(u) = \frac{\exp(-[\kappa W * u + V])}{Z(u)}$ .
Advantages: This approach is "equation-consistent" (respects boundary conditions naturally) and does not require differentiating noisy measurement data.
Optimization: They use a hybrid optimization strategy:
1. Adam: For initial broad exploration (10k–2M iterations).
2. LBFGS: For fine-tuning to the nearest local minimum.
3. Ensemble Multi-start: Multiple runs with random initial guesses to ensure global convergence and assess identifiability.

C. Data Generation

The study evaluates performance under various conditions:

Perfect Data: Exact, dense steady-state solutions.
Realistic Data: Sparse sampling (downsampled grids) and additive Gaussian noise.
Scenarios: Recovery of single functions ( $W$ ), multiple functions ( $W$ and $V$ ), and scalar parameters ( $\kappa$ ) simultaneously.

3. Key Contributions

Extension of UDEs to PDEs: Demonstrates that neural networks can be embedded into PDEs to learn unknown spatial functions (not just dynamics), effectively solving inverse problems for functional parameters.
Novel Loss Function: Introduces the use of the fixed-point residual ( $\|T(u) - u\|$ ) as a loss function for steady-state inference. This bypasses the need for high-order derivatives of noisy data, a common bottleneck in PDE inverse problems.
Systematic Analysis of Identifiability: Provides a comprehensive empirical and theoretical analysis of when functional recovery is possible. They distinguish between:
- Structural Non-identifiability: Fundamental mathematical limitations (e.g., recovering two functions from one solution).
- Practical Non-identifiability: Failures due to noise, sparse data, or poor solution selection.
Validation of "Black Box" vs. "White Box": Shows that while the NN learns the function, the resulting fitted PDE can be treated as a standard mechanistic model for future predictions.

4. Key Results

A. Recovery from Perfect Data

Single Function: The interaction kernel $W$ can be perfectly recovered from a single steady-state solution profile if the solution has a rich spectral content.
Multiple Functions: Recovering both $W$ and $V$ simultaneously requires at least two distinct solution profiles. A single profile is structurally non-identifiable for two unknown functions (infinite combinations of $W$ and $V$ can produce the same $u$ ).
Scalar Parameters: The interaction strength $\kappa$ can be recovered alongside the functions.

B. Impact of Data Quality (Noise and Sparsity)

Noise Tolerance: The method successfully recovers functions under low to moderate noise. However, recovery degrades significantly under high noise levels.
Solution Selection: Not all steady states are equally informative.
- Solutions with richer spectral content (more peaks/oscillations) provide better information for recovery.
- Using a single "uninformative" solution (e.g., a flat or single-peak profile) often leads to failure, even with perfect data.
- Combining multiple solutions improves convergence speed and accuracy.

C. Bifurcation and Branch Analysis

When recovering multiple parameters, the distance between solution profiles matters.
If solutions are taken from the same bifurcation branch, the $\kappa$ distance between them must be large enough to provide distinct information.
Solutions from different branches generally yield better recovery performance than closely spaced solutions on the same branch.

D. Failure Modes

The authors categorize failure modes:

Correct Function, Wrong Prediction: The fitted function minimizes the loss but generates different dynamics (due to bifurcation structure differences).
Wrong Function, Correct Prediction: The fitted function is incorrect, but the resulting solution profile matches the data (non-identifiability).
Structural Failure: Impossible to recover (e.g., 2 unknowns from 1 solution).

5. Significance and Implications

Bridging ML and Physics: The work successfully merges the flexibility of machine learning (universal approximation) with the interpretability of mechanistic modeling. The trained model is not just a predictor but a recovered physical law.
Experimental Design: The findings offer a guide for experimentalists: to recover complex spatial parameters, one must measure multiple, diverse steady states with high spectral richness, rather than relying on a single observation.
Broad Applicability: While tested on aggregation-diffusion equations, the framework is generalizable to other PDEs in fluid dynamics, heat transfer, and pattern formation where spatial coefficients are unknown.
Practical Utility: It enables the use of PDEs in scenarios where parameters are unmeasurable, allowing for data-driven calibration of complex systems for prediction and control.

In summary, the paper establishes a robust, mathematically grounded pipeline for learning unknown spatial functions in PDEs from data, highlighting the critical roles of solution diversity, noise levels, and the specific structure of the fixed-point map in achieving successful recovery.