💻 computer science

Neural operator discovery from heterogeneous trajectories

This paper introduces Neural Operator Discovery (NOD), a framework that learns shared solution operators and system-specific variations directly from heterogeneous trajectories without labeled governing factors, enabling interpretable, zero-shot generalization through a factorized latent-conditioning formulation.

Original authors: Sili Deng, Zituo Chen, Qiaofeng Li, Jiaxin Hu

Published 2026-07-30

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Sili Deng, Zituo Chen, Qiaofeng Li, Jiaxin Hu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you only have the footprints left behind, not the shoes that made them. In the world of physics and engineering, scientists often have mountains of data—snapshots of how things move, flow, or change over time, like the swirling wake behind a boat or the way heat spreads through a metal rod. These are called "trajectories." Usually, to predict what happens next, scientists need to know the exact "rules of the game" for each specific situation: the viscosity of the fluid, the shape of the object, or the temperature. It's like needing to know the exact weight of a shoe to predict how deep a footprint will be. But in the real world, we often don't have those labels. We just have the messy footprints. This paper tackles the challenge of learning the rules of the game and the specific details of the shoe, all at once, just by looking at the footprints.

The authors, a team from MIT and Zhejiang University, have developed a new method called "Neural Operator Discovery" (NOD). Think of it as a super-smart AI detective that doesn't just memorize footprints but learns to understand the style of the walker. They found that by forcing their AI to look at two different walks made by the same person (but starting from different spots), it could figure out the person's unique "style" (the hidden rules) without ever being told who the person was. This allows the AI to predict future movements with high accuracy, even for people it has never seen before, and even in extreme conditions where the rules change drastically.

The Detective's New Trick

Usually, when scientists try to build a computer model to predict how a system (like a fluid or a chemical reaction) will behave, they hit a wall if they don't have the "instruction manual." If they want to predict how water flows around a cylinder, they need to know the water's thickness (viscosity). If they don't have that number, the model is blind.

Previous attempts to fix this were like trying to guess the shoe size by looking at a single footprint. Some methods tried to memorize the entire history of the movement (like looking at the last 10 steps to guess the next one), but this often led to the model getting confused and making wild errors over time. Others tried to compress the data into a hidden "secret code," but that code was often messy and didn't actually tell us anything useful about the physics.

The team behind this paper, led by Sili Deng and colleagues, decided to try a different approach. They treated the problem like a game of "spot the difference" between two versions of the same story.

The "Two-Story" Training Game

Here is how their method, NODnet, works, using a simple analogy:

Imagine you have a library of thousands of stories written by different authors. You don't know who wrote which story, and you don't have their names. You just have the text. Your goal is to build a machine that can finish any story, no matter who wrote it, and even write new stories for authors you've never met.

The authors' trick was to take two different chapters from the same author's book.

Chapter A is used to figure out the author's unique "voice" (their style, vocabulary, and tone).
Chapter B is used to test if the machine can predict what happens next, using that "voice" as a guide.

Crucially, the machine is told: "These two chapters are by the same author, but they start at different points in the plot." This forces the machine to ignore the specific plot details (like "the hero was at the castle") and focus only on the author's unique style (the "governing factors").

In the real world, this means the AI looks at a short burst of data from a fluid flow (the "voice") and uses that to predict the future of a different burst of data from the same fluid setup. By doing this over and over, the AI learns to strip away the noise of the specific starting conditions and isolate the pure "rules" that govern the system.

The Magic of the "Hidden Dimension"

One of the most clever parts of this discovery is how the AI figures out how many hidden rules it needs to learn.

Imagine you are trying to describe a person's walking style. You could use a thousand different words, but maybe you only need two: "stride length" and "pace." If you use too many words, you start describing the color of their socks (irrelevant details). If you use too few, you can't tell the difference between a sprinter and a stroller.

The researchers built a system that automatically tests different numbers of "hidden words" (called latent dimensions). They found that for the Burgers' equation (a model for fluid shocks), the AI naturally settled on 1 hidden dimension. For the FitzHugh–Nagumo system (a model for nerve impulses), it settled on 2. This matched perfectly with the actual number of physical variables changing in those systems (viscosity for the first, two parameters for the second).

This suggests the AI isn't just guessing; it's actually discovering the true, minimal number of rules needed to describe the world. It's like the AI saying, "I don't need to know the color of the socks; I just need to know the stride length."

What They Found (and What They Didn't)

The team tested this idea on three very different physical systems:

Burgers' Equation: A model for fluid flow that can create sharp, sudden shocks (like a sonic boom). They tested it with different viscosities, including a "zero viscosity" case where the fluid is perfectly slippery.
Cylinder Flow Wake: How air or water swirls behind a cylinder. This was a tough test because the data was "sparse" (only 16 examples) and the shape was irregular.
FitzHugh–Nagumo System: A model for how electrical signals travel in nerves, which can behave in complex, oscillating ways.

In all three cases, the NODnet model outperformed the old methods.

Long-term Prediction: While other models would drift off course after a few steps (like a GPS that slowly loses signal), the NODnet model stayed accurate for much longer. For example, in the Burgers' test, the old "autoregressive" models (which just look at the past) had huge errors after 50 steps, while NODnet kept its cool.
Zero-Shot Magic: The model could even predict what would happen for a system it had never seen before. In the nerve impulse test, they trained the AI on a specific set of rules, then asked it to predict a completely new set of rules it had never encountered. The AI didn't just guess; it used the "style" it learned to extrapolate correctly.
The "Inviscid" Limit: In the fluid test, they asked the AI to predict what happens when the fluid has zero viscosity (a state it wasn't explicitly trained on). The AI's internal "voice" changed smoothly and continuously to handle this extreme case, showing it truly understood the physics, not just the math.

The Limits of the Detective

The authors are careful to note that this is a simulation-based discovery. They haven't tested this on real-world, noisy sensor data from a factory or a river yet. They also point out that their method works best when the "rules" change smoothly. If the system suddenly jumps from one type of behavior to a completely different one (like water turning into ice), the current method might struggle.

Furthermore, they found that the way the AI is "taught" matters. If the AI only looks at one step at a time (like a single frame of a movie), it fails to learn the deep rules. It needs to see the whole picture or multiple steps ahead to figure out the "voice" of the system. This suggests that to discover the laws of nature from data, we need to look at the big picture, not just the immediate next step.

Why This Matters

This paper suggests a new way to do science. Instead of needing a human to label every single experiment with its parameters, we might be able to let AI sift through messy, unlabeled data and find the hidden laws for us. It's a step toward "self-supervised" discovery, where machines learn the rules of the universe just by watching it play.

While it's not a magic wand that solves every physics problem today, it offers a promising path forward. It shows that even without a manual, if we look at enough different "footprints" and compare them the right way, we can learn to predict the future of complex systems, from the flow of blood in our veins to the weather patterns in our atmosphere. The AI isn't just memorizing; it's learning to think like a physicist.

Technical Summary: Neural Operator Discovery from Heterogeneous Trajectories

Problem Formulation

In many physical sciences, trajectory data are collected from families of dynamical systems that share a common governing law but differ in unobserved system-specific factors (e.g., physical parameters, geometries, or boundary conditions). Existing data-driven modeling approaches face significant limitations in this setting:

Symbolic Regression: While capable of recovering equations without labels, it is constrained by primitive libraries and computationally expensive for high-dimensional spatiotemporal dynamics.
Parametric Neural Operators (PNO): These require explicit supervision of governing factors as inputs, which is unavailable in the target setting.
Autoregressive Neural Operators (ANO): These condition predictions on past-state windows without explicit latent representations. While they avoid parameter supervision, they accumulate rollout errors and fail to provide an identifiable, low-dimensional representation of system variation.
Latent-Conditioned Models: Existing approaches often trade off expressivity, scalability, or interpretability, or rely on expensive meta-learning procedures.

The central challenge addressed by this work is Neural Operator Discovery (NOD): learning both a shared solution operator and system-specific variation directly from heterogeneous trajectories without access to labeled governing factors.

Methodology: NODnet

The authors propose a factorized realization, NODnet, which decomposes the problem into two components:

Trajectory Encoder ( $f_\phi$ ): Maps a "conditioning" past-state window from a system instance to a low-dimensional latent representation $c$ .
State-Evolution Operator ( $g_\theta$ ): A conditional neural operator that predicts future states based on an initial state and the latent representation $c$ .

To ensure the latent representation captures only system-specific variation (governing factors) rather than transient trajectory features, the method employs three key constraints:

1. Factorized Prediction

Instead of predicting future states directly from a history window (as in ANO), the model factorizes the process:
$c = f_\phi(x'_{0:s}), \quad \hat{x}_{t+1} = g_\theta(x_t; c)$
The encoder infers $c$ from a conditioning trajectory $x'_{0:s}$ , while the decoder predicts the evolution of a separate trajectory $x_{t:t+1}$ from the same system instance.

2. Trajectory-Decoupled Sampling

To prevent the encoder from encoding initial-condition-specific information, training pairs the encoder and decoder using trajectories generated from different initial conditions of the same system instance. This forces the encoder to retain only cross-trajectory invariants (the governing factors) and suppress trajectory-specific features.

3. Dimension Selection

The dimensionality of the latent space is not fixed arbitrarily but selected as the smallest value sufficient to capture variation across system instances. This is achieved via a two-stage procedure:

Stage 1: Identify candidate dimensions where increasing dimensionality no longer yields significant loss improvements beyond random seed variability.
Stage 2: Select the final dimension $d^*$ as the smallest candidate where the corresponding $d+1$ model does not produce additional factor-aligned structure (assessed via Principal Component Analysis).

Key Contributions

Formulation of NOD: Defining the problem of learning shared operators and system-specific variations without parameter supervision.
Factorized Architecture: Introducing a framework that jointly learns a neural operator and a low-dimensional latent representation through factorized prediction and trajectory-decoupled sampling.
Automatic Dimension Selection: A data-driven procedure to determine the intrinsic dimensionality of system variation, matching the number of independent governing factors.
Generalization Modes: The framework supports two inference modes:
- One-shot Inference: Inferring the latent representation from a single observed trajectory to predict unseen initial conditions.
- Zero-shot Inference: Sampling directly from the learned latent space to generate predictions for entirely unseen system instances (factors not seen during training).

Experimental Results

The method was evaluated on three families of partial differential equations (PDEs):

1. Burgers' Equation (Varying Viscosity)

Setup: 1D system with varying kinematic viscosity $\nu$ , including the inviscid limit ( $\nu=0$ ).
Findings: The dimension selection procedure correctly identified $d^*=1$ . The learned latent space exhibited a smooth, monotonic correspondence with viscosity, extending continuously to the inviscid regime.
Performance: The latent-conditioned model (NODnet) significantly outperformed autoregressive baselines (ANO) in long-horizon (50-step) rollouts, maintaining bounded error even in the inviscid limit. Autoregressive variants failed to distinguish viscosity regimes, collapsing to a constant latent representation.

2. Cylinder Flow Wake (Unstructured Mesh, Sparse Data)

Setup: 2D incompressible Navier-Stokes flow with varying viscosity and inlet velocity on an unstructured mesh. Training data was sparse (16 trajectories).
Findings: Correctly identified $d^*=1$ (viscosity). The model successfully generalized to unseen inlet velocities and viscosities outside the training range.
Physical Consistency: The model recovered the correct drag coefficient ( $C_D \approx 1.36$ ) and Strouhal number, whereas the autoregressive baseline exhibited frequency drift and non-physical drag behavior.
Robustness: Despite sparse supervision, the model remained stable, utilizing trajectory-decoupled sampling and contrastive regularization to organize the latent space.

3. FitzHugh–Nagumo Reaction–Diffusion (Multi-Factor)

Setup: 2D system with two independent governing factors: stimulus parameter $k$ and recovery rate $\beta$ .
Findings: Dimension selection identified $d^*=2$ . The latent space organized system instances according to the $(k, \beta)$ parameters with a smooth, approximately invertible mapping (quadratic fit $R^2=0.9937$ ).
Generalization: The model achieved zero-shot generalization to parameter combinations outside the training hull (OOD-Extra).
Supervision Geometry: A hierarchical decoder (supervising multiple time horizons) significantly improved long-horizon prediction accuracy compared to single-step decoders, highlighting the importance of supervision signals sensitive to governing factors over extended time.

Significance and Claims

The paper claims to establish an interpretable paradigm for operator learning in the absence of explicit factor supervision. Key claims include:

Identifiability: The learned latent spaces consistently align with underlying governing factors, organizing system instances in a smooth, approximately invertible structure.
Superior Long-Horizon Prediction: Latent conditioning combined with trajectory-decoupled sampling outperforms both autoregressive and no-conditioning baselines in long-horizon rollouts, reducing error accumulation.
Zero-Shot Extrapolation: The method enables generalization to previously unseen system instances (factors) by sampling directly from the learned latent space, even for regimes like the inviscid limit or parameter combinations outside the training convex hull.
Scalability: The approach scales to high-dimensional spatiotemporal systems and unstructured meshes without requiring explicit parameter labels.

The authors note limitations, including the current focus on continuously varying factors in low-dimensional Euclidean spaces, the use of simulation-generated data, and the need for further investigation into discrete system classes, bifurcating dynamics, and robustness under experimental noise. They state that a formal theoretical characterization of the learned latent properties remains an open question.