🤖 machine learning

Neural operator discovery from heterogeneous trajectories

This paper introduces Neural Operator Discovery (NOD), a framework that learns shared solution operators and system-specific variations directly from heterogeneous trajectories without labeled governing factors, thereby enabling interpretable, generalizable, and stable long-horizon predictions across diverse dynamical systems.

Original authors: Zituo Chen, Qiaofeng Li, Jiaxin Hu, Sili Deng

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zituo Chen, Qiaofeng Li, Jiaxin Hu, Sili Deng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a thousand different videos of water flowing. Some videos show water rushing through a narrow pipe, others through a wide river, and some show water hitting a rock. In every single video, the water follows the same basic rules of physics—it swirls, speeds up, and slows down. But in the real world, scientists often don't have a label on each video telling them exactly how wide the pipe is or how big the rock is. They just have the messy, moving pictures. For a long time, computers trying to predict what happens next in these videos had to be told the "secret recipe" (like the exact pipe width) before they could guess the future. If that secret was missing, the computers would get confused, or they would try to memorize the whole video frame-by-frame, which is slow and prone to making mistakes over time. This paper tackles the problem of teaching a computer to understand the rules of the flow and figure out the secret recipe just by watching the water move, without anyone ever telling it what the recipe is.

The researchers, led by Zituo Chen and Sili Deng, have developed a new method called Neural Operator Discovery (NOD). Think of it as teaching a computer to be a detective. Instead of just memorizing the water's path, the computer learns to separate the "universal laws" of water from the "specific details" of each scene. They built a system with two parts: a "decoder" that knows how to predict the future of the water based on the rules, and an "encoder" that acts like a detective's notebook. The encoder watches a short clip of the water and writes down a tiny, secret note (a "latent representation") that captures only the specific details of that scene, like the pipe size.

The magic trick here is how they train the detective. They don't just show the computer one video and ask it to guess the future. Instead, they use a technique called trajectory-decoupled sampling. Imagine showing the computer a video of water flowing in a pipe, but then asking it to predict the future of a different video of water flowing in that same pipe, starting from a different spot. Because the pipe is the same but the starting spot is different, the computer realizes that the "pipe size" note in its notebook must be the same for both, while the "starting spot" changes. This forces the computer to ignore the temporary, messy details of the start and focus only on the permanent, hidden factors that define the system.

The paper tested this idea on three very different types of physics problems:

Burgers' Equation: A model for how shockwaves form in fluids. The team found that the computer could successfully learn that there was only one hidden factor (viscosity) changing across all the videos. Even when they asked the computer to predict what would happen if the fluid had zero viscosity (a state it had never seen before), the computer's "notebook" smoothly extended to handle it, predicting the shockwaves accurately.
Cylinder Flow Wake: This simulates wind blowing past a cylinder, like a flagpole. The data here was messy and came from an irregular shape. The computer learned to spot that, despite the complexity, the flow was really driven by just one main factor (viscosity). It could predict the swirling "vortex shedding" behind the cylinder for 50 steps into the future without drifting off course, something other methods failed to do.
FitzHugh–Nagumo System: This models how electrical signals travel in nerves. Here, the system was driven by two hidden factors. The computer successfully figured out that it needed a two-dimensional "notebook" to capture the variation. It could then predict the nerve signals for new, unseen combinations of these factors, even if those combinations were outside the range of what it was trained on.

The results suggest that this approach is a powerful way to learn from messy, unlabeled data. The computer didn't just memorize; it organized the different systems into a smooth, logical map where similar systems sit close together. This allowed the computer to make "zero-shot" predictions—guessing the behavior of a completely new system just by sampling a point on this map, without ever seeing a video of that specific system before. While the paper shows these results through computer simulations, it indicates that this method could be a game-changer for understanding complex physical systems where the underlying parameters are unknown or unmeasurable, offering a way to learn the "rules of the game" directly from the players' moves.

Technical Summary: Neural Operator Discovery from Heterogeneous Trajectories

Problem Statement

In physical sciences, trajectory data is frequently collected from systems that share a common governing law but differ in unobserved system-specific factors (e.g., physical parameters, geometries, boundary conditions). In many experimental and legacy simulation datasets, these distinguishing factors are unrecorded or unmeasurable. Consequently, data consists of heterogeneous trajectories generated by multiple underlying configurations without explicit labels for the variables driving their variation.

The central challenge is to learn predictive models that capture both the shared dynamics of a system family and the differences between specific system instances directly from these unlabeled trajectories. Existing approaches face limitations:

Symbolic Regression: Constrained by primitive libraries and computationally expensive for high-dimensional spatiotemporal dynamics.
Parametric Neural Operators (PNO): Require explicit factor inputs, which are unavailable in this setting.
Autoregressive Neural Operators (ANO): Condition on past-state windows, leading to high-dimensional, unstructured representations that accumulate rollout errors and fail to identify system-specific variation.
Latent-Conditioned Models: Often trade off expressivity, scalability, or interpretability, or rely on expensive meta-learning adaptations.

Methodology: Neural Operator Discovery (NOD)

The authors formulate Neural Operator Discovery (NOD) as the problem of learning a shared solution operator and system-specific variation directly from heterogeneous trajectories without parameter supervision. The proposed framework, NODnet, employs a factorized realization consisting of two components:

Trajectory Encoder ( $f_\phi$ ): Maps a past-state window (conditioning trajectory) from a system instance to a low-dimensional latent representation $c$ .
State-Evolution Operator ( $g_\theta$ ): A conditional neural operator that predicts future states based on an initial state and the latent representation $c$ .

To ensure the latent representation captures governing factors rather than transient trajectory features, NODnet integrates three key mechanisms:

1. Factorized Prediction

Instead of predicting future states directly from a past-state window (as in ANO), the model factorizes the process:
$c = f_\phi(x'_{0:s}), \quad \hat{x}_{t+1} = g_\theta(x_t; c)$
The encoder infers a latent $c$ from a conditioning trajectory, while the decoder predicts the evolution of a separate trajectory from the same system instance.

2. Trajectory-Decoupled Sampling (TDS)

To prevent the latent representation from encoding initial-condition-specific information, the encoder and decoder are trained using trajectories generated from different initial conditions of the same system instance.

The encoder infers $c$ from a conditioning trajectory $x'_{0:s}$ .
The decoder predicts the evolution of a distinct trajectory $x_{t:t+1}$ from the same system.
This constraint encourages the encoder to retain only cross-trajectory invariants (the governing factors) and suppress transient features.

3. Dimension Selection

The latent dimension $d$ is selected as the smallest value sufficient to explain differences across system instances. This is achieved via a two-stage procedure:

Stage 1: Identify candidate dimensions where increasing $d$ no longer yields significant improvements in training loss beyond random seed variability.
Stage 2: Select the final dimension $d^*$ as the smallest candidate where the $(d+1)$ model does not produce additional factor-aligned structure in the latent space (assessed via Principal Component Analysis of per-instance latent means).

Key Contributions

Formulation of NOD: A new problem formulation for learning shared operators and system-specific variations without labeled governing factors.
Factorized Latent-Conditioning: A novel architecture that decouples trajectory encoding from state evolution, enabling the discovery of low-dimensional, interpretable latent spaces.
Trajectory-Decoupled Sampling: A training strategy that enforces the extraction of system-level invariants by pairing conditioning and prediction trajectories with different initial conditions.
Data-Driven Dimension Gauging: A systematic procedure to automatically determine the intrinsic dimensionality of system variation, matching the number of independent governing factors without prior knowledge.

Results

The method was validated on three families of partial differential equations (PDEs) and four additional case studies in the supplementary information:

Burgers' Equation (Varying Viscosity):
- Result: The model correctly identified a latent dimension $d^*=1$ , corresponding to the single varying factor (viscosity $\nu$ ).
- Performance: Achieved accurate long-horizon predictions and successfully extrapolated to the inviscid limit ( $\nu=0$ ), a singular regime where classical numerical methods often struggle. The learned latent space exhibited a smooth, monotonic correspondence with viscosity.
- Comparison: Outperformed autoregressive baselines (ANO) and single-system baselines (SNO), particularly in long-horizon rollouts where ANO accumulated significant drift.
Cylinder Flow Wake (Unstructured Mesh):
- Result: Identified $d^*=1$ despite the system being parameterized by both viscosity and inlet velocity, as the dynamics in the tested Reynolds number regime are governed by a single dimensionless group ($Re$).
- Performance: Demonstrated robustness under sparse supervision (trained on only 16 trajectories) and generalization to unseen viscosities and inlet velocities.
- Physical Consistency: Recovered correct drag coefficients and Strouhal numbers, whereas autoregressive baselines exhibited frequency drift and nonphysical drag behavior.
FitzHugh–Nagumo Reaction–Diffusion System:
- Result: Correctly identified $d^*=2$ , matching the two independent governing factors ( $k, \beta$ ).
- Generalization: Demonstrated both one-shot inference (inferring $c$ from a single trajectory) and zero-shot inference (sampling $c$ directly from the learned latent space). The latent space organized system instances in a smooth, approximately invertible structure aligned with the true parameter space, enabling extrapolation beyond the training hull.
- Supervision Geometry: Showed that hierarchical decoders (supervising multiple time horizons) yielded more structured latent representations and stable rollouts compared to single-step autoregressive decoders.

Significance and Claims

The paper establishes an interpretable paradigm for operator learning in the absence of explicit factor supervision. The authors claim that:

Identifiability: The learned latent spaces consistently align with underlying governing factors, exhibiting smooth and approximately invertible structures (diffeomorphisms) with the true factor space, despite being learned without parameter labels.
Generalization: The method supports generalization to previously unseen system instances, including zero-shot extrapolation across regimes (e.g., inviscid limits) and stable long-horizon prediction.
Efficiency: By restricting the latent dimension to the minimal necessary size, the model avoids the "structural slack" that leads to error accumulation in high-dimensional latent spaces.
Complementarity: NOD offers a scalable alternative to symbolic regression for complex, high-dimensional systems where closed-form equations are difficult to recover, and a more structured alternative to autoregressive foundation models for systems with unobserved parameters.

The authors note limitations, including the current focus on continuously varying factors in low-dimensional Euclidean spaces, the use of simulation-generated data, and the need for further investigation into discrete system classes, bifurcating dynamics, and experimental noise robustness.