Factorized Neural Implicit DMD for Parametric Dynamics

This paper proposes Factorized Neural Implicit DMD, a data-driven method that parameterizes the Koopman operator's spectral decomposition via a physics-coded neural field to decouple spatial modes and temporal evolution, thereby enabling stable long-term rollouts, parameter generalization, and spectral analysis for high-dimensional nonlinear dynamical systems.

The Gist

Imagine you are trying to predict how a drop of ink spreads in a glass of water, or how wind swirls around a bridge pillar. These are complex physical systems governed by invisible rules (mathematical equations). Traditionally, to predict these movements, scientists have to solve massive, complicated math problems step-by-step. It's like trying to calculate the path of every single water molecule one by one. It's accurate, but it's incredibly slow and requires a supercomputer.

This paper introduces a new, smarter way to do this called Factorized Neural Implicit DMD. Let's break down what that means using some everyday analogies.

1. The Problem: The "Black Box" vs. The "Recipe"

Current AI methods for predicting physics are often like Black Boxes. You feed them a picture of the ink at the start, and they guess what it looks like a second later. They do this by memorizing patterns.

• The Flaw: If you ask them to predict 100 seconds into the future, they start to hallucinate. The errors pile up, and the ink might turn into a solid block or vanish. Also, if you change the water temperature (a "parameter"), the AI has to relearn everything from scratch.

2. The Solution: The "Musical Orchestra"

The authors propose a method that acts more like a conductor leading an orchestra rather than a guesser.

Instead of guessing the whole picture at once, their AI learns the fundamental "notes" (modes) that make up the system.

• The Analogy: Think of a complex sound (like a symphony) not as a jumble of noise, but as a combination of specific instruments playing specific notes.
• The AI's Job: It learns to identify these "instruments" (spatial patterns) and the "tempo" (how fast they grow or shrink) for any given situation.

3. The Secret Sauce: "Physics-Coded" and "Factorized"

The paper has two main tricks that make this work so well:

A. The "Physics Code" (The Recipe Card)

Imagine you have a master chef. Usually, if you want them to cook a soup with less salt, you have to teach them the whole recipe again.
This AI has a special "Physics Code" (like a recipe card). You can write "Viscosity: Low" or "Obstacle: Big" on the card, and the AI instantly knows how to adjust its "instruments" without relearning the whole song. It understands that changing the salt just changes the flavor, not the entire nature of the soup.

B. "Factorized" (Separating the What from the When)

Most AI models mix up where things are happening with when they happen.
This model separates them:

• The "What" (Spatial Modes): These are the shapes the fluid makes (like a swirl or a wave). The AI learns these shapes using a flexible neural network.
• The "When" (Temporal Evolution): This is just a simple math rule (a clock) that tells the shapes how to move forward in time.
• Why it helps: Because the "clock" is simple and linear, the AI can predict 1,000 steps into the future without getting tired or making mistakes. It's like knowing a song's melody; you can hum it for hours without forgetting the tune.

4. The "Peeling the Onion" Strategy

One of the coolest parts of the paper is how they teach the AI to learn these "notes."

• The Problem: If you ask an AI to learn 10 patterns at once, they all get confused and overlap (like trying to learn 10 songs simultaneously).
• The Solution: They use a Stage-Wise Deflation strategy.
  • Step 1: Teach the AI the most obvious, loud pattern (the first note).
  • Step 2: "Peel it off" (subtract it from the data).
  • Step 3: Now teach the AI the next loudest pattern from what's left.
  • Result: The AI learns a clean, organized list of distinct patterns that don't interfere with each other. This makes the prediction incredibly stable.

5. Real-World Results

The authors tested this on things like:

• Burgers' Equation: Simulating shockwaves in fluids.
• Vortex Streets: Watching wind swirl behind a cylinder (like a bridge pillar).
• Airfoils: Predicting airflow over airplane wings of different shapes.

The Outcome:

• Speed: It is 60 times faster than previous methods.
• Accuracy: It makes fewer mistakes, even when predicting far into the future.
• Generalization: It can handle new shapes and conditions it has never seen before, just by reading the "Physics Code."

Summary

In short, this paper teaches AI to stop trying to memorize every single frame of a movie and instead learn the script and the cast of characters. By understanding the underlying "notes" of physics and separating the shape of the event from the timing, they created a system that is fast, accurate, and can predict the future of complex physical systems without needing a supercomputer. It turns a chaotic, messy prediction problem into a clean, organized musical performance.

Technical Summary

Here is a detailed technical summary of the paper "Factorized Neural Implicit DMD for Parametric Dynamics."

1. Problem Statement

The paper addresses the challenge of modeling the temporal evolution of physical systems governed by parametric Partial Differential Equations (PDEs).

• The Challenge: Physical systems often reside in high-dimensional state spaces with nonlinear dynamics. While traditional numerical solvers are accurate, they are computationally expensive and ill-suited for real-time analysis, control, or long-horizon prediction.
• Limitations of Existing ML Approaches:
  • Physics-Informed Neural Networks (PINNs): Typically fit a single solution surface and lack generalization to unseen parameters.
  • Neural Operators (e.g., FNO, DeepONet): Approximate solution operators at fixed time horizons but often act as "black boxes," suffering from error accumulation and instability during long-term rollouts.
  • Koopman-based Neural Operators: While they offer linear latent dynamics, they often rely on high-dimensional, implicitly defined latent operators that lack explicit spectral structure, limiting generalization across varying physical parameters and geometries.
• Goal: To learn a parametric flow operator that can predict system evolution over time, supporting long-horizon rollouts, generalization to unseen physical parameters (e.g., viscosity, boundary conditions, geometry), and spectral analysis.

2. Methodology: Factorized Neural Implicit DMD

The authors propose a Physics-Coded Neural Koopman Framework that combines the interpretability of Dynamic Mode Decomposition (DMD) with the flexibility of Implicit Neural Representations (INRs).

Core Concept

The method models the system using a reduced-order representation where the system evolution is governed by a parameterized DMD operator. The key innovation is that both the eigenfunctions (spatial modes, $\Phi$) and eigenvalues (temporal evolution, $\Lambda$) are modeled as neural fields conditioned on physical parameters ($\xi$).

Key Technical Components

1. Continuous, Conditioned Koopman Expansion:

   • The system state $u(x, t; \xi)$ is approximated as a linear combination of spatial modes $\Phi(x; \xi)$ and time-dependent coefficients $z(t; \xi)$.
   • The evolution is linear in the latent space: $z(t + \Delta t) = \Lambda(\xi) z(t)$.
   • Unlike standard DMD which learns discrete matrices, this method learns continuous neural fields $\Phi_\theta(x; \xi)$ and $\Lambda_\eta(\xi)$ that can be evaluated at any spatial coordinate and physical parameter setting.

2. Conjugate-Pair Parameterization:

   • To handle complex dynamics, the model learns complex-conjugate pairs of modes.
   • Each primary mode is represented by two real channels (real and imaginary parts) for both the basis function and the eigenvalue.
   • This ensures the reconstructed state remains real-valued and captures oscillatory behaviors (frequency) and growth/decay rates naturally.

3. Sequential Learning with Deflation (Stage-wise Optimization):

   • To avoid mode mixing and ensure orthogonality, the authors employ a stage-wise deflation strategy.
   • Modes are learned sequentially. Once a mode is learned, it is "frozen," and the next mode is trained to be orthogonal to all previous modes by projecting the raw candidate onto the null space of the previously learned subspace.
   • This yields a well-conditioned spectral decomposition with clear physical interpretability.

4. Hybrid Loss Function:

   • Short-Horizon Loss: Enforces accuracy in one-step transitions (standard DMD loss).
   • Long-Horizon Loss: Enforces consistency over multiple time steps (OptDMD style) to prevent error accumulation.
   • The total loss is a weighted sum of both, ensuring the model is accurate locally and stable globally.

3. Key Contributions

1. Physics-Coded Neural Koopman Framework: A novel architecture that generalizes to unseen PDE instances (varying parameters and geometries) while providing robust long-term predictions.
2. Neural Field Parameterization: The use of neural fields to parameterize eigenfunctions ($\Phi$) and eigenvalues ($\Lambda$) conditioned on physics codes. This enables:
   • Compact, low-rank dynamics.
   • Continuous-space evaluation (mesh-agnostic).
   • Explicit spectral structure (eigenvalues and modes are directly accessible).
3. Sequential Learning Strategy: A deflation-based training approach with orthogonality constraints that improves stability, conditioning, and the interpretability of coherent spatial modes.

4. Experimental Results

The method (termed INR-DMD) was evaluated on four diverse parametric PDE benchmarks:

• Burgers' Equation: Varying viscosity.
• 2D Navier-Stokes (Double Shear Layer): Varying initial shear-layer separation.
• Kármán Vortex Street: Varying obstacle position (fluid-solid coupling).
• Airfoil Flow: Varying 6-dimensional shape parameters (geometry and rotation).

Performance Highlights:

• Accuracy: INR-DMD achieved the lowest prediction error (rMSE) across all benchmarks compared to baselines like FNO, KNO, KAE, P-DMD, and ResKoopNet.
• Efficiency: It demonstrated significantly faster inference times (e.g., 60x speedup over P-DMD on Burgers' equation) and lower memory usage.
• Generalization: The model successfully generalized to unseen physical parameters and geometric configurations, accurately reconstructing complex flow structures (e.g., vortex streets) where black-box models failed or produced artifacts.
• Stability: The method maintained stability over long-horizon rollouts, whereas autoregressive and other Koopman-based baselines often suffered from error accumulation or numerical overflow.

Ablation Studies:

• Removing orthogonality constraints led to noisy, correlated modes and divergence.
• Removing long-horizon loss caused numerical instability and error accumulation.
• Removing deflation resulted in poor spectral separation and higher error.
• Removing conjugate pairing broke spectral symmetry and reduced stability.

5. Significance and Impact

• Interpretability: Unlike black-box neural operators, this method exposes the underlying eigenvalues, modes, and stability of the physical process. This allows for spectral analysis and a deeper understanding of the system's dynamic behavior (e.g., identifying dominant frequencies and growth rates).
• Efficiency: By reducing long-term simulation to efficient matrix multiplications in a low-dimensional latent space, the method eliminates the need for repeated, energy-intensive neural network evaluations during inference. This makes real-time control and long-horizon prediction feasible on standard hardware.
• Scientific Machine Learning: The work bridges the gap between classical dynamical systems theory (Koopman/DMD) and modern deep learning, offering a viable, structured alternative to purely data-driven autoregressive models for parametric PDEs. It demonstrates that incorporating explicit dynamical structure is crucial for reliability and generalization in scientific AI.