NN-OpInf: an operator inference approach using structure-preserving composable neural networks

The paper introduces NN-OpInf, a structure-preserving, composable neural network framework for non-intrusive reduced-order modeling that outperforms traditional polynomial methods in accuracy and stability for systems with non-polynomial nonlinearities, albeit at the cost of higher computational training requirements.

The Gist

Imagine you are trying to predict how a complex machine will move in the future. Maybe it's a swirling storm, a burning flame, or a twisting metal beam. To do this accurately, you usually need a super-computer running a massive, detailed simulation. But these simulations are so heavy and slow that you can't run them a million times to test different scenarios (like changing the wind speed or the fuel type).

You need a shortcut. You need a "mini-model" that runs fast but still tells the truth. This is called Reduced-Order Modeling (ROM).

For a long time, the best shortcuts were like Lego bricks. They worked great if the machine's movement could be described by simple, straight lines or basic curves (polynomials). But real life is messy. Storms twist, flames flicker, and metals stretch in ways that don't fit into neat Lego shapes. When the physics get too weird, these old Lego shortcuts break, giving you wrong answers or unstable results.

Enter NN-OpInf (Neural Network Operator Inference). Think of this as a smart, modular toolkit that builds a new kind of shortcut.

Here is how it works, using some everyday analogies:

1. The Problem with the Old Way (The "Lego" Limit)

Imagine trying to describe a complex dance routine using only "step forward" and "step back" commands. If the dancer just walks in a straight line, it's easy. But if they spin, jump, and slide, your simple commands fail.

• The Old Method (P-OpInf): This method tries to force every complex movement into a simple "step forward/back" (polynomial) language. It works well for simple dances but fails miserably for complex ones.
• The New Method (NN-OpInf): This method uses Neural Networks (AI brains) that can learn any shape of movement, not just straight lines. It's like giving the dancer a full vocabulary of moves instead of just two.

2. The Secret Sauce: "Structure-Preserving"

Here is the tricky part. If you just let an AI brain learn the dance, it might invent moves that break the laws of physics.

• Example: In a closed room, energy can't just disappear. If your AI model predicts the dancer suddenly loses all their energy and freezes, that's physically impossible.
• The Innovation: NN-OpInf doesn't just let the AI guess. It forces the AI to wear "physics glasses."
  • If the system needs to conserve energy (like a spinning top), the AI is forced to use a specific mathematical shape (called skew-symmetry) that guarantees energy stays constant.
  • If the system needs to dissipate heat (like a cooling cup of coffee), the AI is forced to use a shape (called positive-definite) that guarantees heat only goes out, never in.

It's like hiring a chef who knows how to cook anything, but you give them a rulebook: "You can use any spice you want, but you must keep the salt level within this specific range." The result is a dish that is both creative and safe.

3. The "Composable" Lego Set

Real-world machines are made of different parts. A car engine has friction (which slows things down), pistons (which push things), and springs (which store energy).

• The Old AI: Tried to learn the whole engine as one giant, messy black box. It was hard to train and often got confused.
• NN-OpInf: Breaks the engine into modules.
  • One AI module learns the "friction" part (and is forced to only slow things down).
  • Another AI module learns the "spring" part (and is forced to only store energy).
  • A third module learns the "push" part.
  • Then, it stitches them together.

This is like building a robot by snapping together specialized arms and legs, rather than trying to mold a whole robot out of a single blob of clay. It makes the model more stable, easier to understand, and much more accurate.

4. The Trade-off: Training vs. Running

• Training (The Homework): Because NN-OpInf is so flexible and has these strict rules, it takes a lot of computer power and time to "study" the data and learn the right moves. It's like a student studying for a very difficult, open-book exam.
• Running (The Test): Once the model is trained, it runs incredibly fast. It's about as fast as the old "Lego" models, but it gives you the correct answer for complex problems where the old models would fail.

Summary: Why This Matters

NN-OpInf is a bridge between the rigid, simple models of the past and the messy, complex reality of the future.

• It's Flexible: It can handle weird, non-linear physics (like burning fuel or twisting metal) that old models couldn't touch.
• It's Safe: It respects the laws of physics (energy, momentum) by design, so it doesn't give you "magic" results that break reality.
• It's Modular: It builds complex models by snapping together specialized, rule-abiding AI components.

In short, if you want to predict how a complex system behaves without running a supercomputer every time, NN-OpInf gives you a fast, reliable, and physics-compliant shortcut that actually works when things get complicated.

Technical Summary

Here is a detailed technical summary of the paper "NN-OpInf: an operator inference approach using structure-preserving composable neural networks."

1. Problem Statement

The paper addresses the limitations of Non-Intrusive Reduced-Order Modeling (NI-ROM) for dynamical systems, specifically focusing on Operator Inference (OpInf).

• Context: Traditional NI-ROMs, such as Polynomial Operator Inference (P-OpInf), learn reduced-order dynamics from snapshot data without accessing the full-order model (FOM) code. P-OpInf assumes the reduced dynamics follow a polynomial structure (linear or quadratic).
• Limitations of P-OpInf: Many physical systems of practical interest (e.g., finite deformation mechanics, reacting flows, non-affine parametric dependencies) do not admit a polynomial operator form. When the underlying physics are highly non-polynomial, P-OpInf suffers from poor accuracy and stability.
• Limitations of Existing NN-ROMs: Previous attempts to replace polynomials with Neural Networks (NNs) often result in "black-box" models that lack physical structure (e.g., conservation laws), are difficult to train (non-convex), and can be unstable or inaccurate compared to intrusive methods.

2. Methodology: NN-OpInf

The authors propose NN-OpInf, a framework that replaces the polynomial ansatz in OpInf with a composable, structure-preserving sum of neural network operators.

Core Formulation

Instead of a single monolithic neural network, the reduced velocity field $\dot{\hat{x}}$ is modeled as an additive sum of $M$ distinct operator blocks:
$$\dot{\hat{x}}(t, \mu) = \sum_{r=1}^{M} \hat{g}_r(\eta_r; w_r)$$
Where:

• $\hat{g}_r$ are neural network-based operators.
• $\eta_r$ are specific inputs (combinations of reduced state $\hat{x}$ and parameters $\mu$).
• $w_r$ are trainable weights.

Key Architectural Features

1. Composability: The framework allows mixing heterogeneous operators. For example, a diffusion term (modeled as Symmetric Positive Semi-Definite) can be added to an advection term (modeled as Skew-Symmetric) and a forcing term.
2. Structure Preservation: The neural networks are parameterized to enforce specific algebraic structures by construction, ensuring physical invariants are respected:
   • Skew-Symmetric Operators: Enforced via strictly lower-triangular matrices ($S - S^T$) to preserve energy (conservative dynamics).
   • Symmetric Positive Semi-Definite (SPSD) Operators: Enforced via Cholesky-like factorization ($LL^T$) to model dissipative dynamics (energy decay).
   • Gradient/Potential Operators: Derived from a scalar potential to enforce Lagrangian/Hamiltonian structures.
3. Training Strategy:
   • Optimization: Uses a hybrid approach alternating between L-BFGS (to quickly find local minima) and ADAM (to escape local minima and refine the solution), addressing the non-convex nature of the problem.
   • Normalization: Employs max-abs normalization to preserve the algebraic structure of operators and the equivalence of norms between reduced and full spaces.
   • Ensembling: Trains multiple independent models and averages their predictions to reduce variance and improve robustness.

Software Implementation

The authors released NN-OpInf, an open-source Python package built on PyTorch. It provides a modular API for defining variables, structured operators, and models, facilitating the construction of complex, composable ROMs.

3. Key Contributions

1. Novel Framework: Introduction of a composable, structure-preserving NN-OpInf framework that overcomes the expressiveness limits of polynomial OpInf while maintaining physical interpretability.
2. Structured Operators: Development of specific neural architectures that enforce skew-symmetry, SPSD, and gradient structures, ensuring stability and conservation laws in the reduced model.
3. Systematic Comparison: The first systematic comparison between neural-network-based and polynomial-based operator inference across multiple problem classes (nonlinear, parametric, and structural mechanics).
4. Open Source: Release of the NN-OpInf software package to enable reproducibility and further research.
5. Cost Analysis: A detailed analysis of computational costs (FLOPs) and training complexity, highlighting the trade-off between higher offline training costs and improved online accuracy/stability.

4. Numerical Results

The authors evaluated NN-OpInf on five distinct problems, comparing it against P-OpInf (linear/quadratic), standard NN-ROMs, and intrusive Galerkin ROMs.

• Burgers' Equation (Quadratic):
  • NN-OpInf with a skew-symmetric operator (NN-OpInf-SS) matched the accuracy of P-OpInf in training but outperformed it in future-state prediction by preserving energy conservation, whereas P-OpInf violated energy constraints over time.
• Nonlinear Convection-Diffusion-Reaction (Non-Polynomial):
  • P-OpInf failed to capture the solution (errors > 5%) due to non-polynomial nonlinearities.
  • NN-OpInf with a composite SPSD + Skew + Forcing operator (NN-OpInf-PSD-f) achieved accuracy comparable to the intrusive Galerkin ROM and was 5–10x more accurate than P-OpInf.
• 2D Nonlinear Heat Conduction:
  • NN-OpInf-SPSD-f significantly outperformed both vanilla NN-OpInf and P-OpInf, demonstrating the necessity of structure preservation for stability in nonlinear diffusion problems.
• Premixed H2-Air Flame (Parametric):
  • In a regime dominated by polynomial terms, P-OpInf was competitive. However, NN-OpInf-PSD-f showed superior out-of-distribution performance on the testing set, demonstrating better generalization for non-affine parameter dependencies.
• 3D Hyper-Elastic Torsion (Lagrangian Structure):
  • For finite deformation mechanics (highly nonlinear), standard P-OpInf and vanilla NN-ROMs were unstable or inaccurate.
  • NN-OpInf using an SPSD Potential operator (preserving Lagrangian structure) was an order of magnitude more accurate than P-OpInf and provided stable, physically consistent trajectories.

5. Significance and Conclusion

• Accuracy & Robustness: NN-OpInf consistently provides higher accuracy and stability than P-OpInf, particularly for systems where polynomial approximations are insufficient. It also outperforms "vanilla" neural network ROMs by embedding physical constraints.
• Trade-offs: The primary cost of NN-OpInf is higher offline training time due to non-convex optimization and iterative gradient descent. However, the online evaluation cost is comparable to quadratic P-OpInf (scaling as $O(K^3)$ for structured operators).
• Impact: This work establishes NN-OpInf as a viable "drop-in" replacement for P-OpInf in complex, non-polynomial systems. It bridges the gap between the interpretability of physics-based models and the expressiveness of deep learning, offering a robust path for reduced-order modeling in multiphysics applications.