Streaming Operator Inference for Model Reduction of Large-Scale Dynamical Systems

This paper proposes Streaming Operator Inference, a non-intrusive model reduction framework that utilizes incremental SVD and recursive least squares to learn accurate reduced-order models from sequential data streams, thereby overcoming the memory limitations of traditional batch methods and enabling online adaptation for large-scale dynamical systems.

The Gist

Imagine you are trying to teach a computer to predict how a complex system, like a swirling storm or a flowing river, will behave in the future. Usually, to do this, you need a massive amount of data. Think of this data as a library containing millions of books, where each book is a "snapshot" of the system at a specific moment in time.

The Old Way: The "All-at-Once" Library
Traditional methods (called "Batch OpInf") try to learn from this system by loading the entire library into the computer's memory at once. They then read every single book simultaneously to find the rules (the "operators") that govern the system's behavior.

• The Problem: For huge systems, like a global weather model or a turbulent engine, the library is too big. It's like trying to fit an entire national archive into a backpack. The computer runs out of memory, or it takes so long to gather all the books that you can't make predictions in real-time. Also, if a new book arrives while you are studying, you have to stop, re-shelve everything, and start over.

The New Way: The "Streaming" Tutor
This paper introduces a new method called Streaming OpInf. Instead of trying to hold the whole library, this method acts like a smart tutor who learns as the books arrive, one by one.

Here is how it works, using two main tricks:

1. The "Sketch Artist" (Incremental SVD)
Imagine you are watching a fast-moving dance troupe. Instead of trying to memorize every dancer's exact position at every second (which is too much data), you only remember the main patterns of movement.

• The Trick: As each new dancer (data snapshot) walks onto the stage, the method quickly updates its mental "sketch" of the main moves. It doesn't store the whole troupe; it just keeps a small, efficient summary of the most important movements. This is called Incremental SVD. It's like compressing a 4K video into a tiny, high-quality GIF that still captures the essence of the dance.

2. The "Live Coach" (Recursive Least Squares)
Now that the tutor has a sketch of the dance, it needs to figure out the rules: "When the lead dancer spins left, the group follows right."

• The Trick: Instead of waiting until the end of the show to figure out the rules, the "Live Coach" updates its understanding instantly every time a new dancer steps in. This is called Recursive Least Squares. It tweaks the rules slightly with every new piece of information, refining its prediction without ever needing to look back at the old data.

Why This Matters (The Results)
The authors tested this on three different "dances":

1. A simple fluid flow (Burgers' Equation): A basic test to see if the math works.
2. A chaotic flame (Kuramoto-Sivashinsky Equation): A messy, unpredictable system where small changes lead to big differences.
3. A massive turbulent channel flow: A real-world simulation of air or water flowing through a pipe, involving nearly 10 million variables. This is the "heavy lifter" that would crash a traditional computer.

The Big Wins:

• Memory Savings: By not storing the whole library, the new method used over 99% less memory for the smaller problems and still saved a massive amount for the huge one. It's like fitting that national archive into a single notebook.
• Speed: Because the computer doesn't have to wait to load everything, it can make predictions much faster (orders of magnitude faster).
• Accuracy: Even though it's learning on the fly with less memory, it predicts the system's behavior just as accurately as the old, heavy method.
• Real-Time Potential: Because it learns as data arrives, it can adapt to new information immediately, making it perfect for "digital twins" (virtual copies of real systems) that need to update in real-time.

In Summary
This paper presents a way to teach computers to understand complex, moving systems without needing a supercomputer with infinite memory. By learning incrementally—updating their "sketches" and "rules" as data streams in—they can handle massive, real-world problems that were previously impossible to solve, all while using a fraction of the storage space.

Technical Summary

Technical Summary: Streaming Operator Inference for Model Reduction of Large-Scale Dynamical Systems

Problem Statement
Projection-based model reduction, specifically Operator Inference (OpInf), enables efficient simulation of complex dynamical systems by learning low-dimensional surrogate models from high-dimensional data without access to the underlying governing equations (non-intrusively). Traditional OpInf operates as a batch learning method, requiring the simultaneous loading of all data snapshots into memory to perform Singular Value Decomposition (SVD) for basis construction and Linear Least Squares (LS) for operator inference. This approach faces two critical barriers in large-scale applications (e.g., climate modeling, fluid dynamics, digital twins):

1. Memory Constraints: High-resolution simulations generate terabytes to petabytes of data, making it infeasible to store all snapshots in memory or on disk simultaneously.
2. Online Adaptation: Many real-world applications require real-time decision-making and model updates as new data arrives sequentially, which batch methods cannot support as they require complete data collection before training.

Existing parallel or domain decomposition strategies address memory bottlenecks by partitioning data spatially but remain batch methods regarding temporal data (requiring simultaneous access to all time snapshots). There is a lack of frameworks that utilize temporal partitioning to process data incrementally as it streams.

Methodology: Streaming OpInf
The authors propose Streaming OpInf, a framework that reformulates the two core components of OpInf to operate on sequentially arriving data streams. The method replaces batch SVD and batch LS with their streaming counterparts:

1. Streaming Basis Construction (Incremental SVD):
   Instead of computing SVD on the full snapshot matrix, the framework employs incremental SVD (iSVD) algorithms to adaptively construct the reduced basis as snapshots arrive. The paper evaluates two specific algorithms:

   • Baker's iSVD: A deterministic method that updates the SVD components via rank-1 updates. It offers minimal memory overhead ($O(nr)$) but accumulates errors with each update, particularly sensitive to small spectral gaps.
   • SketchySVD: A randomized algorithm that compresses data into low-dimensional "sketches" using random matrices. It processes data in a single pass and computes the final SVD only after all data is processed. It offers better scalability for massive datasets with tunable accuracy via sketch sizes but requires slightly more memory ($O(nq)$).

2. Streaming Operator Learning (Recursive Least Squares):
   The framework replaces the batch LS solution with Recursive Least Squares (RLS) to update reduced operators incrementally.

   • Standard RLS: Updates the inverse correlation matrix using the Sherman-Morrison identity. It is computationally efficient ($O(d^2)$ per iteration) but can suffer from numerical instability due to catastrophic cancellation in finite-precision arithmetic.
   • Inverse QR-Decomposition RLS (iQRRLS): A numerically stable variant that propagates the Cholesky factor of the inverse correlation matrix using Givens rotations. It maintains $O(d^2)$ complexity while ensuring stability.

3. Algorithmic Paradigms:
   The paper defines four distinct paradigms based on data availability and computational constraints:

   • iSVD-Project-LS/RLS: Projects both state snapshots and time derivatives onto the POD basis. This is preferred when high-quality derivative data is available, avoiding finite-difference approximation errors, though it incurs an additional data pass.
   • iSVD-LS/RLS (Reformulation): Expresses the LS data matrices directly in terms of the truncated SVD matrices (singular values and right singular vectors) obtained from iSVD. This avoids the explicit projection step and the associated $O(nKr)$ computational cost, making it suitable for extremely large $n$ or when data cannot be revisited.

Key Contributions

1. Framework Development: The proposal of a streaming approach to OpInf that learns reduced models incrementally, enabling scalability to datasets exceeding memory limits and laying the groundwork for real-time model updates.
2. Algorithmic Integration and Comparison: The implementation and systematic comparison of state-of-the-art streaming algorithms (Baker's iSVD, SketchySVD, RLS, and iQRRLS) within the OpInf framework. The authors provide analytical error bounds and numerical evaluations to guide the selection of algorithm combinations based on spectral decay, memory constraints, and accuracy requirements.
3. Scalability Demonstration: The successful application of Streaming OpInf to a large-scale turbulent channel flow simulation with nearly 10 million degrees of freedom, a problem size where traditional batch OpInf is infeasible.

Results
Numerical experiments were conducted on three benchmarks:

• Viscous Burgers' Equation: Demonstrated that Streaming OpInf achieves accuracy comparable to batch OpInf. The iSVD-RLS paradigm reduced memory requirements by over 99% while maintaining state reconstruction errors comparable to intrusive POD.
• Kuramoto-Sivashinsky Equation (KSE): A chaotic system test case. The study showed that Streaming OpInf preserves essential dynamical invariants, including Lyapunov exponents and the Kaplan-Yorke dimension, confirming the method's ability to capture the geometric and dynamical properties of chaotic attractors. The iQRRLS algorithm proved superior to standard RLS in maintaining numerical stability.
• Turbulent Channel Flow: A large-scale 3D simulation ($n \approx 9.4$ million). Using SketchySVD and iSVD-LS, the method achieved a state dimension reduction exceeding 31,000x. The learned reduced operators accurately captured turbulent structures, friction velocity, and wall-normal velocity profiles (log-law) for both training and testing data. The method achieved a 68% total memory reduction compared to batch OpInf for this specific problem, enabling model reduction where batch processing was impossible.

Significance and Claims
The paper claims that Streaming OpInf establishes a scalable framework for reduced operator learning in large-scale and online streaming settings. Its primary significance lies in:

• Breaking Memory Barriers: By eliminating the need to store complete datasets, it enables model reduction for problems with data sizes that preclude batch processing.
• Enabling Real-Time Adaptation: The recursive nature of the method allows for online model updates and predictions, addressing the needs of digital twins and system monitoring.
• Maintaining Accuracy: Despite the streaming constraints, the method achieves accuracy comparable to batch OpInf, with memory savings exceeding 99% in benchmark cases.

The authors note limitations, particularly that for highly advection-dominant flows with slow spectral decay, the reduced rank may still be insufficient to resolve all scales, and the choice of regularization is critical. They suggest future work could explore nonlinear manifold approaches and the integration of uncertainty quantification, but the current work focuses on establishing the linear polynomial operator inference framework in a streaming context.