A Quadratic Order Reduction -- Gaussian Process Ordinary Differential Equation framework for the inference of Large Continuous Dynamical Systems

This paper proposes a robust framework for forecasting complex dynamical systems that integrates Gaussian Process Ordinary Differential Equations with quadratic order model reduction and sphere projection to achieve accurate, stable, and uncertainty-quantified predictions that outperform existing reduced-order modeling methods.

The Gist

Imagine you are trying to predict the future path of a chaotic, swirling storm, or the way heat moves through the Earth's atmosphere. These are "dynamical systems"—complex machines where everything affects everything else. The problem is that these systems are huge, messy, and full of surprises. If you try to simulate them exactly, your computer crashes. If you try to simplify them too much, your predictions become wrong.

This paper introduces a new "smart shortcut" called GPRODE and QGPRODE to solve this problem. Think of it as building a Digital Shadow of a real-world system—a simplified, data-driven twin that can run fast on a computer while still telling you how uncertain it is about its own predictions.

Here is how the authors' method works, broken down into simple concepts:

1. The Problem: The "Over-Confident" Predictor

Most current methods try to learn the rules of a system by looking at past data.

• The Issue: If you just teach a computer to memorize the past, it often gets "over-confident." It might predict a storm will stay calm forever because it saw calm weather yesterday, even though storms are chaotic.
• The Need: We need a model that learns the rules, runs fast, and admits, "I'm not 100% sure, here is a range of possibilities."

2. The Core Idea: The "Gaussian Process" (The Smart Guessing Machine)

The authors use a tool called a Gaussian Process (GP).

• The Analogy: Imagine you are trying to guess the shape of a hill based on a few scattered pebbles (data points). A standard map might draw a straight line between them. A Gaussian Process is like a flexible rubber sheet stretched over the pebbles. It doesn't just guess the height; it also calculates how "wobbly" or uncertain that guess is.
• The Innovation: Usually, GPs are used to guess a static shape. The authors use it to guess the speed and direction (the "drift") of the system at every moment. They turn this into a "Random Differential Equation." Instead of saying "The wind will go North," it says "The wind is likely to go North, but here is the probability it might veer East."

3. The First Model: GPRODE (The Fast Learner)

This is the base model.

• How it works: It looks at the data, learns the "rules of motion" using the rubber sheet (GP), and then simulates the future.
• The Safety Net: Because the rubber sheet can get wobbly if you don't have enough data, the model has a built-in "uncertainty meter." As time goes on and the model looks further into the future, this meter gets bigger. It effectively says, "The further out I predict, the less sure I am," preventing the model from making wild, impossible guesses.
• The Result: It is very fast and gives you a prediction plus a "confidence band" (a margin of error).

4. The Second Model: QGPRODE (The Bounded, Stable Learner)

Sometimes, even with the uncertainty meter, the model might still go crazy (like a car driving off a cliff) because the math gets too complex.

• The Fix: The authors add a "Quadratic Order Reduction."
• The Analogy: Imagine the system is a giant, wild dancer. Instead of trying to track every limb in 3D space, you project the dancer's movement onto a globe (a sphere).
  • By forcing the data to live on the surface of a sphere, you guarantee the dancer can never run off into infinity. They are stuck on the ball, which keeps the simulation stable.
  • They also use a "quadratic" trick (a specific type of math relationship) to capture the complex, curvy movements of the dancer without needing a supercomputer.
• The Result: This version (QGPRODE) is even more stable. It prevents the model from "hallucinating" impossible physics, making it perfect for real-world systems like ocean currents or global temperatures.

5. What They Tested It On

The authors didn't just talk about theory; they tested their "Digital Shadow" on four very different scenarios:

1. The Lorenz System: A classic, chaotic weather model. Their method predicted it better and faster than older methods.
2. A Synthetic Quadratic System: A made-up math problem designed to be tricky. Their method solved it with high precision.
3. The BV-α Model: A complex simulation of fluid flow (like water swirling). Their method captured the flow patterns better than competitors.
4. Earth's Air Temperature: They used real data from the ERA5 dataset to predict global air temperatures. Their model (QGPRODE) was the most accurate and fast enough to update daily as new data arrives.

6. Why This Matters (According to the Paper)

The paper claims this framework is a "Digital Shadow" tool.

• Speed: It is fast enough to update in real-time as new sensor data comes in.
• Trust: It doesn't just give a single number; it gives a range of likely outcomes, telling you how risky the prediction is.
• Stability: Unlike other methods that might crash or go wild when the data is messy, the "sphere projection" keeps the model grounded.

In summary: The authors built a new way to predict the future of complex systems. They combined a "smart guessing machine" (Gaussian Process) with a "stability cage" (sphere projection) to create a tool that is fast, accurate, and honest about its own uncertainty. They proved it works on everything from chaotic math equations to real-world global weather data.

Technical Summary

Technical Summary: A Quadratic Order Reduction - Gaussian Process Ordinary Differential Equation Framework

Problem Statement

Forecasting the evolution of complex dynamical systems remains a fundamental challenge due to pronounced nonlinear interactions, high-dimensional state spaces, and the necessity for rigorous uncertainty quantification. While Reduced-Order Modelling (ROM) techniques, such as Proper Orthogonal Decomposition (POD) combined with Galerkin projection or Dynamic Mode Decomposition (DMD), offer computational benefits, they often suffer from inherent trade-offs between predictive accuracy, numerical stability, and interpretability. Specifically, standard ROMs can experience stability deterioration and error accumulation when applied to nonlinear systems or those with significant parametric variability. Conversely, kernel-based methods offer flexibility and universal approximation capabilities but often lack stability guarantees, requiring ad-hoc regularisation, and may fail to ensure bounded trajectories over long time horizons. The paper aims to address these limitations by developing a framework for "Digital Shadows"—continuously updated data-driven models capable of near real-time forecasting with rigorous uncertainty quantification.

Methodology

The authors propose a two-tiered framework: the Gaussian Process Ordinary Differential Equation (GPRODE) model and its extension, the Quadratically Reduced Gaussian Process Ordinary Differential Equation (QGPRODE).

1. GPRODE Framework

The core of GPRODE is the approximation of the drift term $f$ of an autonomous ordinary differential equation (ODE), $\dot{X} = f(X)$, using a Gaussian Process (GP).

• Drift Inference: Instead of learning the state evolution directly, the model learns the drift function $\dot{X}$ from data. The GP posterior mean $\mu(x)$ approximates the true drift, while the posterior variance $\sigma(x)$ quantifies uncertainty.
• Stochastic Formulation: The deterministic ODE is replaced by a Random Differential Equation (RDE) or Stochastic Differential Equation (SDE). To ensure mathematical well-posedness (specifically Lipschitz continuity of the diffusion coefficient), the authors introduce a modified diffusion term $b(x)$ that bounds the variance growth.
• Augmented Dynamics: The system is formulated as an augmented ODE for the mean trajectory $E(t)$ and a scalar variance proxy $V(t)$. This avoids the computational cost of propagating a full covariance matrix, reducing memory requirements from $O(m^2)$ to $O(1)$ for the variance component.
• Convergence: Theoretically, under assumptions of smooth drift functions and infinitely dense data, the GP posterior mean converges to the true drift, and the stochastic model converges to the underlying deterministic ODE.

2. QGPRODE Framework

To address overfitting and instability in high-dimensional settings with limited data, GPRODE is extended to QGPRODE by integrating Quadratic Model Order Reduction (QMOR) and sphere projection.

• Quadratic Reduction: The high-dimensional state $X(t)$ is decomposed into a low-dimensional latent variable $Z(t)$ on a unit sphere $S^k$ via a quadratic reconstruction: $X(t) \approx c + W_1 Z(t) + W_2 \text{vec}(Z(t) \otimes Z(t))$.
• Sphere Projection: The latent dynamics are constrained to the unit sphere to enforce boundedness. This is achieved by solving an optimisation problem to find a metric matrix $M^\star$ that projects the reduced coordinates onto the sphere.
• Latent Inference: The GPRODE methodology is applied to the latent variable $Z(t)$. The dynamics of the mean and variance in the latent space are modified to ensure the trajectory remains on the sphere (using a symplectic integrator).
• Moment Closure: To reconstruct the full state statistics from the latent moments, the framework approximates higher-order centred moments (third and fourth) of the latent variable, allowing for the computation of the mean and variance of the reconstructed physical state.

Key Contributions

1. Novel Framework: Introduction of GPRODE, which performs inference directly on the continuous-time drift term using GPs, decoupling training numerical schemes from testing integration schemes.
2. Stability and Boundedness: Development of QGPRODE, which combines quadratic order reduction with sphere projection to enforce trajectory boundedness and stability, addressing the unconstrained nature of standard kernel-based autoregressive models.
3. Rigorous Uncertainty Quantification: The framework provides principled uncertainty estimates (variance bounds) that evolve over time, with theoretical convergence guarantees in the infinite-data limit.
4. Computational Efficiency: The method is designed for rapid inference and model updates, scaling linearly with data dimensionality in the testing phase, making it suitable for "Digital Shadow" applications requiring frequent retraining.

Experimental Results

The authors evaluated the proposed methods against Extended Dynamic Mode Decomposition (EDMD), Bagging Optimised DMD (BOPDMD), and Linear and Nonlinear Disambiguation Optimisation (LANDO) across four test cases:

• Lorenz System: GPRODE achieved significantly lower errors (scaled $L_1 \approx 2.22 \times 10^{-4}$) compared to EDMD and BOPDMD, with competitive computational times. It outperformed LANDO in both accuracy and speed.
• Synthetic Quadratic System: QGPRODE demonstrated superior precision (scaled $L_1 = 0.02$) compared to all baselines, validating its ability to capture quadratic latent dynamics.
• BV-$\alpha$ Geophysical Flow Model: In this high-dimensional fluid dynamics case, QGPRODE achieved the lowest errors (scaled $L_1 = 0.02$) while maintaining short training and testing times (<1 second). Standard GPRODE suffered from overfitting due to the lack of rank truncation, whereas QGPRODE's projection stabilized the solution.
• Global Air Temperature (ERA5): Applied to real-world data for forecasting Earth's air temperature, QGPRODE outperformed all baselines in accuracy (scaled $L_1 = 0.05$) while remaining computationally competitive. The model successfully reproduced spatial distributions and provided uncertainty bounds that contained the true state.

In all cases, the proposed methods demonstrated a better balance of accuracy and computational cost compared to existing ROM techniques, particularly in scenarios involving nonlinear dynamics and limited data availability.

Significance and Claims

The paper claims that the GPRODE and QGPRODE frameworks provide a robust and stable tool for forecasting complex dynamical systems with rigorous uncertainty quantification. The authors assert that their approach successfully bridges the gap between the flexibility of kernel methods and the stability requirements of dynamical systems.

The significance of the work lies in its ability to:

• Provide a priori guarantees on computational complexity during both training and testing.
• Enable fast model updates when new data becomes available, a critical feature for operational Digital Shadows.
• Offer principled uncertainty quantification that evolves with the prediction horizon, unlike many deterministic ROMs.
• Achieve high-fidelity forecasting for large-scale systems (e.g., global weather, fluid dynamics) where traditional methods struggle with stability or overfitting.

The authors conclude that while the framework is currently limited by the monotonic increase of variance (a trade-off for efficiency), it represents a significant step forward in creating reliable, data-driven digital twins for complex engineering and environmental systems. Future work is proposed to extend the methodology to controlled dynamical systems and parameter-space reduced-order models.