Nonlinear projection-based model order reduction with machine learning regression for closure error modeling in the latent space

This paper presents a novel nonlinear projection-based model order reduction framework that utilizes Gaussian process regression and radial basis function interpolation to model closure errors in the latent space, offering improved efficiency, interpretability, and data efficiency compared to deep neural network approaches in complex fluid dynamics applications.

The Gist

Imagine you are trying to predict the weather. You have a supercomputer running a massive, incredibly detailed simulation of the atmosphere. It tracks every single molecule of air, cloud, and wind current. This is the "High-Dimensional Model" (HDM). It's accurate, but it takes days to run a single forecast. You need a faster way to get the answer, but you can't just throw away the details, or your prediction will be wrong.

This is the problem of Model Order Reduction (MOR). Scientists want to build a "mini-me" version of that supercomputer simulation—a small, fast model that still captures the essential behavior of the weather.

The Problem: The "Flat Map" vs. The "Rolling Hills"

For simple things, you can flatten the data onto a straight line or a flat sheet (a linear model). But weather, and many other physical phenomena like shockwaves in air or turbulent water, are messy and curved. They live on a complex, twisted shape (a "nonlinear manifold").

If you try to flatten a rolling hill onto a flat piece of paper, you lose the hills and valleys. In the past, scientists tried to fix this by using Deep Neural Networks (ANNs)—essentially, complex, "black box" AI brains—to learn how to fold and unfold that paper correctly. These AI brains worked well, but they had two big flaws:

1. They were opaque: You couldn't easily explain why the AI made a specific prediction. It was a mystery.
2. They were hungry: They needed mountains of data to learn. If you didn't have enough data, they would fail or require you to run the slow supercomputer even more times just to feed the AI.

The New Solution: The "Smart Compass" and the "Rubber Sheet"

This paper introduces two new, simpler tools to replace the "black box" AI: Gaussian Process Regression (GPR) and Radial Basis Function (RBF) interpolation.

Think of the problem like this:
You have a main map (the "Retained Modes") that shows the big picture. But this map is missing some fine details (the "Discarded Modes"). In the old method, you used a complex AI to guess the missing details based on the big picture.

The new method uses two different approaches to guess those missing details:

1. Gaussian Process Regression (GPR) is like a "Smart Compass with a Confidence Meter."
   Instead of just guessing, GPR looks at the data points you have and draws a smooth curve through them. Crucially, it also tells you how sure it is about that curve. It's like a compass that says, "I'm 99% sure the path goes this way, but if you go too far off the known trail, I'm less certain." This makes the model interpretable (you can see the logic) and efficient (it doesn't need as much data to get it right).

2. Radial Basis Function (RBF) is like a "Rubber Sheet."
   Imagine you have a few pins stuck in a rubber sheet representing your data points. If you pull on one pin, the whole sheet stretches and deforms in a predictable, mathematical way. RBF uses this stretching logic to fill in the gaps between your data points. It's a very fast, deterministic way to guess the missing details without needing a complex neural network.

The "Latent Space" Secret

The paper uses a clever trick called "Latent Space Closure." Imagine you are trying to describe a complex dance.

• The Old Way: You try to describe every single muscle movement of the dancer (too much data!).
• The New Way: You describe the dancer's main pose (the "Retained Modes"). Then, you use your "Smart Compass" (GPR) or "Rubber Sheet" (RBF) to automatically figure out the subtle, hidden movements (the "Discarded Modes") that must happen to make that pose look real.

This allows the model to stay tiny (fast) but still capture the complex, wiggly details of the real physics.

The Test Drives

The authors tested this on two very difficult scenarios:

1. The Shockwave Problem (Burgers' Equation): Imagine a shockwave (like a sonic boom) ripping through a 2D square of air. These waves are sharp and move fast.

   • Result: The new methods (GPR and RBF) were just as accurate as the complex AI, but they were 43 to 47 times faster than the original super-slow simulation. They also handled the sharp shockwaves much better than the old "flat map" methods, which tended to get shaky and oscillate.

2. The Car Aerodynamics Problem (Ahmed Body): Imagine simulating the turbulent, swirling air behind a car (the "Ahmed Body") to see how drag affects fuel efficiency. This is a 3D, chaotic, swirling mess.

   • Result: The new methods were incredibly efficient. The RBF method, in particular, was a superstar. It achieved a 333 times speedup in wall-clock time and nearly 10,000 times speedup in CPU time compared to the full simulation, while keeping the error incredibly low (under 2.5%).

The Takeaway

This paper shows that you don't always need a giant, complex "black box" AI to solve difficult physics problems. Sometimes, simpler, more transparent tools like GPR and RBF are better.

• They are faster: They need less data to train.
• They are clearer: You can understand how they work (interpretability).
• They are just as accurate: They handle complex, messy physics (like shockwaves and turbulence) just as well as the heavy-duty AI, but with a fraction of the cost.

In short, the authors found a way to make the "mini-me" models not only smaller and faster, but also smarter and easier to trust.

Technical Summary

Technical Summary: Nonlinear Projection-Based Model Order Reduction with Machine Learning Regression for Closure Error Modeling

Problem Statement
Parametric numerical simulations of complex physical phenomena, particularly in computational fluid dynamics (CFD), often rely on high-dimensional models (HDMs) that pose significant computational bottlenecks. While Projection-Based Model Order Reduction (PMOR) offers a strategy to construct low-dimensional surrogate models, linear subspace methods frequently fail for nonlinear systems due to the Kolmogorov $n$-width barrier. This barrier limits the efficiency of linear approximations when solutions exhibit complex structures, such as propagating shocks or turbulent wakes.

Previous advancements in nonlinear PMOR have utilized deep Artificial Neural Networks (ANNs) to model "closure errors" within a latent space, effectively capturing the relationship between retained and discarded modes. However, this approach presents two primary limitations:

1. Lack of Interpretability: The "black-box" nature of deep ANNs hinders the development of rigorous theoretical a priori error estimates or indicators.
2. Data Scarcity: Training deep ANNs often requires extensive datasets. In scenarios where the solution manifold has a very low intrinsic dimension (requiring a small number of retained modes, $n$), the available training data from high-dimensional snapshots may be insufficient for effective deep learning, potentially necessitating the generation of additional, costly high-dimensional snapshots.

Methodology
This paper proposes a generalized framework for nonlinear PMOR that replaces deep ANNs with interpretable machine learning regression techniques for modeling the closure error in the latent space. The core methodology involves:

1. Latent Space Decomposition: The solution manifold is approximated using a partitioned Reduced-Order Basis (ROB). The high-dimensional solution $u$ is approximated as $u \approx u_{ref} + Vq + \bar{V}\bar{q}$, where $q$ represents primary generalized coordinates (retained modes) and $\bar{q}$ represents secondary coordinates (discarded modes).
2. Closure Error Modeling: Instead of assuming a linear relationship, a nonlinear map $\mathcal{N}: \mathbb{R}^n \to \mathbb{R}^{\bar{n}}$ is learned offline such that $\bar{q} = \mathcal{N}(q)$. This map captures the closure error associated with truncating the basis.
3. Regression Alternatives: The paper investigates two interpretable regression methods to construct $\mathcal{N}$:
   • Gaussian Process Regression (GPR): A probabilistic, non-parametric approach using kernels (specifically Matérn kernels with $\nu=1.5$) to model the mapping. It provides analytical Jacobians and uncertainty estimates.
   • Radial Basis Function (RBF) Interpolation: A deterministic approach using weighted linear combinations of radial kernels (e.g., inverse multiquadric) to approximate the map. It also supports analytical Jacobians.
4. Integration with LSPG and ECSW: These nonlinear maps are integrated into the Least-Squares Petrov-Galerkin (LSPG) projection framework for solving the reduced-order system. To ensure computational efficiency during the online phase, the Energy-Conserving Sampling and Weighting (ECSW) hyperreduction technique is employed to decouple online costs from the original HDM dimension.

Key Contributions

• Interpretability and Theoretical Potential: By utilizing GPR and RBFs, the framework moves away from the opacity of deep ANNs. The closed-form nature of these regressors allows for the derivation of analytical Jacobians and offers a pathway toward rigorous a priori error estimation, addressing a critical theoretical gap in ANN-based PMOR.
• Data Efficiency: The proposed methods require significantly less training data than deep ANNs. They can effectively model closure errors using only the initial set of high-dimensional solution snapshots, eliminating the need to generate additional expensive snapshots for training when the reduced dimension $n$ is very small.
• Generalized Framework: The paper demonstrates that these regression techniques can be seamlessly integrated into the existing LSPG-ECSW pipeline, maintaining the independence of computational complexity from the HDM dimension $N$.

Results
The methodology was validated on two demanding applications:

1. 2D Parametric Inviscid Burgers Problem:

   • Setup: A shock-dominated flow problem with $N=125,000$ degrees of freedom.
   • Performance: The hyperreduced models (HPROM-GPR and HPROM-RBF) achieved speedups of approximately 43–47 times relative to the HDM, with relative errors below 2%.
   • Comparison: These results were comparable to HPROM-ANN (speedup ~43x) but with significantly reduced training data requirements and improved interpretability. The nonlinear models effectively captured moving shocks with fewer oscillations compared to traditional affine HPROMs.

2. Ahmed Body Turbulent Wake Flow:

   • Setup: A 3D turbulent flow simulation ($N \approx 1.7 \times 10^7$) using the Detached Eddy Simulation (DES) model.
   • Performance: At a reduced dimension of $(n, \bar{n}) = (39, 597)$, HPROM-GPR and HPROM-RBF achieved wall-clock speedups of ~64 times and CPU speedups of ~1,930 times, maintaining high accuracy (DTW errors < 1%).
   • Extreme Reduction: HPROM-RBF was tested with even lower dimensions ($n=5$), achieving a wall-clock speedup of 333 times and a CPU speedup of ~9,990 times with a 2.5% error margin.
   • Comparison: In this case, GPR and RBF closures outperformed HPROM-ANN in terms of speedup and achieved comparable or superior accuracy, while requiring less training time (e.g., 2.1 minutes for RBF vs. 50.7 minutes for ANN).

Significance and Claims
The paper claims that replacing deep ANNs with GPR and RBF interpolation in latent-space closure error modeling significantly broadens the scope of nonlinear PMOR. The primary significance lies in:

• Enhanced Efficiency: Achieving substantial computational speedups (up to three orders of magnitude in wall-clock time and five in CPU time) for complex, high-dimensional problems.
• Robustness in Data-Scarce Regimes: The ability to construct highly accurate, low-dimensional models ($n \ll n_{tra}$) without the need for massive training datasets or additional high-dimensional simulations.
• Interpretability: Providing a transparent, mathematically tractable alternative to deep learning, which paves the way for future theoretical developments in error estimation and adaptive sampling.

The authors conclude that these techniques effectively mitigate the Kolmogorov barrier in complex flows, offering a balance of accuracy, efficiency, and interpretability that is particularly well-suited for shock-dominated and turbulent parametric problems.