⚡ electrical engineering

Learning Nonlinear Reduced-Order Models of Fluid Flows with InfoVAE

This paper proposes an efficient reduced-order modeling framework for chaotic fluid flows by combining an Information Maximizing Variational Autoencoder (InfoVAE) with an Easy-Attention-based Transformer to overcome conventional autoencoder limitations, yielding interpretable, disentangled latent representations that align with dominant Proper Orthogonal Decomposition modes.

Original authors: Manal ZETTAM, Mouhcine MOUDA, Mohammed MADIAFI, Omar ASKOUR, Mustapha OUJAOURA, Jamal EZZAHAR

Published 2026-08-10

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Manal ZETTAM, Mouhcine MOUDA, Mohammed MADIAFI, Omar ASKOUR, Mustapha OUJAOURA, Jamal EZZAHAR

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict the path of a single leaf swirling in a chaotic river. The water doesn't just flow; it twists, turns, and collides with itself in a dizzying dance of tiny eddies and massive currents. This is the world of fluid dynamics, the science of how liquids and gases move. While we have powerful mathematical rules (called the Navier-Stokes equations) that describe this motion perfectly, solving them for real-world situations is like trying to count every single grain of sand on a beach while the tide is coming in. It requires supercomputers and takes forever. To make things faster, scientists use "reduced-order models." Think of these as a sketch of the river instead of a photograph. Instead of tracking every drop of water, they try to find the few key patterns that explain most of the movement, like the main swirls, allowing us to predict the future without doing all the heavy lifting.

However, these sketches have a problem. Traditional methods are like trying to draw a complex, twisting tornado using only straight lines. They work okay for simple, predictable flows, but when the fluid gets messy, chaotic, and truly turbulent, straight lines just can't capture the wild curves. This is where a new approach using artificial intelligence steps in, trying to learn the "shape" of the chaos rather than just forcing it into a straight line.

In this paper, a team of researchers from Morocco proposes a clever new way to draw these sketches using a specific type of AI called an InfoVAE (Information Maximizing Variational Autoencoder). They tested this idea on computer simulations of water flowing past two flat plates. In one test, the water flowed smoothly and predictably (like a rhythmic dance), and in the other, it went wild and chaotic (like a mosh pit). The researchers found that their new AI method was excellent at compressing the complex water data into a tiny, simple summary. It learned to separate the important "moves" of the fluid from the noise, creating a low-dimensional map that looked very similar to the patterns found by older, linear methods but handled the messy parts much better. They then paired this AI map with another tool called a Transformer to predict how the flow would change over time. While the method worked very well for the smooth, rhythmic flow and showed great promise for the chaotic one, the team noted that they hit some computer limits when trying to run the chaotic simulation for a long time. Still, the results suggest that this AI approach is a powerful new tool for understanding and predicting the wild behavior of fluids without needing to simulate every single drop.

The Story of the Flow

Imagine you are a director trying to film a movie about a river. You have a camera that records every single water molecule. That's a lot of data! If you tried to store every frame of every molecule, your hard drive would explode. So, you decide to hire a smart assistant (the AI) to watch the movie and write a summary.

The old way of doing this was like a rigid librarian. The librarian would say, "Okay, the river moves in straight lines. We will only track the water moving left, right, up, and down." This works fine if the river is calm. But if the river starts spinning, swirling, and crashing into itself, the librarian gets confused. The straight lines don't fit the curves, and the summary becomes a mess. This is what happens with the traditional "Proper Orthogonal Decomposition" (POD) method mentioned in the paper. It's great for simple flows but struggles when things get nonlinear and chaotic.

The researchers in this paper decided to try a different kind of assistant: an InfoVAE. Think of this AI as a creative artist who doesn't just look for straight lines. Instead, it looks for the essence of the movement. It asks, "What are the few key 'moves' that make up this dance?"

They tested this artist on two different "movies":

The Rhythmic Dancer (Re = 40): Here, the water flows past two plates and creates a regular, repeating pattern of swirling vortices. It's like a metronome. The researchers found that the InfoVAE could compress this entire movie into just 2 key "moves" (latent dimensions). It captured 94% of the energy of the flow, which is almost as good as the old librarian method (which captured 98.4%). But here's the cool part: the AI didn't just copy the old method; it learned these patterns on its own, and the patterns it found looked exactly like the swirling wakes you'd expect to see in real life.
The Mosh Pit (Re = 100): In this scenario, the water flows past the plates at a different angle and moves much faster. The flow becomes chaotic, unpredictable, and wild. This is the "mosh pit" where everything is bumping into everything else. The old librarian (POD) would struggle here because the straight lines just don't work. The InfoVAE, however, was able to find 6 key "moves" to describe this chaos. It successfully compressed the massive amount of data into a tiny summary that still looked like the real thing.

How They Did It

The researchers built a machine learning system with two main parts. First, they used the InfoVAE to act as a compressor. It took the huge, high-definition video of the water flow and squished it down into a tiny, low-dimensional "latent space." You can think of this latent space as a secret code. Instead of storing millions of numbers for every water drop, the AI stores just a few numbers that represent the main swirls and waves.

The paper explains that older AI models (called standard Variational Autoencoders) often have a problem where they underutilize the latent space. They might ignore the secret code and just try to guess the picture from memory, or they might get confused about what the code actually means. The InfoVAE fixes this by using a special rule (a mathematical objective function) that forces the AI to keep the secret code useful and informative. It makes sure the code actually tells the story of the water flow, rather than just being random noise.

Once the AI had this secret code, they used a second tool called an Easy-Attention Transformer to predict the future. Imagine you have a sequence of secret codes representing the river's dance. The Transformer looks at the last few codes and guesses what the next one will be. It's like predicting the next step in a dance routine. The researchers trained this on 90% of the data and tested it on the remaining 10%.

What They Found

The results were quite promising. For the rhythmic flow (Re = 40), the AI's secret code was so good that when they turned it back into a picture of the water, it looked almost identical to the original. The "swirls" the AI found matched the real physics perfectly. The paper notes that even if they tried to make the secret code bigger (adding more dimensions), the AI naturally decided that only 2 of them actually mattered, ignoring the rest. This shows the AI was smart enough to know exactly how much information it needed.

For the chaotic flow (Re = 100), the results were also encouraging. The AI managed to learn a compact representation of the messy flow. The training graphs showed that the error (how wrong the AI was) went down steadily, and the "regularization" (the rule keeping the code organized) stayed stable. This suggests the AI didn't just memorize the data but actually learned the underlying rules of the chaotic dance.

However, the paper is honest about its limits. The researchers wanted to run more tests and calculate more detailed numbers for the chaotic flow, but their computer (the Marwan HPC cluster) kept crashing or running out of time before they could finish the long simulations. Because of this, they couldn't report every single statistic they wanted for the chaotic case. They also noted that they only tested two specific speeds (Reynolds numbers of 40 and 100) and only in two dimensions (flat, 2D water). They didn't test it on 3D flows (like real rivers in 3D space) or at even higher speeds.

The Takeaway

So, what does this all mean? The paper suggests that using an InfoVAE is a powerful way to simplify complex fluid flows. It's better than the old "straight line" methods because it can handle the messy, twisting, chaotic parts of fluid motion. It creates a small, efficient summary of the flow that keeps the important physics intact.

While it's not a magic wand that solves every fluid problem yet (especially for 3D chaos at high speeds), it's a very strong step forward. The researchers showed that by combining this smart AI compressor with a predictor, we can get accurate, fast models of fluid behavior. It's like finally having a director who can summarize a chaotic mosh pit into a few key dance moves, allowing us to predict the next move without needing to watch every single person in the crowd. The future, they suggest, involves taking this idea and applying it to even more complex, three-dimensional, and high-speed fluid problems.

Technical Summary: Learning Nonlinear Reduced-Order Models of Fluid Flows with InfoVAE

Problem Statement
Turbulent fluid flows are characterized by multiscale, nonlinear, and chaotic dynamics, making them computationally expensive to simulate using high-fidelity methods like Direct Numerical Simulation (DNS). While Reduced-Order Models (ROMs) aim to capture essential flow dynamics with lower dimensionality, traditional linear projection methods such as Proper Orthogonal Decomposition (POD) and Dynamic Mode Decomposition (DMD) often struggle to represent strongly nonlinear and advection-dominated flows. Furthermore, recent data-driven approaches using Variational Autoencoders (VAEs) face significant challenges, specifically "posterior collapse," where the latent variables fail to retain information about the input, and a difficult trade-off between reconstruction accuracy and latent regularization.

Methodology
The authors propose a framework combining an Information Maximizing Variational Autoencoder (InfoVAE) with an Easy-Attention-based Transformer to develop a nonlinear ROM for fluid flow prediction.

InfoVAE Architecture:
- Unlike standard VAEs that maximize the Evidence Lower Bound (ELBO), InfoVAE utilizes a more flexible objective function to explicitly control the mutual information between input data and latent variables.
- The objective function is defined as: $L_{InfoVAE} = L_{rec} + \alpha L_{KL} + \lambda L_{MMD}$ , where $L_{rec}$ is the reconstruction loss (Mean Squared Error), $L_{KL}$ is the Kullback–Leibler divergence, and $L_{MMD}$ is the Maximum Mean Discrepancy.
- The inclusion of the MMD term and the hyperparameter $\alpha$ (set to 0.01) helps mitigate posterior collapse and ensures the learned latent space remains informative and disentangled.
- The architecture consists of a convolutional encoder (6 layers) and a transposed convolutional decoder (6 layers) with ELU activation functions.
- Latent dimensions were selected based on flow complexity: $d=2$ for periodic flow ($Re=40$) and $d=6$ for chaotic flow ($Re=100$).
Temporal Prediction:
- The latent representations generated by the InfoVAE are fed into an Easy-Attention Transformer.
- This module predicts the temporal evolution of the latent variables ( $r(t+1)$ ) using a sequence-to-sequence approach with a mean-squared-error loss.
- The Transformer employs four encoder blocks with four attention heads, residual connections, and time-based embeddings.
Datasets:
- The framework was evaluated on two-dimensional viscous flow datasets over two collinear plates, available via Zenodo.
- Case 1 (Periodic): Reynolds number $Re=40$, 90° plate angle, 1,001 snapshots.
- Case 2 (Chaotic): Reynolds number $Re=100$, 80° plate angle, 150,000 snapshots.

Key Contributions

Novel Framework: The integration of InfoVAE with an Easy-Attention Transformer for fluid flow reduced-order modeling.
Comparative Analysis: A direct comparison between the proposed InfoVAE approach and classical POD regarding reconstruction accuracy, latent dimension sensitivity, and energy capture.
Nonlinear Representation: Investigation into the ability of InfoVAE to learn latent representations that capture complex flow structures beyond the linear subspaces of POD.
Disentangled Latent Space: Demonstration that InfoVAE can produce interpretable latent features consistent with dominant flow modes.

Results

Periodic Flow ($Re=40$):
- The InfoVAE achieved a reconstruction energy ( $E$ ) of 94% with a latent dimension of $d=2$ , compared to 98.4% for POD.
- The learned spatial modes exhibited the same shedding flow patterns as the first two POD modes.
- Frequency spectrum analysis of the temporal coefficients confirmed that both methods captured the same characteristic frequency ( $f_c = U_\infty/c$ ).
- The correlation matrix of the latent variables indicated a high degree of independence, suggesting effective disentanglement.
- The authors noted that increasing the latent dimension beyond 2 resulted in negligible amplitudes for extra modes, confirming the regularization's effectiveness in avoiding artificial mode creation.
Chaotic Flow ($Re=100$):
- The framework successfully learned a compact latent representation for the more complex, asymmetric flow.
- Training monitoring showed steady convergence of the reconstruction loss (MSE) and latent regularization (KL divergence), indicating a balance between accuracy and regularization that prevents posterior collapse.
- The model demonstrated satisfactory generalization on validation data.
- Limitation: Due to HPC node failures interrupting long training runs, comprehensive quantitative metrics (such as full reconstruction energy and correlation matrices) for the $Re=100$ case were not fully computed at the time of publication, though qualitative results were positive.

Significance and Claims
The paper claims to provide a proof of concept for the effectiveness of modern machine learning techniques in developing compact, robust nonlinear ROMs for complex fluid flows. The authors assert that:

The InfoVAE-based approach successfully captures essential flow dynamics and coherent structures, even in highly nonlinear and chaotic regimes where linear methods like POD may falter.
The extracted latent features behave similarly to dominant POD modes, validating the physical interpretability of the nonlinear reduced-order model.
Coupling the learned latent space with an Easy-Attention Transformer enables accurate long-term temporal prediction of reduced dynamics.
The framework offers a viable alternative for spatio-temporal flow estimation, though the current study is limited to 2D numerical datasets and fixed Reynolds numbers.

The authors conclude that while the approach shows promise, future work is required to extend the framework to three-dimensional turbulent flows and higher Reynolds number regimes.

The Story of the Flow

How They Did It

What They Found

The Takeaway

Technical Summary: Learning Nonlinear Reduced-Order Models of Fluid Flows with InfoVAE

More like this