A posteriori closure of turbulence models: are symmetries preserved?

This paper evaluates an a posteriori turbulence closure for a shell model that integrates physical equations into a neural network, finding that while it successfully reproduces high-order statistical moments, it fails to preserve scale invariance symmetries near the cutoff, revealing a fundamental limitation for subgrid-scale modeling.

The Gist

Imagine you are trying to predict the weather for the next month. You have a supercomputer, but even with all its power, it can't track every single molecule of air, every tiny swirl of wind, and every drop of rain. There are simply too many details.

So, scientists use a trick called Large Eddy Simulation (LES). Think of it like watching a storm from a helicopter. You can clearly see the massive, swirling clouds (the "large eddies"), but you can't see the tiny, chaotic gusts of wind right under the helicopter blades (the "subgrid scales"). To make the simulation work, you have to guess what those tiny gusts are doing based on the big clouds you can see. This guessing game is called closure.

For decades, scientists have used rough rules of thumb to make these guesses. But recently, they started using Artificial Intelligence (AI) to learn the rules directly from data.

The Experiment: A Simplified Universe

In this paper, the researchers didn't try to simulate the entire atmosphere (which is too hard). Instead, they used a "toy universe" called a Shell Model.

• The Analogy: Imagine a set of Russian nesting dolls, but instead of being inside each other, they are arranged in a line. Each doll represents a different size of wind swirl. The biggest doll is the biggest storm; the smallest doll is the tiniest breeze.
• The Problem: The AI is only allowed to see the big dolls. It has to guess what the tiny, hidden dolls are doing so the whole line of dolls behaves realistically.

The New Approach: "Learning by Doing"

Most AI models are trained like a student taking a pop quiz. They are shown a picture of the big dolls and asked, "What are the small dolls doing right now?" If they get it wrong, they are corrected immediately. This is called an a priori approach.

The researchers in this paper tried something different, which they call Solver-in-the-Loop (or A Posteriori).

• The Analogy: Instead of a pop quiz, imagine the AI is the pilot of a drone flying through a storm. It doesn't just guess the wind; it flies the drone. If it guesses the wind wrong, the drone drifts. The AI learns not just to be right for one second, but to keep the drone flying smoothly for a long time, even if it makes small mistakes along the way. It learns how its own errors ripple through time.

The Results: A Tale of Two Successes

The researchers tested their AI pilot and found it was incredibly good at some things, but had a specific blind spot.

1. The Good News (The Big Picture):
The AI was fantastic at predicting the average behavior of the storm. It could tell you how much energy the big clouds had, how fast they were spinning, and the general "mood" of the turbulence. If you looked at the statistics of the big dolls, the AI's simulation looked almost identical to the real thing.

2. The Bad News (The Hidden Symmetry):
Here is where the paper gets interesting. In turbulence, there is a hidden "rule of symmetry." It's like a secret handshake between the big dolls and the tiny dolls. No matter how you zoom in or out, the relationship between the sizes of the swirls should look the same (this is called scale invariance).

• The Discovery: The researchers found that while the AI was great at the big picture, it broke this secret handshake near the cutoff point (where the big dolls end and the tiny hidden ones begin).
• The Metaphor: Imagine a dance floor where everyone is dancing in a perfect, repeating pattern. The AI kept the big dancers moving perfectly. But right at the edge of the stage, where the big dancers hand off the rhythm to the invisible tiny dancers, the AI got the rhythm slightly wrong. The tiny dancers started stumbling, and the perfect pattern broke.

Why Did This Happen?

The authors suggest the problem isn't that the AI wasn't "smart" enough or that they didn't give it enough data. The problem is structural.

• The Memory Problem: The AI they used is Markovian. In plain English, this means it has no memory. It only looks at the current state of the big dolls to guess the tiny ones. It doesn't remember how the big dolls got there.
• The Real World: In real turbulence, the tiny swirls don't just react to the current big swirl; they react to the history of the big swirls. It's like trying to predict a car's speed by only looking at the gas pedal right now, without remembering if the driver just slammed on the brakes five seconds ago.
• The Solution: To fix this, the AI needs to be given a "memory" (mathematically, a memory kernel) so it can remember the past interactions between the big and small scales.

The Takeaway

This paper is a success story with a warning label.

• Success: We can now build AI that simulates turbulence very accurately for the big, visible parts of the flow.
• Warning: If we want to simulate the extreme events (the rare, wild fluctuations) or the exact interaction between the visible and invisible parts, a simple "guess the next step" AI isn't enough. We need an AI that understands the history of the flow, not just the present moment.

It's like saying, "We built a car that drives perfectly on a straight highway, but it struggles to navigate a sharp turn because it forgot where it came from." The next step is to give the car a memory so it can handle the whole journey.

Technical Summary

1. Problem Statement

Turbulence modeling, particularly for Large Eddy Simulation (LES), faces the challenge of accurately representing the effects of unresolved subgrid scales (SGS) on resolved scales. Traditional phenomenological models (e.g., eddy diffusivity) often fail to capture complex statistics, particularly intermittency and high-order moments. While Deep Learning (DL) offers a data-driven alternative, most existing approaches rely on a priori training (matching instantaneous ground truth), which suffers from distribution shift when deployed in time-evolving simulations. Furthermore, it remains unclear whether DL-based closures can preserve fundamental physical symmetries, such as the scale invariance of Kolmogorov multipliers, which are critical for capturing the correct multifractal nature of turbulence.

2. Methodology

The authors employ Shell Models (specifically the Sabra model) as a computationally tractable proxy for 3D homogeneous isotropic turbulence. These models preserve key features like energy cascades, intermittency, and anomalous scaling while reducing the degrees of freedom.

• Solver-in-the-Loop (A Posteriori) Approach:
  • Unlike standard a priori methods, the neural network (NN) is embedded directly within the time-stepping solver.
  • The system is "unrolled" over $m$ time steps during training. The NN predicts the two missing shells ($u_{N_c+1}, u_{N_c+2}$) based on the three preceding resolved shells.
  • The loss function minimizes the deviation between the LES trajectory and the fully resolved ground truth trajectory over a multi-step horizon, rather than matching instantaneous terms. This forces the network to learn how its errors propagate and to mitigate covariate shift (the mismatch between training and deployment input distributions).
• Network Architecture:
  • A fully connected Multi-Layer Perceptron (MLP) with 7 hidden layers (256 neurons each) and ReLU activation.
  • Input: Resolved shells $(u_{N_c-2}, u_{N_c-1}, u_{N_c})$.
  • Output: Predicted subgrid shells $(u_{N_c+1}, u_{N_c+2})$.
• Training Strategy:
  • Optimized using Adam with a learning rate of $10^{-4}$.
  • Tested on initial conditions and time horizons significantly longer than those in the training set to assess robustness.

3. Key Contributions

1. Rigorous Evaluation of A Posteriori Closures: The paper moves beyond standard validation (e.g., energy spectra) to test whether learned closures preserve non-trivial correlations between resolved and unresolved variables.
2. Symmetry Analysis: The authors investigate the preservation of Kolmogorov multipliers (ratios of velocity increments between adjacent shells). These multipliers are known to exhibit universal, scale-invariant statistics in the inertial range.
3. Identification of Structural Limitations: The study reveals that while the model excels at reproducing resolved-scale statistics, it fundamentally fails to preserve scale invariance near the cutoff, pointing to a limitation of Markovian closures.

4. Key Results

• Resolved-Scale Statistics: The LES-NN model demonstrates high accuracy in reproducing structure functions up to order $p=6$ and the temporal evolution of energy, matching the ground truth closely even near the cutoff scale.
• Joint Probability Density Functions (PDFs):
  • The model captures the general shape of the joint PDF between resolved input energy and unresolved output energy.
  • Failure Point: The model underestimates the tails of the distribution, indicating a failure to capture the strength of cross-scale correlations and rare extreme events.
• Kolmogorov Multipliers (Scale Invariance):
  • Ground Truth: Amplitude and phase multipliers collapse onto universal curves across the inertial range, confirming scale invariance.
  • LES-NN Model: The learned closure breaks this symmetry near the cutoff. The multipliers no longer exhibit scale invariance, and the joint PDF of multiplier products shows significant deviations in both the mean and tails.
• Correlation Functions: Using an analogy to the 2D XY model (spin correlations), the authors show that while the model captures long-range decay, it misestimates correlations for shells close to the cutoff, indicating a loss of phase and amplitude coherence in that region.

5. Significance and Discussion

• The "Magnifying Glass" Effect: The analysis of multipliers and joint PDFs acts as a sensitive diagnostic tool. It reveals systematic errors in the closure that are invisible when looking only at resolved-scale statistics (like energy spectra).
• Root Cause Analysis: The authors argue that the failure is not due to insufficient network capacity or loss function design. Instead, it stems from the Markovian nature of the closure.
  • According to the Mori-Zwanzig (MZ) formalism, the exact evolution of resolved variables requires a memory kernel (delayed interactions) and an orthogonal dynamics term (noise from unresolved-unresolved interactions).
  • The current Markovian NN closure omits these non-local-in-time effects, leading to the breakdown of scale invariance and the inability to fully reconstruct resolved-unresolved correlations.
• Future Directions:
  • Incorporating non-Markovian effects (learning memory kernels or orthogonal dynamics) via data-driven MZ approaches.
  • Enforcing physical symmetries (like multiplier scale invariance) as hard constraints or penalties in the loss function.
  • Reformulating the problem directly in terms of multipliers (hidden-symmetry framework).

Conclusion

The paper demonstrates that while solver-in-the-loop deep learning provides stable and accurate closures for resolved-scale statistics in turbulence shell models, it currently fails to preserve fundamental symmetries (scale invariance of multipliers) near the cutoff. This failure is attributed to the inherent lack of memory in Markovian closures. The work suggests that future robust subgrid models must integrate physical symmetries and non-Markovian dynamics to accurately capture the multifractal nature of turbulence.