Comparison of data-driven symmetry-preserving closure models for large-eddy simulation

This paper demonstrates that while unconstrained and symmetry-preserving data-driven neural networks both outperform classical large-eddy simulation closures in accuracy, enforcing physical symmetries is crucial for generating more physically consistent velocity-gradient statistics and improving the overall quality of the learned closure.

The Gist

Imagine you are trying to predict the weather, but you don't have a supercomputer powerful enough to track every single raindrop, gust of wind, and swirl of air. That's the problem scientists face with turbulence (chaotic fluid flow like air over a wing or smoke from a chimney).

To solve this, they use a technique called Large-Eddy Simulation (LES). Think of it like looking at a storm through a foggy window. You can see the big, swirling clouds (the "large eddies"), but the tiny, chaotic ripples are blurred out. The problem is, those tiny ripples still affect the big clouds. If you ignore them, your simulation eventually falls apart.

To fix this, scientists use a "closure model"—a mathematical guess that estimates what those tiny, invisible ripples are doing so the big picture stays accurate.

The Old Way vs. The New Way

For decades, scientists used simple, hand-crafted formulas (like the Smagorinsky or Clark models) to make these guesses. They work okay, but they are like using a blunt knife to cut a steak; they get the job done but lack precision.

Recently, scientists started using Artificial Intelligence (Neural Networks) to learn these guesses from data. It's like teaching a robot to watch a million storms and learn exactly how the tiny ripples behave. This is much more accurate, but it has a big flaw: The robot doesn't know the rules of physics.

If you rotate the storm 90 degrees, the physics should look the same. But a standard AI might get confused and give a completely different answer just because the picture turned sideways. This breaks the laws of physics and makes the simulation unstable.

The Three "Robots" in This Paper

The authors of this paper wanted to see if they could teach AI to respect the symmetries of physics (the idea that the laws of nature don't change if you rotate, flip, or shift the view). They built and compared three different types of AI "robots":

1. The "Unconstrained" Robot (Conv):

   • Analogy: A student who is told to memorize the answers but isn't told the rules of the game.
   • How it works: It's a standard neural network. It's very flexible and learns fast, but it doesn't inherently know that "up" is the same as "down" if you flip the world.
   • Result: It was accurate at predicting the numbers, but it sometimes gave physically weird results (like a storm behaving differently just because you rotated the map).

2. The "Tensor-Basis" Robot (TBNN):

   • Analogy: A student who is given a set of Lego bricks that are pre-shaped to fit the laws of physics. They can only build things that are structurally sound.
   • How it works: Instead of guessing the whole storm, this AI guesses the coefficients (the numbers) for a pre-defined set of "physics-safe" building blocks. No matter how the AI learns, the final result is guaranteed to obey the rules of rotation and reflection.
   • Result: It was very stable and physically consistent, though it sometimes missed the "wild" details of the turbulence.

3. The "Group-Convolution" Robot (G-conv):

   • Analogy: A student who is forced to wear a magic suit that automatically translates their thoughts into the correct physics language. If they think "left," the suit automatically knows that "left" means "right" if the world is flipped.
   • How it works: This is a complex neural network architecture where the math is built into the very structure of the network. Every single calculation is forced to respect the symmetry groups (rotations and flips).
   • Result: Like the Lego robot, it was perfectly physically consistent. However, it was computationally heavy (slow) because the "magic suit" was very bulky.

What Did They Find?

The researchers ran these robots against a "gold standard" simulation (Direct Numerical Simulation) to see who performed best.

• Accuracy: All three AI robots were better than the old, hand-crafted formulas. They predicted the stress of the tiny ripples much more accurately.
• Stability: The "Unconstrained" robot was fast and accurate, but it was a bit "sloppy" with the laws of physics. The two "Symmetry-Preserving" robots (Lego and Magic Suit) produced results that looked much more like real physics, especially when looking at the complex shapes of the swirling air.
• The "Teardrop" Test: The authors used a specific test involving the shape of swirling air (called the velocity gradient distribution). The symmetry-preserving models recreated the famous "teardrop" shape of turbulence perfectly. The unconstrained model got the shape slightly wrong, even though its numbers looked okay.
  • Metaphor: Imagine trying to draw a perfect circle. The unconstrained robot drew a circle that was mathematically close to the right size but looked a bit squashed. The symmetry-preserving robots drew a perfect circle every time.

The Big Takeaway

The paper concludes that baking the laws of physics directly into the AI's brain is worth it.

Even though the "Unconstrained" robot was fast and got the numbers right, the "Symmetry-Preserving" robots produced a simulation that felt more "real" and consistent. It's the difference between a GPS that gets you to the destination quickly but takes you through a wall, versus a GPS that respects traffic laws and road geometry, ensuring a safe and logical journey.

In short: If you want an AI to simulate nature, don't just let it learn from data; teach it the rules of the game first. The "Lego" approach (Tensor-Basis) seemed to offer the best balance of speed and physical correctness, while the "Magic Suit" (Group-Conv) was the most rigorous but slowest.

Technical Summary

Here is a detailed technical summary of the paper "Comparison of data-driven symmetry-preserving closure models for large-eddy simulation" by Agdestein and Sanderse.

1. Problem Statement

Large-Eddy Simulation (LES) is a computationally tractable alternative to Direct Numerical Simulation (DNS) for turbulent flows, but it requires a closure model to represent the effects of unresolved sub-filter scales (SFS).

• The Challenge: Classical closure models (e.g., Smagorinsky, Clark's gradient) often lack accuracy or stability. Recently, data-driven neural networks (NNs) have shown promise in improving accuracy but suffer from instability and poor generalizability when deployed in simulations.
• The Root Cause: A primary source of instability is the violation of fundamental physical symmetries (Galilean invariance, rotational invariance, reflectional invariance, and scaling invariance) inherent in the Navier-Stokes equations. Standard neural networks do not automatically enforce these symmetries, potentially introducing spurious forces or unphysical behavior.
• The Goal: To compare different strategies for embedding these symmetries into data-driven LES closures to determine if enforcing symmetry improves physical consistency and simulation stability compared to unconstrained networks.

2. Methodology

The authors compare three data-driven closure architectures against classical models (Smagorinsky and Clark's gradient) using forced homogeneous isotropic turbulence. All models are trained on data generated from a high-resolution DNS ($N=810$) filtered to a coarser LES grid ($N=128$).

A. Symmetry Constraints

The study focuses on preserving the symmetries of the incompressible Navier-Stokes equations:

1. Galilean Invariance: Achieved by using the Velocity Gradient Tensor (VGT, $\bar{A}$) as input rather than velocity ($\bar{u}$).
2. Scaling Invariance: Achieved by factoring out $\Delta^2 |\bar{A}|^2$ from the output, ensuring dimensional consistency.
3. Rotational and Reflectional Equivariance: This is the core variable being tested. The models are required to be equivariant under the octahedral group ($G$), which includes 48 discrete rotations and reflections (multiples of $\pi/2$) compatible with a Cartesian grid.

B. Model Architectures

1. Tensor-Basis Neural Network (TBNN):

   • Approach: Uses Pope's tensor basis theory (refined by Stallcup et al.). The subgrid stress is expressed as a linear combination of basis tensors ($T^{(k)}$) constructed from the VGT, weighted by scalar coefficients predicted by a neural network.
   • Symmetry: Symmetry is enforced structurally by the choice of basis tensors and invariants. The network only predicts scalar coefficients based on invariants of the VGT, guaranteeing equivariance regardless of the network's internal architecture.
   • Input: 5 invariants of the normalized VGT.
   • Output: 7 scalar coefficients.

2. Group-Convolutional Neural Network (G-conv):

   • Approach: Uses group-equivariant convolutions. The hidden layers operate in the regular representation of the octahedral group ($\mathbb{R}^{48}$), where each channel corresponds to a group element.
   • Symmetry: Enforced via weight projection. The weights are constrained to commute with the group action matrices (permutation matrices). The authors introduce a novel eigendecomposition-based weight sharing technique to compress the projection operator, reducing the number of learnable parameters while maintaining equivariance.
   • Input/Output: Transforms physical tensors into the regular representation and back.

3. Unconstrained Convolutional Neural Network (Conv):

   • Approach: A standard feed-forward CNN (implemented with $1\times1\times1$ kernels for point-wise operation) that takes the VGT as input and predicts the stress tensor directly.
   • Symmetry: No explicit symmetry enforcement. It relies on the training data (isotropic turbulence) to implicitly learn the symmetries.

C. Evaluation Metrics

• A-priori: Prediction error of the stress tensor against the reference DNS data.
• A-posteriori: Stability and accuracy of the LES simulation over time.
• Equivariance Error: Quantifies how much the model output changes when the input is rotated/reflected (should be near machine precision for symmetric models).
• Physical Statistics: Energy spectra, dissipation rates, and the joint distribution of velocity gradient invariants ($q$ and $r$), which characterize flow topology (vorticity vs. strain).

3. Key Contributions

1. Systematic Comparison: A rigorous side-by-side comparison of TBNN, G-conv, and unconstrained CNNs for LES, isolating the impact of symmetry enforcement.
2. Novel Weight Projection: Introduction of an eigendecomposition-based method to compress the weight projection operator for octahedral group convolutions, making G-conv computationally feasible without sacrificing the strict equivariance constraint.
3. Discrete SFS Definition: Use of a discretization-consistent sub-filter stress definition that accounts for the specific numerical scheme (pseudo-spectral with dealiasing), ensuring the training targets are physically consistent with the LES solver.
4. Insight on Symmetry vs. Accuracy: Demonstration that while unconstrained networks can achieve similar prediction errors (MSE), they fail to capture physical consistency in higher-order statistics.

4. Key Results

A. Prediction Accuracy (A-priori)

• Tensor Prediction: All three data-driven models significantly outperformed classical Smagorinsky and Clark models in predicting the stress tensor.
• Ranking: G-conv and Conv achieved the lowest tensor prediction errors, followed closely by TBNN.
• Equivariance:
  • TBNN and G-conv: Achieved equivariance errors at machine precision ($\sim 10^{-15}$), confirming strict adherence to symmetry.
  • Conv: Exhibited a significant a-priori equivariance error of 5.8%, indicating it failed to learn rotational/reflectional symmetry despite training on isotropic data.

B. Simulation Stability and Accuracy (A-posteriori)

• Stability: The classical Clark model diverged at $t \approx 2.1$. All neural network models remained stable.
• Solution Error: All three data-driven models produced nearly identical LES solution errors, which were lower than the classical models.
• Surprising Finding: Despite the Conv model's lack of equivariance, its trajectory error was comparable to the symmetric models. This suggests that for isotropic turbulence, strict symmetry is not strictly necessary for trajectory accuracy, but it is crucial for physical consistency.

C. Physical Consistency (The Critical Differentiator)

• Energy Spectra:
  • TBNN: Showed spurious energy pile-up at high wavenumbers (insufficient dissipation), similar to the "No-model" case.
  • Conv & G-conv: Matched the reference spectrum well but were slightly overly dissipative.
• Velocity Gradient Statistics ($q-r$ plane):
  • The joint distribution of invariants $q$ (strain) and $r$ (rotation) is a stringent test of turbulence physics.
  • Conv: Deviated significantly from the reference "teardrop" shape, particularly in the tails.
  • Symmetry-Preserving Models (TBNN, G-conv): Produced distributions much closer to the reference, capturing the correct balance between vorticity-dominated and strain-dominated regions.
  • Conclusion: Enforcing symmetry improves the physical consistency of the learned closure, even if aggregate error metrics (like MSE) do not show a difference.

D. Computational Cost

• G-conv: Approximately 7x slower than Conv during inference due to the expanded 48-channel representation, despite having a similar number of trainable parameters.
• TBNN: Inference cost comparable to Conv. It is the most efficient symmetry-preserving option because it encodes symmetry in the basis rather than the network architecture.

5. Significance and Conclusion

• Symmetry is Essential for Physics, Not Just Accuracy: The paper demonstrates that while unconstrained networks can mimic the "average" behavior of turbulence, they fail to reproduce the correct statistical topology (velocity gradient distributions) without symmetry constraints.
• TBNN as the Practical Choice: For LES applications requiring both symmetry preservation and computational efficiency, TBNN is superior to G-conv. It enforces symmetry through the input/output formulation (tensor basis) rather than expensive architectural constraints, offering similar accuracy to G-conv with significantly lower computational overhead.
• Future Directions: The authors note that current data-driven models lack Reynolds number invariance (they are trained at a single Re). Symmetry-preserving models may offer better generalization to unseen Reynolds numbers by reducing the function space. Additionally, extending these methods to anisotropic flows (where full rotational equivariance might be physically inappropriate) and incorporating spatial context (larger stencils) are identified as key next steps.

In summary, the paper establishes that enforcing symmetries in data-driven LES closures is critical for physical fidelity, and Tensor-Basis Neural Networks offer the most efficient and effective path forward for implementing these constraints.