🤖 machine learning

The finite expression method for turbulent dynamics with high-order moment recovery

This paper proposes a two-stage data-driven framework that combines the Finite Expression Method for discovering deterministic dynamics with generative models for capturing residual stochasticity, enabling accurate recovery of governing equations and high-order statistical moments in turbulent systems.

Original authors: Xingjian Xu, Di Qi, Chunmei Wang

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Xingjian Xu, Di Qi, Chunmei Wang

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the chaotic dance of a storm. The wind swirls, rain lashes, and energy moves in unpredictable ways. Scientists call this turbulence. Trying to write a single equation that perfectly predicts every gust of wind is nearly impossible because the system is full of random "noise" and complex interactions.

This paper proposes a clever two-step strategy to solve this puzzle, like a detective teaming up with a psychic to solve a mystery.

The Problem: The "Noisy" Storm

Turbulent systems (like weather or ocean currents) have two main parts:

The Deterministic Part: The predictable rules, like gravity or how water swirls around a rock.
The Stochastic Part: The random chaos, like sudden gusts of wind or unpredictable temperature shifts.

Old methods tried to guess the whole thing at once, but they often got lost in the noise or missed the complex rules. They were like trying to hear a whisper in a rock concert.

The Solution: A Two-Stage Detective Team

The authors created a framework called FEX (Finite Expression Method) combined with Generative Models. Think of it as a two-stage investigation.

Stage 1: The "Rule Finder" (FEX)

First, the team uses a method called FEX to find the "rules of the game."

How it works: Imagine you have a giant box of Lego bricks (mathematical operators like plus, minus, multiply, sine, etc.). FEX is a smart robot that tries to snap these bricks together in different ways to build a structure that matches the data.
The Goal: It ignores the random noise for a moment and focuses on finding the clean, closed-form equation that explains the main movements. It's like looking at a chaotic crowd and realizing, "Ah, everyone is actually following a specific dance pattern, even if they are bumping into each other."
The Result: FEX successfully discovers the exact mathematical formulas for how the system interacts, including how energy is conserved (like a bank account where money is just moved between accounts, never created or destroyed). It does this without needing a pre-written list of possible formulas; it builds them from scratch.

Stage 2: The "Chaos Tamer" (Generative Models)

Once the Rule Finder has done its job, there is still a problem: the real world is messy. The Rule Finder's equation is great, but it's not perfect. There are tiny errors and random fluctuations it missed.

The Goal: This is where the second team steps in. They use Generative Models (like a "Chaos Tamer") to study the leftover mistakes.
How it works: They look at the difference between what the Rule Finder predicted and what actually happened. They ask, "What does this leftover randomness look like?" Is it a small jitter? A big spike?
The Result: The Generative Model learns the "personality" of this randomness. It doesn't just guess a number; it learns the shape of the chaos. When predicting the future, it adds this learned chaos back into the Rule Finder's prediction.

Why This Matters

By splitting the job, the team gets the best of both worlds:

Interpretability: We know why the system moves because the first stage gives us a clear, readable equation (like $F = ma$).
Accuracy: We capture the complex, high-order statistics (the "deep" details of the chaos) that simple averages miss.

The Proof: The "Triad" Test

To prove this works, the authors tested it on a simplified model called a Stochastic Triad.

The Analogy: Imagine three dancers holding hands. If one spins, it pulls the others. Sometimes they push, sometimes they pull. The authors tested this system under five different "moods":
- Equipartition: Everyone dancing evenly.
- Forward Cascade: Energy flowing from the leader to the followers.
- Dual Cascade: Energy flowing back and forth.
- Periodic/Random: The dancers are being pushed by a rhythmic beat or a random, unpredictable hand.

The Results:

The "Rule Finder" (FEX) correctly identified the dance steps (the equations) in all five moods, even when the music was very loud and chaotic.
The "Chaos Tamer" (Generative Models) successfully recreated the complex, high-level statistics of the dance, predicting not just the average movement, but the rare, wild swings that happen in turbulent systems.

The Bottom Line

This paper shows that you don't have to choose between a simple, understandable model and a complex, accurate one. By using a "Rule Finder" to get the physics right and a "Chaos Tamer" to handle the randomness, you can accurately predict the behavior of turbulent systems, capturing details up to the fifth level of complexity (moments), which previous methods struggled to do.

Technical Summary: The Finite Expression Method for Turbulent Dynamics with High-Order Moment Recovery

Problem Statement
Turbulent dynamical systems are characterized by strong nonlinear interactions, intrinsic instabilities, and stochastic effects that generate coupled statistical quantities, particularly non-zero higher-order moments. Modeling these systems is challenging due to imperfect model errors, unresolved scales, and limited observational data. While recent data-driven methods have shown success in learning local or short-term dynamics, they often rely on low-order statistical representations (e.g., mean and variance) which are insufficient to capture the essential behaviors of turbulent dynamics. Furthermore, existing symbolic discovery methods like Sparse Identification of Nonlinear Dynamical Systems (SINDy) and its variants rely on predefined function libraries. This limitation restricts their ability to represent complex forcing expressions that fall outside the chosen basis and often leads to violations of underlying conservation laws (e.g., energy-preserving exchanges) when estimating coefficients in highly stochastic environments. The core problem addressed is the accurate recovery of both the deterministic governing dynamics (including nonlinear interactions and external forcing) and the stochastic components necessary to reproduce high-order statistical moments up to order five.

Methodology
The authors propose a two-stage data-driven modeling framework that integrates symbolic regression with generative models to jointly identify governing dynamics and predict key statistical quantities.

Stage I: Deterministic Dynamics Discovery via Finite Expression Method (FEX)
The first stage employs the Finite Expression Method (FEX) to discover closed-form expressions for the deterministic component of the dynamics, denoted as $C(u, t)$ . Unlike library-based methods, FEX represents candidate expressions as binary trees constructed from a finite set of elementary unary and binary operators (e.g., $\sin, \exp, +, \times$ ). This allows for hierarchical compositions that capture nested functional forms beyond the linear span of predefined basis functions.
- Search Mechanism: The method utilizes a mixed discrete–continuous optimization framework. A neural network controller generates candidate operator sequences (discrete search), while a gradient-based optimizer refines the continuous parameters (scaling and bias) for a fixed sequence.
- Objective: The goal is to minimize the empirical loss between the predicted time increments and the observed data, effectively recovering nonlinear interaction terms and external forcing without predefined libraries.
- Component-wise Learning: To improve scalability, the framework learns the symbolic representation for each state component separately before assembling the full vector-valued model.
Stage II: Stochastic Error Correction via Generative Models
Since FEX models only the deterministic component, a second stage is introduced to model the residual stochastic components, which include unmodeled stochastic increments and model errors arising from coefficient estimation.
- Residual Definition: The residual $R_n$ is defined as the difference between the observed state evolution and the prediction from the Stage I FEX model: $R_n = u(t_{n+1}) - u(t_n) - h C^*(u(t_n))$ .
- Generative Models: Three representative generative models are employed to learn the distribution of these residuals:
  1. Training-Free Diffusion Model (TFDM): Formulates the stochastic process via forward and reverse stochastic differential equations (SDEs), approximating the score function numerically without explicit training of a score network in the traditional sense, though a network is trained to map Gaussian noise to the residual distribution.
  2. Stochastic Residual Activation Network (SRAN): A lightweight network mapping Gaussian noise to stochastic residuals, regularized to match the second-order statistics (covariance) of the residuals.
  3. Variational Autoencoder (VAE): Uses a latent-variable framework with probabilistic encoding and decoding to capture stochastic structure.
- Inference: During inference, the learned symbolic dynamics (Stage I) and the generative model (Stage II) are combined to produce full estimated trajectories, enabling the faithful recovery of high-order moments involving highly non-Gaussian distributions.

Key Contributions

Accurate Symbolic Discovery with Limited Data: The framework demonstrates uniformly high skill in recovering correct deterministic expression formulas and identifying nonlinear interaction coefficients across regimes ranging from deterministic to strongly stochastic. The learned coefficients respect intrinsic physical constraints, such as energy-conserving transfers, through the FEX fine-tuning process.
High-Order Moment Recovery: By decoupling deterministic discovery from stochastic modeling, the framework successfully predicts statistical moments up to order five, capturing essential turbulent behaviors that low-order methods miss.
Out-of-Training-Domain Generalization: Under constant noise assumptions, the Stage II generative models learn the distribution of residuals, enabling trajectory resampling beyond the training regime and improving long-term estimation of high-order moments.
Theoretical Consistency: The paper provides theoretical analysis establishing the consistency of the symbolic estimator and quantifying estimation errors in terms of data size and numerical discretization, relying on ergodicity assumptions and Itô–Taylor expansions.

Numerical Results
The framework was evaluated on the stochastic triad model, a three-dimensional system representing a Galerkin truncation of nonlinear systems, across five distinct regimes:

Regimes Tested: Equipartition (near-Gaussian), Forward Energy Cascade, Dual Energy Cascade, Periodic Forcing, and Random Forcing.
Symbolic Discovery: FEX accurately recovered governing equations, including nonlinear interaction terms and complex forcing expressions (e.g., sinusoidal and exponential forms), across all regimes. In contrast, Weak SINDy often approximated forcing terms polynomially and failed to identify interaction terms correctly under strong stochastic influence.
Noise Tolerance: FEX maintained accurate coefficient recovery (errors within $10^{-6}$ to $10^{-2}$ ) even as noise levels increased from deterministic to strongly stochastic regimes.
Data Efficiency: Increasing the sample size improved coefficient estimation, with significant gains observed between 1,000 and 4,000 samples, though diminishing returns occurred beyond this threshold due to inherent stochastic variability.
Moment Prediction: All three generative models (TFDM, SRAN, VAE) captured qualitative behaviors. SRAN showed the closest agreement with ground truth for lower-order moments, while TFDM showed only minor deviations. The framework successfully predicted fourth- and fifth-order moments, though accuracy decreased in the random cascade regime where strong time-dependent stochastic forcing made recovery more challenging.
Time-Dependence: A comparison between time-independent and time-dependent modeling strategies for TFDM revealed a trade-off: time-dependent modeling captured short-term oscillatory covariance better but became unstable for long-term horizons ( $T > 10$ ), whereas time-independent modeling offered smoother, more stable long-term trajectories at the cost of systematic bias.

Significance and Claims
The paper claims that integrating interpretable symbolic discovery (FEX) with data-driven stochastic modeling (generative models) offers a robust solution for complex turbulent systems. The significance lies in the ability to:

Recover closed-form expressions for nonlinear interactions and external forcing without predefined libraries, preserving physical conservation laws.
Accurately characterize high-order statistical quantities (up to order five) that are critical for understanding turbulent behaviors but difficult to capture with standard data-driven methods.
Provide a consistent theoretical framework where the symbolic estimator is proven to be consistent under ergodicity assumptions.
Demonstrate that this two-stage approach remains effective even in challenging settings with multiple sources of stochasticity, such as random cascade regimes, highlighting its potential applicability to complex stochastic systems where traditional methods fail to capture essential high-order dynamics.

The authors note that while the framework is effective, the computational cost of TFDM scales with the number of time steps and data volume, and future work is needed to develop more scalable implementations and extend the method to multiscale coupled stochastic effects and stochastic PDEs.