🤖 machine learning

Gradient-free online learning of subgrid-scale dynamics with neural emulators

This paper proposes a gradient-free online learning framework that utilizes neural emulators to approximate non-differentiable solvers, enabling the effective training of subgrid-scale parametrizations for chaotic climate systems without requiring direct differentiation of the original numerical models.

Original authors: Hugo Frezat, Ronan Fablet, Guillaume Balarac, Julien Le Sommer

Published 2026-08-07

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Hugo Frezat, Ronan Fablet, Guillaume Balarac, Julien Le Sommer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict the weather or the ocean currents using a giant computer simulation. These simulations are like massive digital maps of the Earth, but they have a problem: they are too coarse to see the tiny, swirling details like a single raindrop or a small whirlpool. To fix this, scientists use "shortcuts" called parameterizations. Think of these as a chef's secret recipe for a sauce; the chef knows the sauce should taste a certain way, so they add a pinch of this and a dash of that without cooking every single ingredient from scratch. In climate models, these shortcuts represent the tiny, fast-moving physics that the computer can't see directly.

For a long time, scientists tried to teach computers to make these shortcuts better by showing them the "answer key" (what the tiny physics should look like) and asking the computer to guess. This is called "offline" learning. But it's like teaching a driver only by showing them pictures of a road; when they actually get behind the wheel, they crash because they didn't learn how the car feels while moving. The better way is "online" learning, where the computer learns while driving, adjusting its shortcuts in real-time based on how the simulation behaves. However, there's a catch: the real-world computer code used for weather is built like a black box made of old, rigid metal. You can't easily peek inside to see how a tiny change affects the whole system, which makes "online" learning impossible with standard tools.

This paper introduces a clever workaround: a "ghost driver" or a neural emulator. The authors built a separate, smooth, and flexible AI that mimics the behavior of the rigid, old computer code. This ghost driver is easy to peek inside and understand. They use this ghost to teach the shortcut recipe (the parameterization) how to drive in real-time. Once the recipe is learned using the ghost, they swap it back into the real, rigid car. The result is a shortcut that works almost as well as if it had been trained on the real car, but without needing to break the car open to do it.

The Problem: The Black Box and the Missing Link

Climate scientists are trying to build better models to predict our changing planet. These models are essentially giant math equations that simulate how air and water move. But computers can't calculate every single molecule of air; they have to group them into big chunks. This means they miss the tiny, chaotic swirls that happen between the chunks. To fill in these gaps, they use subgrid-scale (SGS) models—basically, educated guesses about what the tiny swirls are doing.

Traditionally, scientists trained these guesses using an "offline" approach. They would run a super-detailed simulation (the "truth"), chop it up into big chunks, and then train a machine learning model to guess the missing swirls based on the chunks. It's like trying to learn to play soccer by watching a highlight reel of goals but never actually running on the field. While this works okay for statistics, it often leads to unstable simulations that crash or produce wild, unrealistic weather when run for long periods.

The "online" approach is the gold standard. Here, the machine learning model learns while it is running the simulation. It sees how its own guesses affect the future of the weather and adjusts itself to keep the simulation stable. The problem? The computer code used for real climate models is often written in old languages (like Fortran) and is so complex that it's a "non-differentiable" black box. In plain English, you can't easily calculate how a tiny change in the model's settings ripples through the entire system to affect the final result. Without this ability to trace the ripples (gradients), standard online learning is impossible.

The Solution: The "Ghost Driver"

The authors, led by Hugo Frezat, proposed a creative solution: neural emulation. Instead of trying to force the old, rigid code to be flexible, they built a new, flexible AI "emulator" that learns to mimic the behavior of the old code.

Think of the real climate solver as a heavy, old-fashioned steam engine. It works great, but you can't easily see the gears turning inside to understand how to tweak it. The authors built a "ghost engine"—a neural network that acts exactly like the steam engine but is made of smooth, flexible plastic. Because this ghost engine is built with modern AI tools, you can see the gears and calculate exactly how to tweak it.

They used a two-step training process to make this work:

Step 1: Train the Ghost. First, they taught the ghost engine to mimic the real steam engine. They fed it data from the real engine and adjusted the ghost until it could reproduce the engine's movements perfectly.
Step 2: Train the Shortcut. Next, they used the ghost to train the "shortcut" (the SGS model). Because the ghost is flexible, they could use online learning techniques to teach the shortcut how to behave while interacting with the ghost engine.

Crucially, they realized that if they tried to train the ghost and the shortcut at the same time, the shortcut would just learn to compensate for the ghost's mistakes. To stop this, they trained them separately. They also introduced a special trick: instead of just telling the shortcut "make the weather look right," they told it "make the tiny swirls look right." This prevented the shortcut from accidentally fixing errors in the ghost engine that it shouldn't be fixing.

The Results: A Stable Ride

The team tested this method on two different systems. First, they used a simplified, chaotic system called the Lorenz-96, which is like a toy model of the atmosphere with fast and slow moving parts. They found that their method produced a shortcut that was almost as good as the "perfect" online training, but without needing to modify the original code.

Then, they moved to a more complex test: a quasi-geostrophic system, which simulates the swirling winds and currents of the ocean and atmosphere. This is a notoriously difficult problem because the tiny swirls can either calm things down or make them explode into chaos.

In these simulations, they compared their new method against older techniques:

The "Offline" Model: This model learned from a static dataset. It failed quickly, causing the simulation to crash as energy built up uncontrollably.
The "State" Loss Model: This model tried to learn by looking at the big picture (the overall weather state). It was unstable and produced wild oscillations.
The "Subgrid" Loss Model (The Winner): This model, trained with the authors' two-step method and focusing on the tiny swirls, remained stable for 250 times longer than the training period. It successfully reproduced the energy patterns and stability of the "perfect" online model, even though it never saw the original, rigid code's internal gears.

What This Means

The paper suggests that we don't need to rewrite the massive, decades-old climate codes to use modern AI. Instead, we can build a "ghost" version of the code to do the heavy lifting of training. This opens the door to training much more stable and accurate climate models that can run for years without crashing.

However, the authors are careful to note that this isn't a magic wand yet. The "ghost" engine isn't perfect; it has small errors. In their tests, these small errors sometimes led to slight inaccuracies in the large-scale weather patterns, though the tiny details were spot on. They suggest that future work needs to make the ghost engines even better, perhaps by teaching them to focus on specific large-scale features like wind speeds, not just the swirling vorticity.

Ultimately, this research offers a promising path forward: a way to bridge the gap between the rigid, proven physics of the past and the flexible, powerful learning of the future, all without having to take the engine apart.

Technical Summary: Gradient-free online learning of subgrid-scale dynamics with neural emulators

Problem Statement
The paper addresses a critical bottleneck in developing hybrid numerical models for Earth System science, specifically the training of machine learning (ML)-based subgrid-scale (SGS) parametrizations. While "online" training strategies (where the ML component is trained while interacting with the numerical solver) have been shown to produce more stable and accurate simulations than "offline" approaches, they typically require the numerical solver to be differentiable. Most production-grade climate and fluid dynamics codes are legacy systems written in low-level languages (e.g., Fortran, C) and are not differentiable. Re-implementing these codes in automatic differentiation (AD) frameworks is often prohibitively expensive and complex. Consequently, online learning is currently inaccessible for many state-of-the-art models, forcing researchers to rely on less stable offline strategies or gradient-free optimization methods that may not scale well.

Methodology
The authors propose a generic, two-step algorithm that enables online learning for SGS parametrizations without requiring the original solver to be differentiable. The core innovation is the use of a neural emulator—a differentiable neural network trained to approximate the coarse-resolution solver's dynamics.

The proposed workflow consists of two sequential stages:

Neural Emulator Training: A differentiable emulator $E$ is trained to approximate the coarse-resolution solver $g$ . This step minimizes a "state" loss function, ensuring the emulator accurately reproduces the temporal evolution of the coarse variables.
SGS Parametrization Training: Once the emulator is fixed, the SGS model $M$ is trained online using the emulator in place of the original solver. Crucially, the authors introduce a compensated loss function. Instead of minimizing the error in the coarse state variables (which would allow the SGS model to absorb errors from the imperfect emulator), the loss function is re-targeted to minimize the error directly on the SGS terms (the missing physics). This prevents the SGS model from compensating for the emulator's approximation biases, isolating the learning objectives of the two components.

This approach allows for gradient propagation through the temporal integration steps via the differentiable emulator, effectively simulating an online training environment for a non-differentiable solver.

Key Contributions

Gradient-Free Online Learning: The paper demonstrates a method to perform online, a posteriori training of SGS parametrizations for non-differentiable legacy codes by leveraging a differentiable neural emulator.
Bias Separation via Loss Re-targeting: The authors identify that jointly training the emulator and the SGS model leads to error compensation, where the SGS model learns to fix the emulator's mistakes rather than the true physics. They propose a two-step sequential training scheme with a specific "subgrid" loss function that targets the SGS term directly, thereby minimizing the propagation of approximation biases.
Algorithmic Framework: The paper outlines a generic algorithm (Algorithm 1) that can be applied to various dynamical systems, decoupling the need for solver differentiability from the benefits of online learning.

Results
The methodology was validated on two systems:

Lorenz-96 System: A chaotic, two-timescale system. The authors showed that the emulator-based approach with the subgrid loss could recover the stability and accuracy of a reference online model trained on a differentiable solver. Models trained with the standard "state" loss on the emulator failed to match the reference performance, highlighting the necessity of the bias-compensation strategy.
Quasi-Geostrophic (QG) System: A two-dimensional turbulent flow system known for its instability with offline strategies. The study compared models trained with different loss functions (state vs. subgrid) and emulator complexities (small vs. large).
- Stability: The proposed method produced stable simulations over temporal horizons 250 times longer than the training data, whereas offline-trained models and those using the state loss with small emulators crashed.
- Accuracy: The model trained with the subgrid loss and a large-capacity emulator achieved performance comparable to the reference online model regarding small-scale quantities (vorticity distributions and enstrophy spectra).
- Limitations: While small-scale metrics were well-reproduced, systematic biases remained at large scales, attributed to the inherent imperfection of the neural emulator. The authors noted that the choice of emulator architecture significantly impacted the final SGS performance.

Significance and Claims
The paper positions this work as a "step towards learning parametrization with online strategies for climate models." The authors claim that their approach opens new avenues for learning stable parametrizations for non-differentiable legacy codes, which constitute the majority of production Earth System models. They emphasize that their method recovers "most of the benefits of online strategies" without the computational burden of re-implementing solvers in AD languages.

The authors remain modest regarding the current state of the technology, acknowledging that:

The approach relies heavily on the quality of the neural emulator; imperfect emulation introduces biases that the SGS model cannot fully correct.
The method is currently demonstrated on simplified systems (L96 and QG), and deploying it to full-complexity Earth System models requires further development.
Systematic comparisons with other gradient-free techniques (e.g., reinforcement learning, ensemble Kalman inversion) are needed to determine the optimal strategy for practical applications.

Ultimately, the paper argues that neural emulation provides a practical alternative for enabling differentiable programming paradigms in hybrid climate modeling, potentially extending to tasks like model calibration and variational data assimilation without requiring adjoint solvers.

The Problem: The Black Box and the Missing Link

The Solution: The "Ghost Driver"

The Results: A Stable Ride

What This Means

More like this