⚡ electrical engineering

Why Deterministic AI Weather Models Fail at Extremes and Physical Conservation: From the Perspective of Probabilistic Mean Field

This paper argues that deterministic AI weather models fail to capture extremes and conserve physical laws because minimizing mean-square error forces them to predict conditional means rather than full transition distributions, a limitation that probabilistic generative approaches can overcome by better representing intrinsic atmospheric variability.

Original authors: Yihui Ding, Baoheng Yao, Jingsong Yang, Wenlei Peng

Published 2026-07-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yihui Ding, Baoheng Yao, Jingsong Yang, Wenlei Peng

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the atmosphere and the ocean as a giant, chaotic dance floor. The air and water are constantly swirling, colliding, and spinning in complex patterns that follow strict, invisible rules of physics. For decades, scientists have tried to predict where this dance will go next using massive supercomputers that solve these physics equations step-by-step. But there's a new player on the floor: Artificial Intelligence (AI). These AI models are like super-fast dancers who have watched millions of hours of past weather videos. They can predict the next move in a fraction of a second, using a tiny fraction of the energy of traditional computers. This is a big deal because it could help us warn people about storms, plan for renewable energy, and manage floods much faster than before.

However, there's a catch. While these AI dancers are incredibly fast and often get the "average" move right, they sometimes look a bit too smooth. They tend to miss the wild, crazy spins (extreme weather) and occasionally stumble over the basic rules of the dance (physical laws like conservation of energy). Scientists have been scratching their heads, wondering why these super-smart AI models keep smoothing out the rough edges of reality. Is it because they haven't seen enough extreme weather in their training? Is it a flaw in the math? This paper dives deep into that mystery, proposing a unified theory that explains why these models behave the way they do, using a concept called "probabilistic mean field" to show that the very method used to make them accurate might be the reason they fail at the extremes.

The "Average" Trap: Why AI Weather Models Smooth Out Reality

So, what did the researchers at the Second Institute of Oceanography and Shanghai Jiao Tong University actually find? They discovered that the problem isn't just a lack of data or a bad algorithm; it's a fundamental mathematical consequence of how most current AI weather models are taught to learn.

Think of it like this: Imagine you are trying to guess the next move of a dancer based on their past moves. If you are trained to minimize your "average mistake" (a math concept called Mean Squared Error, or RMSE), your brain naturally starts looking for the average of all possible next moves. If the dancer could either spin left, spin right, or jump, and you want to be "right" on average, you might predict a weird, half-spin, half-jump move that never actually happens in real life. You've found the mathematical center, but you've lost the reality of the dance.

The authors argue that current AI weather models are doing exactly this. By training them to minimize the average error, they are teaching the AI to predict the conditional expectation—essentially, the statistical average of all possible futures. In the real world, weather is a "nonlinear" system, meaning that if you take two valid weather states and average them together, the result is usually not a valid weather state. It's like mixing hot and cold water to get "lukewarm" water; that's fine. But if you mix a hurricane and a calm day, you don't get a "medium" storm that follows the laws of physics; you get a mathematical ghost that violates the rules of nature.

The Experiments: From Simple Dice to Swirling Fluids

To prove this, the team ran two clever experiments.

First, they created a simple "toy" world using a Markov chain (think of it as a board game where your next move depends only on where you are right now). They trained two AI "players": one that tried to guess the exact next number (minimizing error) and one that tried to guess the full list of probabilities for every possible next number.

The Result: Both players got the "average" score almost perfectly. But the error-minimizing player was terrible at capturing the spread of possibilities. It collapsed all the complex possibilities into a single, safe, boring guess. It was like a student who memorized the average test score but couldn't tell you what the actual questions were. The paper shows that this "average" guess often lands in a spot that isn't even a legal state in the game.

Second, they moved to a more realistic simulation of a swirling fluid (the Kelvin–Helmholtz instability), which mimics how air and water mix in the atmosphere. They compared a standard AI model (trained to minimize error) against a "Diffusion Model" (a type of AI that learns to generate a whole range of possible futures, like a creative artist).

The Result: The standard AI model produced a forecast that looked incredibly smooth. It had a slightly lower error score on paper, but when you looked closely, it had smoothed out all the tiny, turbulent swirls and sharp edges that make weather real. It was like a photo that had been heavily blurred to remove noise.
The Diffusion Model, however, generated a whole family of possible futures. While the average of these futures looked just as smooth as the standard AI model, the individual samples were sharp, detailed, and kept the tiny swirls and extreme gradients intact.

The "Overconfident" AI

The paper suggests that the core issue is entropy, or information. Real weather is messy, uncertain, and full of possibilities (high entropy). A standard AI model, by giving you just one single "best guess," is essentially saying, "I am 100% sure this is what will happen." This is overconfidence. It's like a weather forecaster saying, "It will rain exactly at 2:03 PM," when in reality, it might rain anytime between 1:00 and 4:00 PM.

Because the AI is so confident in its single, smooth average, it fails to predict extremes. If a storm is going to be a 100 mph hurricane or a 50 mph gale, the AI might predict a safe 75 mph wind, which is the average but a disaster for anyone trying to prepare for the worst. Furthermore, because that "average" state is a mathematical mix of different physical realities, it often breaks the laws of physics, like failing to conserve energy or mass.

What This Means for the Future

The authors conclude that the solution isn't just to tweak the training data or change the loss function slightly. The problem is that deterministic models (ones that give a single answer) are fundamentally ill-suited for a chaotic, probabilistic world like our weather.

Instead of asking the AI, "What is the single most likely weather tomorrow?", we should be asking, "What are all the possible weathers tomorrow, and how likely are they?" The paper points toward generative models (like Diffusion models) as the future. These models don't just guess one number; they sample from a distribution, creating a set of possible futures that captures the true uncertainty and complexity of the atmosphere.

In short, the paper suggests that to truly master weather prediction, AI needs to stop trying to be a perfect "average" guesser and start acting more like a creative ensemble, generating a rich variety of realistic, physically consistent possibilities. This shift could be the key to finally getting AI to handle the extreme events and physical laws that have been holding it back.

Technical Summary: Why Deterministic AI Weather Models Fail at Extremes and Physical Conservation

Problem Statement
While deterministic AI weather and ocean models (e.g., Pangu-Weather, FuXi) have demonstrated superior computational efficiency and lower mean-square error (RMSE) compared to traditional physics-based numerical weather prediction (NWP) systems, they suffer from critical structural deficiencies. These include the over-smoothing of small-scale structures, systematic underestimation of extreme events, and violations of physical conservation laws (e.g., mass, energy, and momentum budgets). Existing explanations for these failures are largely phenomenological, attributing them to data imbalances, the "frequency principle" of neural networks, or the lack of embedded physical equations. This paper argues that these limitations stem from a fundamental theoretical mismatch: deterministic models trained with RMSE minimize the error of the conditional mean rather than learning the full probabilistic transition law of the atmosphere.

Methodology
The authors propose a unified theoretical framework based on the Probabilistic Mean Field perspective, treating atmospheric and oceanic evolution as a discrete-time Markov process. The study employs two complementary experiments to validate this theory:

Finite-State Markov Experiment:
- A toy model representing atmospheric evolution as a finite-state Markov chain with varying state space sizes ( $K=3, 10, 100$ ).
- Two Multi-Layer Perceptrons (MLPs) are trained on the same transition data:
  - MLP(RMSE): Trained with Mean Squared Error (MSE) to predict a scalar next state (deterministic regression).
  - MLP(Cross-Entropy): Trained with Cross-Entropy loss to predict the full conditional probability distribution (distributional learning).
- The study compares the inferred transition matrices and the KL divergence against the ground truth.
Kelvin–Helmholtz (KH) Fluid Experiment:
- A spatially extended, chaotic fluid system governed by 2D compressible Euler equations, simulated using the Trixi.jl DGSEM solver.
- A 200-member ensemble is generated by applying stochastic perturbations to the initial conditions and time-stepping, creating a Monte Carlo approximation of the conditional distribution $p(x_{t+\Delta t} | x_t)$ .
- Two U-Net architectures with identical backbones are trained:
  - Deterministic U-Net: Trained with global RMSE to predict the conditional mean field.
  - Probabilistic EDM (Diffusion) Model: Trained to sample from the conditional distribution $p(x_{t+\Delta t} | x_t)$ .
- Forecasts are evaluated via autoregressive rollouts, comparing spectral power, vorticity RMSE, and global momentum conservation.

Key Theoretical Contributions
The paper establishes two primary propositions:

MSE Minimization Yields Conditional Expectation: Under the Markov assumption, minimizing RMSE forces a deterministic model to learn the conditional expectation $E[x_{t+\Delta t} | x_t]$ rather than the full transition law $p(x_{t+\Delta t} | x_t)$ . This explains why low RMSE correlates with over-smoothing; the mean of a multi-modal or heavy-tailed distribution often falls in low-probability regions or fails to capture extremes.
Violation of Nonlinear Dynamics: The conditional mean is a linear combination of potential physical states. Since the governing equations of atmospheric flow are nonlinear, a linear combination of valid solutions is generally not a valid solution itself. Consequently, the "optimal" RMSE forecast is structurally incapable of satisfying the physical conservation laws inherent to the nonlinear system, leading to systematic physical non-conservation.

Experimental Results

Markov Experiment: Both MLP models achieved nearly identical RMSE on the test set. However, the RMSE-trained model collapsed the transition distribution into a single "nearest state" (effectively a one-hot vector), resulting in high KL divergence from the true distribution. The Cross-Entropy model successfully recovered the full transition matrix, including multi-modality and tail structures.
KH Fluid Experiment:
- Spectral Dissipation: The deterministic U-Net exhibited significant spectral damping at high wavenumbers ( $k \geq 0.3$ ), losing fine-scale turbulent structures. Individual diffusion samples retained these high-frequency details, closely matching the truth.
- Physical Conservation: The deterministic model systematically violated global momentum conservation (drifting by +2–5%), whereas diffusion samples maintained momentum budgets within $\leq 0.5\%$ of the reference.
- Ensemble Mean vs. Deterministic: The ensemble mean of the diffusion samples visually and spectrally resembled the deterministic forecast (both being smoother than the truth), confirming that the deterministic model effectively learns the ensemble mean.
- Extreme Values: Deterministic models consistently underestimated extreme values, while diffusion samples captured the full range of variability, including rare extremes.

Significance and Claims
The paper claims that the failure of deterministic AI weather models is not merely a result of specific loss functions (like MSE vs. MAE) or data scarcity, but an inherent consequence of deterministic point estimation in a probabilistic, nonlinear system.

Information Entropy Mismatch: Deterministic forecasts represent a low-entropy, overconfident prediction that discards the intrinsic uncertainty of the atmosphere. This "entropy deficit" is the root cause of the smoothing and physical violations.
Paradigm Shift: The authors argue that future AI weather systems must shift from point estimators to distributional learners (e.g., diffusion models, GANs). These generative approaches learn the full conditional transition law $p(x_{t+\Delta t} | x_t)$ , thereby preserving distributional information, maintaining physical consistency, and providing reliable probabilistic forecasts for extremes.
Re-evaluation of Metrics: The study suggests that low RMSE in deterministic models may be misleading, as it reflects an accurate approximation of the conditional mean rather than a faithful representation of the underlying dynamical system's complexity.

The paper concludes that while deterministic models may offer computational speed, their structural inability to represent the full transition law limits their utility for extreme event prediction and physically consistent reconstruction, necessitating a move toward generative, probabilistic frameworks.

The "Average" Trap: Why AI Weather Models Smooth Out Reality

The Experiments: From Simple Dice to Swirling Fluids

The "Overconfident" AI

What This Means for the Future

Technical Summary: Why Deterministic AI Weather Models Fail at Extremes and Physical Conservation

More like this