🔬 physics

Evaluating Skill and Stability of ArchesWeather and ArchesWeatherGen under Multi-Decadal Climate Simulations

This paper demonstrates that the weather forecasting models ArchesWeather and ArchesWeatherGen can be successfully adapted into stable, long-term forced atmospheric models that accurately reproduce ERA5 climatology, large-scale circulations, and interannual variability when conditioned on sea surface temperature and sea ice cover under the AIMIP Phase 1 protocol.

Original authors: Renu Singh, Robert Brunstein, Antonia Jost, Thomas Rackow, Claire Monteleoni, Yana Hasson, Christian Lessig, Guillaume Couairon

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Renu Singh, Robert Brunstein, Antonia Jost, Thomas Rackow, Claire Monteleoni, Yana Hasson, Christian Lessig, Guillaume Couairon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Earth's atmosphere as a giant, chaotic dance floor. For decades, scientists have tried to predict the steps of this dance using massive, complex rulebooks called "physics-based models." These rulebooks are incredibly accurate but require supercomputers the size of buildings to run, making it slow and expensive to simulate even a single year of climate.

Recently, a new type of dancer has emerged: Machine Learning (ML) models. These are like students who have watched thousands of hours of the dance (historical weather data) and learned to mimic the moves perfectly for the next few minutes or days. They are fast and cheap.

This paper asks a big question: Can these fast, AI dancers keep dancing for 45 years straight without tripping over their own feet?

Here is the breakdown of what the researchers did and found, using simple analogies:

1. The Two Dancers: ArchesWeather and ArchesWeatherGen

The researchers took two specific AI models originally built for short-term weather forecasting and tried to turn them into long-term climate simulators.

ArchesWeather (The Deterministic Dancer): This model is like a strict choreographer. If you give it the same starting position, it will always perform the exact same dance. It's very good at predicting the "average" move.
ArchesWeatherGen (The Probabilistic Dancer): This model is more like a jazz improviser. It knows the average move, but it also understands that the real dance has randomness and surprise. It generates a range of possible dances (an "ensemble"), allowing scientists to see not just one outcome, but the full spectrum of possibilities, including rare, extreme events.

2. The New Challenge: Adding the "Ocean" to the Dance

Weather models usually just look at the air. But to simulate climate (long-term trends), you need to know what the ocean is doing, because the ocean acts like a slow-moving battery that heats or cools the air over years.

The researchers gave these AI models a new instruction: "Look at the monthly average temperature of the ocean and the amount of sea ice, and let that guide your dance."

They followed a standardized rulebook called AIMIP (similar to a standardized test for climate models) to see if the AI could handle this new, long-term pressure.

3. The Results: Did They Stay on Beat?

The researchers let the models dance for 45 years (from 1979 to 2024) and compared their performance to the "gold standard" (ERA5, which is a high-quality record of real-world weather).

Stability: The models didn't crash. They kept dancing for 45 years without spiraling out of control.
The Annual Cycle: Just like real seasons, the models got hotter in summer and colder in winter. They captured the "beat" of the year perfectly.
The Big Patterns: The models successfully recreated massive global weather patterns, such as:
- The Monsoons: The seasonal rains in India.
- El Niño: The warming of the Pacific Ocean that affects weather worldwide.
- The Jet Streams: The fast rivers of wind in the upper atmosphere.
The "Tail" of the Dance: This is where the ArchesWeatherGen (the jazz improviser) shined. While the strict dancer (ArchesWeather) was good at the average moves, it missed the wild, extreme spins (like extreme humidity or strong winds). The probabilistic model captured these rare, extreme events much better, matching the real world's "outliers."

4. The "What If" Scenario: Heating Up the Ocean

To test if the models could predict future climate change, the researchers artificially warmed the ocean in the simulation by 2°C and 4°C (like turning up the thermostat on the dance floor).

The Reaction: The models reacted correctly. When the ocean got hotter, the air got hotter, humidity increased, and wind patterns shifted.
The Limitation: The models reacted, but their reaction was smaller than what traditional physics models predicted.
- The Analogy: Imagine you tell a student, "The room is now 4 degrees hotter." The student might say, "Okay, I'll feel a little warmer," but they might not fully realize the room is boiling because they were trained mostly on "normal" rooms. The AI is a bit cautious when faced with conditions it hasn't seen before (out-of-distribution).

5. The Verdict

The paper concludes that these AI models are surprisingly good at simulating long-term climate, even though they were originally built for short-term weather.

The Good: They are fast, stable, and capture the "average" climate and the "extreme" tails of the distribution very well.
The Catch: When asked to predict a future that is significantly different from the past (like a much hotter ocean), they are a bit conservative. They don't fully "commit" to the extreme warming that physics suggests might happen.

In short: The researchers successfully taught fast AI weather models to dance the long, slow waltz of climate change. They didn't just mimic the steps; they captured the rhythm, the seasons, and even the occasional wild spin, proving they could be a powerful, efficient tool for understanding our changing planet.

Technical Summary: Evaluating Skill and Stability of ArchesWeather and ArchesWeatherGen under Multi-Decadal Climate Simulations

Problem Statement
Climate simulations are essential for understanding future climate responses to anthropogenic forcings, yet traditional Earth System Models (ESMs) rely on solving partial differential equations (PDEs) with fine spatial discretization. This approach creates a significant computational bottleneck, requiring thousands of supercomputer cores to simulate a single year, thereby limiting spatiotemporal resolution, ensemble sizes, and the number of scenario runs. While data-driven models have achieved tremendous success and efficiency in weather forecasting, their adaptation to long-term climate simulation remains an open challenge. Specifically, extending short-term weather models (trained on days) to multi-decadal climate emulators requires incorporating the slow-moving physics that drive long-term evolution, such as radiative forcing and ocean-atmosphere coupling, without the computational cost of fully coupled physical models.

Methodology
The authors adapt two machine learning models, ArchesWeather (a deterministic model) and ArchesWeatherGen (a probabilistic flow-matching model), to function as forced atmospheric models under the AI Model Intercomparison Project (AIMIP) Phase 1 protocol. This protocol is analogous to the Atmospheric Model Intercomparison Project (AMIP), utilizing historical sea surface temperature (SST) and sea ice cover (SIC) as boundary conditions.

Data and Preprocessing: The models are trained on daily-averaged ERA5 data regridded to $1^\circ$ resolution. To remove the daily cycle and facilitate long-term dynamics, the dataset consists of daily averages of 6 surface and 6 upper-air variables across 13 pressure levels.
Forcing and Conditioning: The models are conditioned on monthly mean SST and SIC. These forcings are concatenated to surface variables before embedding.
Model Architecture: Both models utilize a transformer backbone improved upon Pangu-Weather.
- ArchesWeather-Mx4: A deterministic ensemble of four independently trained models (85M parameters each) trained to predict the next state ( $X_{t+\delta}$ ) given the previous two states and forcings.
- ArchesWeatherGen: A flow-matching generative model (45M parameters) trained on the normalized residuals between the true state and the ArchesWeather-Mx4 predictions. This allows for probabilistic forecasting and ensemble generation without perturbing initial conditions.
Training and Rollout: Models are trained on 1979–2013 data. Rollouts are initialized from October 1, 1978, and run autoregressively for 45 years (1979–2024). For warmed scenarios, SST forcings are uniformly shifted by +2K and +4K.
Evaluation Metrics: The study employs Climatology RMSE, Cramér–von Mises (CM) distance for distributional skill, Taylor diagrams for large-scale patterns (Annular Modes), Southern Oscillation Index (SOI) for ENSO, the Webster-Yang Index for the Indian Monsoon, and return period analysis for extremes.

Key Contributions

Protocol Adaptation: The work successfully adapts weather forecasting models into forced atmospheric climate models by integrating monthly SST and SIC as conditioning variables, adhering to the AIMIP Phase 1 experimental setup.
Stable Long-Term Rollouts: The authors demonstrate that these models, originally designed for 24-hour weather prediction, can produce stable 45-year climate simulations with a stable annual cycle and minimal drift.
Probabilistic Extension: The introduction of ArchesWeatherGen enables ensemble-based uncertainty quantification and the capture of distribution tails, addressing a limitation of deterministic weather models in climate contexts.
Comprehensive Benchmarking: The study provides a rigorous comparison against state-of-the-art physics-based models (e.g., MPI-ESM2-HR, CESM2) and ERA5 reanalysis across climatology, spectral energy, large-scale circulation modes, and extreme events.

Results

Climatology and Trends: Both models faithfully reproduce ERA5's climatology and large-scale circulations. The annual cycle of surface air temperature is stable, and the models capture warming trends, though they slightly underestimate the trend magnitude in the test period compared to ERA5.
Distributional Skill: ArchesWeatherGen outperforms deterministic counterparts and physics-based models in capturing the full distribution of climate states, particularly for variables with high-frequency content like specific humidity. It successfully reproduces the tails of distributions and maintains spectral energy at shorter wavelengths, avoiding the smoothing effects often seen in deterministic models.
Large-Scale Patterns:
- Annular Modes: The models accurately reproduce the Northern (NAM) and Southern (SAM) Annular Modes, with correlation coefficients and standard deviation ratios comparable to or better than numerical models.
- ENSO: The forced models successfully simulate the El Niño-Southern Oscillation cycle, capturing strong El Niño and La Niña events and the correct power spectral density (2–7 year range), which unforced configurations fail to do.
- Monsoon: The Webster-Yang monsoon index and wind vector fields closely match ERA5, showing correct circulation systems over South Asia.
Extremes: Return period analysis indicates that ArchesWeatherGen captures extreme wind and temperature events more accurately than the deterministic ensemble or the physics-based MPI-ESM2-HR.
Warmed Scenarios (+2K/+4K SST): Under out-of-distribution (OOD) warming forcings, the models remain stable and respond with increased surface air temperatures and specific humidity. The specific humidity response aligns closely with the Clausius-Clapeyron approximation (~6.1%/K). However, the magnitude of the temperature response is smaller than that of physical models (e.g., GFDL, CESM), and the models exhibit a poleward shift in the Southern Hemisphere jet stream.

Significance and Limitations
The paper claims that ArchesWeather and ArchesWeatherGen represent a significant step toward efficient, data-driven climate emulation. The primary significance lies in demonstrating that weather models, when properly conditioned on boundary conditions, can simulate multi-decadal climate dynamics with stability and fidelity comparable to physics-based models, but at a fraction of the computational cost. The generative model (ArchesWeatherGen) specifically addresses the need for uncertainty quantification and accurate representation of extremes.

The authors modestly acknowledge limitations:

Response Magnitude: The climate response to OOD warming forcings is smaller than expected and smaller than that of physical models. The authors hypothesize this is due to "regression to the mean" effects inherent in the deterministic base model and the lack of a dedicated coupling module between the atmosphere and ocean variables.
Computational Cost: While more efficient than PDE-based models, the diffusion-based generative approach (ArchesWeatherGen) is more expensive than deterministic inference.
Idealized Setup: The +2K/+4K scenarios are idealized (uniform warming) and do not represent a fully coupled physical reality, serving as a first assessment of suitability rather than a definitive climate projection.

The work concludes that while these models are remarkably effective for simulating historical climatology and variability, improving their response to future climate scenarios requires further research into better conditioning methods and dedicated coupling mechanisms.

1. The Two Dancers: ArchesWeather and ArchesWeatherGen

2. The New Challenge: Adding the "Ocean" to the Dance

3. The Results: Did They Stay on Beat?

4. The "What If" Scenario: Heating Up the Ocean

5. The Verdict

More like this