📊 statistics

Evaluating the performance of GCM trajectories using Weather Type frequencies for persistence and transitions: the Iberian Peninsula and Lamb classification

This study introduces a novel framework for evaluating CMIP6 GCM performance over the Iberian Peninsula by analyzing Lamb Weather Type persistence and transitions alongside daily frequencies, revealing that while many models capture basic circulation patterns, only 12 trajectories (notably from the EC-Earth3 family) accurately reproduce day-to-day atmospheric dynamics, particularly highlighting a performance gap in the central and southern Mediterranean regions.

Original authors: Elsa Barrio-Torres, Swen Brands, Jesús Asín, Jesús Abaurrea, Zeus Gracia-Tabuenca

Published 2026-05-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Elsa Barrio-Torres, Swen Brands, Jesús Asín, Jesús Abaurrea, Zeus Gracia-Tabuenca

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Earth's atmosphere as a giant, complex dance floor. Every day, the air moves in specific patterns, like dancers following a choreography. Some days, the dancers move in a slow, steady circle (a high-pressure system); other days, they swirl wildly (a low-pressure system). In Spain, these dance moves are crucial because they decide whether you get a scorching hot summer or a cool, rainy one.

Scientists use giant computer programs called Global Climate Models (GCMs) to predict how this dance will look in the future. But here's the problem: sometimes these computers get the choreography wrong. They might get the types of dances right (e.g., "today is a circle dance"), but they might get the flow wrong (e.g., "the dancers switch to a swirl dance too quickly" or "they stay in a circle dance for too long").

This paper is like a strict dance critic reviewing 36 different computer programs to see which ones are the best at mimicking the real atmosphere over the Iberian Peninsula (Spain and Portugal).

The Old Way vs. The New Way

The Old Way (Daily Frequency):
Imagine judging a dance troupe only by counting how many times they did a "circle dance" in a year. If the real world did it 100 times and the computer did it 100 times, the old method would say, "Great job!" But this misses the point. What if the computer did all 100 circle dances on the same day, and then never danced again? That's a terrible simulation of reality.

The New Way (Transitions and Persistence):
This paper introduces a smarter way to judge. It asks two questions:

Frequency: Did the computer do the right types of dances the right number of times?
The Flow (Transitions & Persistence): Did the computer know how to switch from one dance to another?
- Persistence: If the real world stayed in a "heatwave dance" for three days in a row, did the computer do the same?
- Transitions: If the real world switched from a "windy dance" to a "calm dance," did the computer switch at the right time?

The Experiment

The researchers took 36 different computer models (from the CMIP6 project) and compared their "dance moves" against the ERA5 dataset, which acts as the "gold standard" recording of what actually happened in the atmosphere between 1979 and 2005.

They used a specific classification system called Lamb Weather Types (think of this as a dictionary with 27 different dance moves, ranging from "Pure Anticyclone" to "Unclassified Chaos").

The Filtering Process

The researchers didn't just look at the final score; they used a strict elimination process:

The Frequency Check: First, they checked if the models got the daily counts right. 20 models failed this test and were kicked out.
The Flow Check: Of the remaining 16, they checked if the models could handle the switching between dances (transitions) and staying in a dance (persistence).
The Final Cut: 4 more models failed this second, harder test.

The Result: Only 12 models made the cut. These are the only ones the researchers trust to simulate the future climate of the Iberian Peninsula accurately.

The Winners and Losers

The Champion: Models from the EC-Earth3 family were the clear winners. Specifically, the EC-Earth3-AerChem model was the "star of the show," performing consistently well across the entire region. It was the best at both counting the dances and getting the transitions right.
The Regional Struggle: The study found a "geographical gap." Most models were great at predicting the weather in the Northwest (near the Atlantic Ocean). However, they struggled significantly in the Central and Southern parts (near the Mediterranean Sea). It's as if the models know how to dance on a wooden floor but trip on the sand.
The Losers: Models like INM-CM5 and TAIESM1 generally performed poorly and were filtered out early.

Why This Matters

The paper concludes that looking at just "how often" a weather type happens isn't enough. It's like judging a movie by only counting the number of times the hero appears, without checking if the plot makes sense.

By checking the transitions (how the weather changes from day to day), the researchers found a much sharper tool for picking the best models. They discovered that many models that looked good on paper (daily counts) actually failed to capture the real, dynamic behavior of the atmosphere.

In short: If you want to know what the future holds for heatwaves and droughts in Spain, don't just trust any climate model. Trust the ones that have proven they can not only count the days but also get the story of the weather right. And right now, the EC-Earth3 family is the best storyteller.

1. Problem Statement

The Iberian Peninsula is a climate change hotspot, highly vulnerable to extreme summer heatwaves and droughts driven by persistent large-scale atmospheric circulation patterns. To reliably project future climate impacts, Global Climate Models (GCMs) must accurately reproduce these circulation dynamics.

However, traditional GCM evaluation often focuses solely on the daily frequency distribution of weather patterns. This approach fails to capture the temporal dynamics (persistence and day-to-day transitions) of atmospheric circulation, which are critical for understanding the duration and intensity of extreme events like heatwaves. There is a lack of objective, systematic frameworks that evaluate GCMs based on both the frequency of weather types and their conditional evolution (transitions).

2. Methodology

The authors propose a novel methodological framework to evaluate and select GCM trajectories based on Lamb Weather Types (WTs) (specifically the Jenkinson–Collison classification, yielding 27 categories). The study focuses on the summer months (June–September) from 1979 to 2005 over the Iberian Peninsula (30 grid points).

Key Components:

Data Sources:
- Reference: ERA5 reanalysis data.
- Models: 36 historical trajectories from CMIP6 GCMs.
Statistical Framework:
- Joint Relative Frequency (rf): Calculates the probability of a specific WT occurring on day $d$ and a specific WT on day $d-1$ .
- Daily Distribution: The marginal probability of a WT occurring ( $P(WT_d = i)$ ).
- Conditional Distribution (Transitions): The probability of a WT occurring on day $d$ given a specific WT on day $d-1$ ( $P(WT_d = i | WT_{d-1} = j)$ ). This includes persistence ( $i=j$ ).
Similarity Measures:
- The study compares three metrics: Overlap coefficient, Bhattacharyya coefficient, and Hellinger distance.
- Selection: The Overlap coefficient was selected as the primary metric because it demonstrated superior discriminatory power and robustness against rare weather types compared to the other measures.
Filtering and Selection Algorithm:
1. Local Filtering: A trajectory is retained only if it achieves an Overlap score $\ge 0.8$ ( $t_{sim}$ ) for both daily and conditional distributions in at least 10 out of 30 grid points ( $k$ ).
2. Regional Ranking: Surviving trajectories are ranked using three integrated metrics:
  - Daily Reproduction (DR): Sum of Overlap scores for daily frequencies.
  - Conditional Reproduction (CR): Weighted sum of Overlap scores for transition probabilities (local and regional weighting).
  - Persistence Reproduction (PerR): Measures the accuracy of reproducing the probability of a WT persisting for two consecutive days.

3. Key Contributions

Methodological Advancement: The paper moves beyond static frequency analysis to evaluate 24-hour atmospheric dynamics. It introduces a rigorous framework for assessing transition probabilities and persistence, which are often overlooked in GCM selection.
Objective Selection Criteria: It establishes a filtering process that discards models failing to reproduce either daily frequencies or transition dynamics, ensuring selected models are robust in both static and dynamic aspects.
Discriminative Power: The study demonstrates that conditional metrics (transitions/persistence) are far more discriminative than daily frequency metrics alone. Many models that pass daily frequency checks fail significantly when transition dynamics are evaluated.
Open Source: The associated code for this framework is made publicly available via GitHub, allowing application to any region globally.

4. Results

Filtering Outcomes:
- Initial Pool: 36 CMIP6 trajectories.
- Daily Filter: 20 models were rejected for failing to reproduce daily WT frequencies adequately (Overlap < 0.8 in >1/3 of grid points). 16 remained.
- Transition Filter: 4 additional models were rejected for failing to reproduce conditional distributions (transitions/persistence).
- Final Selection: 12 trajectories were retained for further analysis.
Model Performance:
- Top Performer: The EC-Earth3 family consistently outperformed other models. Specifically, the ec earth3 aerchem trajectory exhibited the best and most consistent performance across the entire Peninsula and all metrics (DR, CR, PerR).
- Geographical Disparities:
  - Northwest: Models generally performed well in reproducing circulation patterns.
  - Central & Southern Mediterranean: Significant performance gaps were observed. Models struggled to accurately represent circulation dynamics in these regions, regardless of the family.
- Specific Model Behaviors:
  - mpi esm 1 2 hr performed well in the Northwest.
  - cnrm family models showed localized dominance in the Southeast under specific conditions.
  - taiesm1 and hadgem3 gc31 mm showed poor performance in specific regions (Southwest) and were deemed less suitable for local analyses.
Metric Correlation:
- Daily Reproduction (DR) and Conditional Reproduction (CR) scores were highly correlated.
- Persistence Reproduction (PerR) was found to be largely independent of DR and CR, providing a unique and critical dimension for model ranking.

5. Significance and Conclusion

This study establishes that assessing Weather Type persistence and transitions is a superior tool for GCM selection compared to evaluating daily distributions alone.

Implication for Impact Studies: Since extreme heat and droughts in the Iberian Peninsula are driven by persistent circulation patterns, selecting models that only get the "average" frequency right but fail to capture the "persistence" of those patterns could lead to significant underestimation of extreme event risks.
Model Selection: The EC-Earth3 family, particularly the aerchem variant, is identified as the most reliable for studying the Iberian Peninsula's summer climate.
Future Application: The framework is designed to be region-agnostic. It can be applied to other regions (e.g., the Niño region) and extended to future climate scenarios, provided the models first pass the historical performance filter. This ensures that projections of future changes are based on models that have proven capable of reproducing the fundamental dynamics of the current climate.

In summary, the paper provides a robust, objective, and dynamic methodology for filtering climate models, emphasizing that temporal evolution (transitions) is just as critical as spatial frequency in climate model evaluation.