TianQuan-S2S: A Subseasonal-to-Seasonal Global Weather Model via Incorporate Climatology State

The paper introduces TianQuan-S2S, a novel global subseasonal-to-seasonal weather forecasting model that integrates climatological states into patch embeddings and utilizes an uncertainty-augmented Transformer to overcome the limitations of over-smoothing and inadequate climate representation, thereby outperforming both traditional numerical methods and advanced data-driven models in deterministic and ensemble forecasting.

The Gist

Imagine you are trying to predict the weather not just for tomorrow, but for the next two to six weeks. This is a notoriously difficult task. It's like trying to predict exactly where a leaf will land in a swirling storm, or guessing the exact outcome of a game of billiards after the balls have been hit 50 times.

This paper introduces a new AI system called TianQuan-S2S (which roughly translates to "Weather Hub") designed to solve this specific problem. Here is how it works, explained through simple analogies.

The Problem: The "Blurry Photo" Effect

Current weather models, even the super-smart AI ones, have a major flaw when looking far into the future.

• The Issue: If you ask a standard AI to predict the weather 30 days out, it starts to get lazy. It forgets the specific details (like a sudden storm front or a heatwave) and just gives you a "blurry" average.
• The Analogy: Imagine taking a photo of a busy city street. If you zoom out too far, the people and cars disappear, and you just see a gray blob. That is what happens to weather models over time; they lose the "fine print" and become too smooth to be useful. This is called Model Collapse.

The Solution: TianQuan-S2S

The authors built a new system that fixes this by using two clever tricks. Think of it as giving the AI a "cheat sheet" and a "shake-up" mechanism.

Trick 1: The "Cheat Sheet" (Climatology)

Standard AI models try to guess the future based only on the current weather (e.g., "It's raining now, so it will rain later"). But for long-term predictions, the current weather isn't enough. You need to know the season.

• The Analogy: Imagine you are guessing what a friend is wearing.
  • Old Way: You look at what they are wearing right now (a t-shirt) and guess they will wear a t-shirt in two weeks.
  • TianQuan Way: You look at what they are wearing now, PLUS you check the calendar. You know it's December in Canada. So, you guess they will be wearing a heavy coat, even if they are currently in a t-shirt indoors.
• How it works: The model feeds the AI a "Climatology" map. This is a 30-year average of what the weather usually does at this time of year. The AI learns to blend the "current chaos" with the "seasonal rules," preventing it from drifting into nonsense.

Trick 2: The "Shake-Up" (Uncertainty & Noise)

Weather is chaotic. If you leave a model alone, it tends to settle into a boring, predictable pattern (the "blurry photo" mentioned earlier).

• The Analogy: Think of a choir singing a song. If everyone sings exactly the same note at the exact same time, it sounds flat. But if you ask them to add a little bit of natural variation (some slightly louder, some slightly softer), the sound becomes rich, realistic, and alive.
• How it works: The model injects a tiny bit of random "noise" (like static on a radio) into its thinking process at every single step. This forces the AI to explore different possibilities rather than just settling on one average answer. It keeps the forecast "sharp" and realistic, preventing the model from collapsing into a boring blur.

Why This Matters

The authors tested their new model against:

1. Supercomputers: The traditional, heavy-duty weather machines used by governments (like ECMWF).
2. Other AIs: The current state-of-the-art AI weather models (like FuXi and ClimaX).

The Result: TianQuan-S2S won.

• It was more accurate at predicting temperature, wind, and pressure 15 to 45 days out.
• It didn't get "blurry" as fast as the others.
• It could run in seconds on a standard computer, whereas the supercomputers take hours or days.

The Big Picture

This paper is a breakthrough because it admits that AI alone isn't enough for long-term weather. You have to teach the AI about the "big picture" (the seasons) and keep it "awake" with a little bit of chaos (noise).

By combining the stability of history (climatology) with the flexibility of modern AI, TianQuan-S2S helps farmers plan harvests, energy companies manage power grids, and emergency teams prepare for storms weeks before they happen. It turns a blurry guess into a sharp, reliable forecast.

Technical Summary

1. Problem Statement

Subseasonal-to-Seasonal (S2S) forecasting (15 to 45 days ahead) is critical for agriculture, energy, and disaster management but remains a significant challenge due to the chaotic nature of the atmosphere. The paper identifies two primary limitations in current data-driven approaches:

• Insufficient Climate Modeling: Existing models often rely heavily on initial weather states, which lose predictive power over extended horizons. They fail to adequately incorporate climatology (long-term statistical averages and slow-varying climate modes), which is essential for S2S timescales.
• Model Collapse: As forecast lead times increase, data-driven models tend to degrade, producing "over-smoothed" predictions that lose fine-scale details and realistic variability. This phenomenon, termed "model collapse," results in forecasts that converge to a mean state, losing the ability to predict specific weather events.

2. Methodology: TianQuan-S2S

The authors propose TianQuan-S2S, a global weather forecasting model designed to bridge the gap between initial conditions and long-term climate states. The architecture consists of three core components:

A. Patch Embedding with Climatology Integration

Instead of treating climatology as a separate post-processing step, the model integrates it directly into the input representation:

• Convolutional Feature Extraction: A multi-layer convolutional network extracts features from the initial weather state ($X$) and the climatology state ($X_{clim}$). This includes spatial convolutions to capture regional variations and channel convolutions to model relationships between variables (e.g., temperature, pressure, wind).
• Attention-Based Fusion: An attention mechanism adaptively fuses the enhanced initial features ($F_X$) and climatology features ($F_{clim}$). It computes spatial weights to determine the relative importance of initial conditions versus climatology at each grid point, ensuring the model leverages climate priors effectively for long-term predictions.
• Fourier Embedding: To preserve spatial (latitude/longitude) and temporal (seasonal cycles) information, the model uses Fourier encoding to map periodic positional signals into the latent space.

B. Uncertainty-Augmented Transformer

To combat model collapse and over-smoothing, the authors modify the standard Vision Transformer (ViT) architecture:

• Learnable Noise Injection: Inspired by traditional perturbed forecasting (ensemble methods), the model injects learnable Gaussian noise into the feature representations at every Transformer block during the forward pass.
• Mechanism: Unlike methods that only perturb initial conditions (which lose impact over time), TianQuan-S2S continuously injects noise ($E^{(n+1)} = E^{(n)} + h_n(E^{(n)}) + g_n(E^{(n)}) \cdot \mathcal{N}(0, I)$). This prevents the forecast from collapsing onto a single deterministic trajectory, helping the model capture the growth of uncertainty and maintain realistic variability over long lead times.

C. Training Strategy

• Multi-Model Approach: Rather than a single autoregressive model that accumulates errors, the authors train separate models for specific lead-time segments (e.g., predicting days 15–20, 20–25, etc., based on a 5-day input window). These are combined to form the full 15–45 day forecast.
• Loss Function: The model is trained using a latitude-weighted Mean Squared Error (MSE) to account for the varying area of grid cells near the poles versus the equator.

3. Key Contributions

1. Climatology Integration: The paper highlights and implements the direct incorporation of climatology states into the patch embedding layer via attention-based fusion, addressing the lack of long-term climate context in existing S2S models.
2. Mitigation of Model Collapse: By introducing learnable Gaussian noise at every Transformer layer, the model effectively prevents over-smoothing and maintains fine-scale details in long-lead forecasts.
3. State-of-the-Art Performance: The proposed framework outperforms both traditional Numerical Weather Prediction (NWP) systems (ECMWF-S2S) and advanced data-driven baselines (FuXi-S2S, ClimaX) in both deterministic and ensemble forecasting settings.

4. Experimental Results

Experiments were conducted on the ERA5 reanalysis dataset (1979–2018) with resolutions of $1.40625^\circ$and$5.625^\circ$.

• Deterministic Forecasting:
  • TianQuan-S2S achieved significant improvements in RMSE and Anomaly Correlation Coefficient (ACC) across key variables: 2m Temperature (T2m), 850hPa Temperature (T850), 500hPa Geopotential (Z500), and 10m Wind Speed (Wind10).
  • It outperformed FuXi-S2S and ClimaX, particularly at longer lead times (beyond 25 days), where baseline models suffered from error accumulation and model collapse.
• Ensemble Forecasting:
  • The model generated a 51-member ensemble (1 unperturbed + 50 noise-perturbed runs).
  • It showed superior performance in probabilistic metrics (CRPS, SME, RQE) compared to baselines, indicating better reliability and sharpness in uncertainty quantification.
  • Notably, the ensemble mean RMSE remained below the climatology baseline for T2m, T850, and Z500, whereas some baselines failed to beat climatology at longer horizons.
• Ablation Studies:
  • Removing climatology fusion caused a significant drop in performance for lead times >25 days.
  • Removing noise injection led to over-smoothing and higher RMSE, confirming the necessity of the uncertainty-augmented design.
  • Optimal noise scale ($\sigma \approx 1$) was identified for balancing regularization and accuracy.

5. Significance

TianQuan-S2S represents a significant step forward in AI-driven weather forecasting by addressing the specific challenges of the subseasonal-to-seasonal timescale.

• Bridging the Gap: It successfully bridges the gap between short-term weather prediction (driven by initial conditions) and long-term climate prediction (driven by climatology).
• Practical Impact: By outperforming the operational ECMWF-S2S system and advanced AI models, it offers a more accurate, computationally efficient (seconds vs. hours) alternative for critical decision-making in agriculture and energy sectors.
• Methodological Insight: The work demonstrates that incorporating physical priors (climatology) and explicitly modeling uncertainty (learnable noise) are essential strategies to prevent the "model collapse" that plagues long-horizon deep learning forecasts.

The code and implementation are open-sourced at https://github.com/zhangminglang42/TianQuan.