🤖 machine learning

Otter Weather: Skillful and Computationally Efficient Medium-Range Weather Forecasting

The paper introduces Otter Weather, a highly efficient spatiotemporal forecasting model that democratizes high-performance medium-range weather prediction by significantly outperforming both traditional numerical models and resource-intensive AI architectures in skill while requiring drastically lower computational budgets for training.

Original authors: Cristiana Diaconu, Jonas Scholz, Aliaksandra Shysheya, Stratis Markou, Payel Mukhopadhyay, Miles Cranmer, Richard E. Turner

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cristiana Diaconu, Jonas Scholz, Aliaksandra Shysheya, Stratis Markou, Payel Mukhopadhyay, Miles Cranmer, Richard E. Turner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world of weather forecasting as a high-stakes cooking competition. For decades, the winners have been "Numerical Weather Prediction" (NWP) models. These are like master chefs who follow incredibly complex, physics-based recipes. They are accurate, but they require a massive, industrial-sized kitchen (supercomputers) and a huge budget to run. Only a few elite labs can afford to cook with them.

Recently, a new type of chef emerged: AI Weather Models. These chefs learn by tasting thousands of dishes (data) rather than memorizing physics formulas. They have become so good that they often beat the master chefs. However, there's a catch: to train these AI chefs, you still need a massive, expensive kitchen with hundreds of high-tech ovens (GPUs). This means only the richest labs can create their own AI chefs; everyone else is stuck trying to tweak recipes left over by the big labs.

Enter "Otter Weather."

The authors of this paper built a new AI weather model called Otter. Their goal was to democratize weather forecasting—to prove you don't need a billion-dollar kitchen to cook a Michelin-star meal. Here is how they did it, using simple analogies:

1. The "Swiss Army Knife" vs. The "Custom Tool"

Most top-tier AI weather models are like custom-made Swiss Army knives with specialized blades for every single task (like a specific blade just for wind, another for temperature). These are great, but they are heavy, expensive to build, and hard to maintain.

Otter is different. It uses a standard, off-the-shelf tool (a 2D Swin Transformer) that is already widely used in other fields like language processing and image recognition.

The Analogy: Instead of building a custom engine for a car, Otter takes a highly efficient, mass-produced engine used in millions of cars and tunes it perfectly for driving on weather roads.
The Result: Because it uses standard parts, it runs on hardware that is cheap and common. It doesn't need a supercomputer cluster; it can be trained on a single, powerful graphics card.

2. The "Speed Run" vs. The "Marathon"

Training the biggest AI weather models is like running a marathon with a backpack full of bricks. It takes weeks and costs thousands of dollars in electricity.

Otter's Approach: Otter is like a sprint. It trains in less than 3.5 days on a single powerful computer (an A100 GPU).
The Comparison: The paper claims Otter is 100 times more efficient than the most advanced, resource-heavy models. It achieves results that are nearly as good as the giants, but for a fraction of the cost.

3. The "Crystal Ball" (Deterministic) vs. The "Weather Forecast" (Probabilistic)

The Deterministic Model (The Single Guess): Otter can predict exactly what the weather will be. In tests, this version was 9.6% more accurate than the current standard (IFS HRES) after 24 hours, while costing almost nothing to train.
The Probabilistic Model (The Range of Possibilities): Weather is uncertain. A good forecast shouldn't just say "It will rain"; it should say "There's a 70% chance of rain."
- Usually, making these "range of possibilities" predictions requires even more computing power.
- Otter's Trick: The authors found a clever way to turn their fast, single-guess model into a probabilistic one without rebuilding it from scratch. They used a technique called Monte Carlo Dropout (which is like asking the model to make the same guess 10 times with slightly different "moods" to see how much the answer varies).
- The Result: They created a "Deep Ensemble" (a team of models working together) that is twice as skilled as other lightweight models and beats the massive, expensive "GenCast" model by over 2%, all while using 10 times less computing power.

4. The "Universal Translator" (Cross-Domain Test)

To prove that Otter isn't just a one-trick pony, the authors tested it on a completely different problem: Acoustic Scattering.

The Analogy: Imagine teaching a chef who is great at making soup to suddenly cook a complex chemical reaction in a test tube.
The Result: Otter took this new task "out of the box" (without any special retraining for physics) and crushed the previous state-of-the-art model (Walrus). This suggests that Otter's efficient design isn't just good for weather; it might be a great template for solving all kinds of complex scientific puzzles involving time and space.

The Big Picture

The paper argues that we don't need to rely on a few giant labs to solve the world's weather problems. By stripping away unnecessary complexity and using standard, highly optimized tools, Otter Weather proves that:

Small teams (even those with just one GPU) can build world-class weather models.
Speed matters: You can iterate and improve models much faster when you don't have to wait weeks for training.
Efficiency is key: You can get 95% of the performance of the "super-models" for 1% of the cost.

In short, Otter is the "budget-friendly, high-performance" solution that opens the door for universities, startups, and smaller agencies to create their own weather forecasting tools, rather than just borrowing someone else's.

Technical Summary: Otter Weather

Problem Statement

The field of medium-range weather forecasting is undergoing a paradigm shift from traditional Numerical Weather Prediction (NWP) to data-driven AI models. While state-of-the-art (SOTA) AI models have demonstrated performance comparable to or exceeding NWP baselines, they typically require massive training budgets involving hundreds of GPUs or TPUs. This computational barrier restricts high-performance weather prediction to well-resourced AI laboratories, hindering scientific independence, rapid model iteration, and the ability of under-resourced groups to design models tailored to specific regions or variables. Furthermore, many SOTA architectures rely on heavy, domain-specific inductive biases (e.g., custom Graph Neural Networks or complex spherical geometry operators) that lack the hardware optimization of mainstream Large Language Model (LLM) and computer vision techniques, further inflating training costs.

Methodology

The authors introduce Otter Weather, a highly efficient spatiotemporal forecasting model designed to democratize AI weather prediction by prioritizing general-purpose, hardware-optimized components over bespoke, domain-specific complexity.

Architectural Design

Otter is built upon a 2D Swin Transformer arranged in a symmetric U-Net configuration (Encoder, Bottleneck, Decoder). Key architectural choices include:

Input Representation: All input variables (surface and atmospheric across 13 pressure levels) and time embeddings are concatenated along the channel dimension and tokenized via strided convolutions.
Backbone Modifications: The model maintains a constant token dimensionality ( $D$ ) across the U-Net, replaces standard GeLU MLPs with SwiGLU feed-forward networks, and substitutes relative position biases with 2D Rotary Positional Embeddings (RoPE).
Topology: Unlike models that apply custom attention masks to approximate a cylindrical Earth topology (masking poles), Otter removes masking entirely, treating the topology as a torus to enable cross-boundary communication and simplify implementation.
Optimization: Training utilizes the Muon optimizer, which orthogonalizes parameter updates to accelerate convergence, combined with gradient accumulation to offset matrix-operation overheads at small batch sizes.
Regularization: The model employs strong regularization typical of LLMs, including weight decay ($WD=0.15$), dropout ( $p=0.1$ ), and stochastic depth.

Probabilistic Forecasting

To address predictive uncertainty, the authors extend Otter to probabilistic forecasting by training via the Continuous Ranked Probability Score (CRPS). They explore two pipelines:

Fine-tuning: Retrofitting a deterministic checkpoint with a CRPS objective.
From-scratch: Training a probabilistic model directly with CRPS.
Stochasticity is injected via Monte Carlo (MC) Dropout rather than Adaptive Layer Normalization (AdaLN), leveraging existing dropout layers to avoid parameter overhead. Deep ensembles are constructed efficiently by reusing checkpoints generated during standard hyperparameter tuning (specifically varying learning rates) during the Rollout Fine-Tuning (RFT) phase, rather than training independent models from different random seeds.

Experimental Setup

Dataset: ERA5 reanalysis data (1979–2019 for training, 2020 for evaluation) at a spatial resolution of 1.5°.
Task: Predicting atmospheric states at a 6-hour lead time, with longer horizons generated via autoregressive rollout.
Metrics: Deterministic performance is measured via latitude-weighted Root Mean Squared Error (RMSE); probabilistic performance via CRPS and Spread-Skill Ratio (SSR).
Cross-Domain Validation: The architecture was also applied to an acoustic scattering Partial Differential Equation (PDE) task from the Well benchmark to test generalizability.

Key Contributions

Democratizing Deterministic Weather: Otter establishes a new skill-compute Pareto frontier. The base deterministic model outperforms the operational IFS HRES baseline by 9.6% at a 24-hour lead time while requiring fewer than 3.5 A100-days for training. The scaled Otter-XL achieves a 14.9% improvement over IFS HRES with ~10 A100-days, matching the performance of industrial-scale GraphCast (which requires ~130x more compute) while using standard transformer components.
Low-Compute Probabilistic Forecasting: The authors demonstrate that architectural efficiency transfers to probabilistic settings. The base Otter probabilistic model achieves competitive results with a total budget under 8 A100-days, outperforming resource-intensive baselines like ArchesWeatherGen and GenCast while using an order of magnitude less compute.
Cross-Domain Generality: When applied to the acoustic scattering PDE task without domain-specific tuning, Otter outperformed the SOTA foundation model Walrus, suggesting that the design principles (Muon, SwiGLU, RoPE, balanced backbones) are transferable across physical systems.
Practical Guide and Ablation Study: The paper provides a systematic ablation study challenging the necessity of Earth-specific geometric priors. It documents that standard, hardware-optimized components are sufficient for high-skill forecasting and offers a practical training recipe for resource-constrained practitioners.

Results

Deterministic Performance: At 1.5° resolution, Otter (Base) improves RMSE by 9.6% over IFS HRES. Otter-XL improves this to 14.9%, surpassing GraphCast's 14.6% gain while using ~130x less training compute.
Probabilistic Performance: Otter-XL (Deep Ensemble) achieves a 9.7% CRPS improvement over the IFS ENS baseline with ~30 A100-days of compute. This represents nearly a two-fold skill increase over comparable lightweight models and outperforms GenCast by over 2% with a 10-fold reduction in compute.
Efficiency: The deterministic model trains in under 3.5 A100-days on a single GPU. Probabilistic deep ensembles are generated "for free" by repurposing hyperparameter tuning checkpoints, avoiding the >20 A100-day costs associated with other efficient models.
Ablation Insights: The study found that balanced, deeper U-Net configurations outperform wide/shallow ones. Muon consistently outperformed AdamW, especially in short training regimes. Omitting attention masking (torus topology) incurred negligible performance drops. Techniques like Mixture of Experts (MoE) and Neighborhood Attention were found to offer marginal gains that did not justify their complexity.

Significance

The paper argues that highly competitive global weather forecasting does not require massive compute budgets or rigid, domain-specific inductive biases. By integrating general-purpose advances from the LLM and vision communities, Otter Weather pushes the skill-compute Pareto frontier for both deterministic and probabilistic modeling.

The significance of this work lies in its potential to democratize weather forecasting. It frees researchers from dependence on industry-scale compute and the architectural constraints of large foundation models, enabling task-specific training from scratch and rapid iteration cycles. The authors posit that a healthy forecasting ecosystem requires not only expensive, general-purpose foundation models but also lightweight, cheaply trainable alternatives. This diversity is essential for scientific sovereignty (allowing adaptation to specific institutional needs) and operational robustness (aggregating systems with different failure modes to reduce correlated biases). The successful transfer of these principles to PDE tasks further suggests a path toward efficient, domain-agnostic modeling across the physical sciences.