🔬 physics

Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling

This paper demonstrates that Earth observation foundation model embeddings (TESSERA) serve as effective, transferable sub-grid descriptors that significantly improve the accuracy of probabilistic weather downscaling for temperature and wind speed across diverse regions and forecasting models by encoding persistent surface properties that coarse global grids fail to resolve.

Original authors: Pedro Sousa (Department of Computer Science, University of Cambridge), Will Tebbutt (Department of Engineering, University of Cambridge), Sadiq Jaffer (Department of Computer Science, University of Ca

Published 2026-08-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pedro Sousa (Department of Computer Science, University of Cambridge), Will Tebbutt (Department of Engineering, University of Cambridge), Sadiq Jaffer (Department of Computer Science, University of Cambridge), Robin Young (Department of Computer Science, University of Cambridge), Anil Madhavapeddy (Department of Computer Science, University of Cambridge), Richard E. Turner (Department of Engineering, University of Cambridge)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather for your specific backyard. You have a super-smart computer model that looks at the sky over a huge area, say a whole state, and gives you a forecast. But this model sees the world in big, fuzzy blocks, like a low-resolution video game. It knows the average temperature for a 25-kilometer square, but it doesn't know that your backyard is right next to a dense forest, or that your neighbor's house is built on a steep hill, or that the city heat is baking the pavement nearby. These tiny, local details are what we call "sub-grid" effects. They are the difference between a generic forecast and a prediction that actually tells you if you need a jacket or sunglasses.

For a long time, scientists have tried to fix this blurry picture by hand-crafting rules. They would tell the computer, "If the location is high up, make it colder," or "If it's near water, make it windier." It's like trying to describe a complex painting by only listing the colors used. But the real world is messy and full of surprises. What if, instead of giving the computer a list of rules, we gave it a "smart camera" that had already learned to recognize the texture of the world? This is the big question: Can we use a massive, pre-trained AI that has studied satellite photos of the entire Earth to understand the local ground, and then use that understanding to sharpen our weather forecasts?

This paper says "yes," and it does so with a clever trick. The authors took a powerful Earth observation tool called Tessera, which is like a super-advanced map that has "seen" the Earth's surface in high definition for a whole year. Tessera doesn't just see a pixel; it understands the vibe of that pixel—whether it's a forest, a city, a snowy field, or a rocky cliff. The researchers compressed this massive knowledge into a tiny, 16-number "ID card" for every spot on the ground.

They then fed these ID cards into a weather downscaling model (a system designed to turn the big, fuzzy forecast into a sharp, local one). The results were surprisingly good. When they tested this on 2-meter temperature and 10-meter wind speed across five very different parts of the world (from Europe to Australia), the model with the "smart camera" ID cards got much better at guessing the weather. Specifically, it improved the accuracy of temperature predictions by 11.5% and wind speed predictions by 6.2% compared to models that only used basic height maps.

Here is the really cool part: the paper found that these two types of weather need different kinds of help. For temperature, the old-school method of just looking at how high a place is (elevation) works pretty well. The "smart camera" didn't add much new texture there; it mostly just acted as a safety net when there weren't enough weather stations nearby to learn from. But for wind, the story is different. Wind is tricky; it gets blocked by trees, funneled through valleys, and slowed down by buildings. The basic height map couldn't capture this well. The "smart camera" ID cards, however, were fantastic at explaining why the wind was blowing the way it was. They provided the missing details about the ground's surface that the simple height maps missed.

The authors also tested if this trick works when the weather forecast itself is imperfect (like a real-world forecast made days in advance, not just a perfect historical record). It does! The improvements held up even when the input data was a forecast from a different AI model called Aurora. Perhaps most impressively, they simulated a scenario where a new weather station network is being built in Norway. Before any of those new stations had collected a single day of data, the model using the "smart camera" ID cards was already predicting wind speeds better than the old methods. It was able to say, "This spot in Norway looks a lot like a spot in Europe we've already studied, so the wind should behave similarly," effectively borrowing knowledge from the past to predict the future.

In short, this paper suggests that we don't need to reinvent the wheel to understand local weather. By letting a pre-trained AI that has already "seen" the Earth describe the ground, we can make our weather forecasts sharper, more accurate, and much better at handling the tiny, local details that matter most to us. It's like upgrading from a blurry map to a high-definition GPS that knows exactly what your neighborhood looks like.

Technical Summary: Earth Observation Embeddings as Sub-Grid Descriptors for Probabilistic Weather Downscaling

Problem Statement
Global weather reanalyses and forecasts, such as ERA5, operate on coarse spatial grids (approximately 25 km), which limits their ability to resolve local weather variability driven by sub-grid processes. Near-surface conditions at specific locations often deviate significantly from the coarse-grid cell average due to unresolved terrain, land cover, surface roughness, and built structures. While statistical downscaling methods exist to bridge this gap, they typically rely on hand-crafted topographic descriptors (e.g., elevation, slope) or require dense local observational history. The challenge addressed by this work is to improve off-grid, probabilistic downscaling at arbitrary, previously unobserved locations without relying on local station history, by leveraging transferable representations of the local surface environment.

Methodology
The authors propose augmenting a Convolutional Conditional Neural Process (ConvCNP), a translation-equivariant model designed for mapping coarse fields to predictive distributions at arbitrary points, with a learned local surface descriptor derived from Earth Observation (EO) foundation models.

Data Sources:
- Atmospheric Input: Coarse 0.25° (∼25 km) ERA5 reanalysis fields (and later, Aurora AI forecasts) containing 20 dynamic variables and 13 static surface properties.
- Targets: Instantaneous 2 m temperature and 10 m wind speed from the GHCNh station network across five climatically diverse regions (Europe, US, East Asia, Southern Africa, Australia).
- Surface Representation: Tessera, a self-supervised foundation model trained on a year of Sentinel-1 and Sentinel-2 imagery, providing 128-dimensional embeddings for every 10 m pixel.
Patch Compression (VAE):
A single Tessera pixel (10 m) is too local to capture the full range of surface properties affecting a weather station. To address this, the authors pre-train a Variational Autoencoder (VAE) to compress a 64×64 pixel patch (approx. 640 m) of Tessera embeddings into a compact 16-dimensional latent vector ( $z_T$ ). This vector captures terrain context, land-cover heterogeneity, and coastal-inland transitions without requiring the downscaler to learn this bottleneck from sparse station data.
Downscaling Architecture:
The ConvCNP processes the coarse atmospheric grid $Z$ through a convolutional encoder. The resulting features are evaluated at an off-grid target location $x^*$ via a smooth kernel. This representation is concatenated with two per-location descriptors:
- $e$ : A hand-crafted topographic descriptor (elevation, $\Delta$ elevation, and multi-scale topographic position index).
- $z_T$ : The compressed Tessera surface descriptor.
  A Multi-Layer Perceptron (MLP) maps this combined input to the parameters ( $\mu, \sigma$ ) of a predictive distribution (Gaussian for temperature, Truncated Normal for wind speed).
Training and Evaluation:
The model is trained via maximum likelihood on historical data (2010–2020) with a strict hold-out strategy: 15% of stations are withheld entirely from training, and evaluation occurs on future years (2022). This tests generalization to unseen locations and times.

Key Contributions

Learned Sub-Grid Descriptor: The construction of a compact, task-agnostic local surface descriptor from EO foundation model embeddings, compressing high-resolution surface dynamics into a vector suitable for weather downscaling.
Consistent Skill Gains: Demonstration that injecting the Tessera descriptor into the ConvCNP improves both point accuracy (MAE, RMSE) and probabilistic skill (CRPS) across five regions for both temperature and wind speed.
Variable-Specific Mechanisms: Qualitative differentiation of how the embedding aids different variables. For temperature, the embedding acts primarily as a transferable land-surface prior when station data is sparse, as elevation explains most sub-grid structure. For wind speed, the embedding organizes residual structure more effectively than topography alone, capturing missing land-surface information.
Robustness to Forecast Inputs: Validation that the method remains effective when the coarse input is changed from reanalysis (ERA5) to medium-range AI forecasts (Aurora), with benefits persisting across forecast leads up to 72 hours.
Sample Efficiency: Evidence that the embedding improves downscaling performance for newly deployed station networks before any local observations are available, outperforming baselines that rely solely on accumulated local data.

Experimental Results

Skill Improvement: Across all regions and variables, the model with Tessera outperformed the baseline (ConvCNP with topography only). Overall, CRPS improved by 11.5% for 2 m temperature and 6.2% for 10 m wind speed.
Sparse vs. Dense Regions: For temperature, the improvement was most pronounced in regions with sparse station networks, suggesting the embedding compensates for the lack of local training data. For wind speed, improvements were consistent regardless of station density.
Dense Map Analysis: In dense downscaling experiments (5 km resolution), the Tessera model introduced significantly more fine-scale amplitude for wind speed (3–4× increase in Iberia and Norway) compared to the baseline, while temperature fields remained largely elevation-driven.
Residual Analysis: Random forest probes on ERA5-interpolation residuals showed that topography best organizes temperature residuals, whereas Tessera embeddings best organize wind speed residuals.
Cold Start Performance: In a simulated Norwegian deployment, the Tessera-enhanced model reduced wind speed MAE by ~15–17% at "cold start" (zero local observations) compared to the baseline, a gap that persisted even after six years of local data accumulation.

Significance and Claims
The paper claims to provide the first evidence that long-timescale Earth observation embeddings can support short-timescale weather downscaling where sub-grid departures are systematically structured by persistent surface properties. The authors position this work as establishing EO embeddings as a practical interface between global weather models and site-specific forecasting.

The significance lies in the ability to compose two independently trained foundation models: one representing the evolving atmospheric state (Aurora/ERA5) and the other representing the local surface (Tessera). This composition yields more skillful local predictions than either coarse forecasts or topography-conditioned downscalers alone, particularly in scenarios where local observational history is unavailable. The authors note that while the method improves sample efficiency and generalization, it does not claim to replace physical modeling or solve all downscaling challenges, such as precipitation or temporally aggregated quantities, which require separate investigation.

More like this