🤖 AI

SLED: Scalable Location Encoding via Distillation

The paper introduces SLED, a scalable and lightweight distillation-based framework that efficiently learns high-quality, multimodal location embeddings from diverse geospatial data using small batch sizes, thereby overcoming the computational and scalability limitations of existing state-of-the-art location encoders while achieving superior performance across numerous benchmark tasks.

Original authors: Kevin Lane, Zhongying Wang, Esther Rolf, Morteza Karimzadeh

Published 2026-08-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kevin Lane, Zhongying Wang, Esther Rolf, Morteza Karimzadeh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Earth as a giant, living library where every single spot on the planet—from the peak of Mount Everest to a quiet corner of a city park—has a story to tell. For decades, scientists have been trying to read these stories using "Earth Observations," which are just fancy photos and scans taken from satellites. These images are like massive, high-definition textbooks, but they are so huge and come in so many different languages (some show light, some show radar, some show heat) that it's incredibly hard to organize them. To make sense of this, researchers created "location encoders." Think of these as magical address books that can instantly tell you everything about a place just by knowing its latitude and longitude, without you even needing to look at a photo.

However, building these magical address books has been a bit like trying to fill a swimming pool with a teaspoon. The old methods required massive amounts of computer power and huge groups of data to work, often getting confused when similar-looking places were treated as different. They also struggled to mix different types of satellite "languages" together without forcing them to match up perfectly in time and space, which is a tedious and expensive process. This is where a new idea comes in: what if we could teach a small, smart student to learn from a giant, already-smart teacher? This is the core of a new approach called "distillation," where a smaller model learns by copying the answers of a larger one, making the whole process faster, cheaper, and much more flexible.

Enter SLED (Scalable Location Encoding via Distillation), a new framework developed by researchers at the University of Colorado Boulder that changes the game for how we teach computers to understand our planet. Instead of the old, clunky method of forcing different satellite images to line up perfectly like puzzle pieces, SLED treats the location itself (the latitude and longitude) as the "glue" that holds everything together. Imagine you are at a party where everyone speaks a different language. The old way required everyone to translate their sentences into a single common language before they could talk. SLED, however, acts like a super-smart translator who listens to the location of the person speaking and instantly understands the meaning, regardless of the language they are using. This allows the system to learn from a mix of different satellite sensors—like the optical cameras of Sentinel-2, the radar eyes of Sentinel-1, and the Landsat satellites—without needing them to be perfectly synchronized.

The paper shows that this new method is a massive leap forward in efficiency. While previous state-of-the-art models needed to process huge batches of 16,000 to 32,000 images at once to learn, SLED can learn effectively with batches as small as 128. This is like going from needing a stadium full of people to solve a math problem to needing just a single classroom. The result is that SLED can be trained up to 47 times faster than current top models, using a fraction of the computing power and time.

In their experiments, the researchers tested SLED on a diverse set of 19 different tasks, ranging from predicting climate variables like frost frequency and growing seasons to classifying land cover and estimating population density. They found that SLED not only kept pace with the best existing models but often outperformed them, especially when trained on a mix of different satellite data. For instance, on tasks related to climate and elevation, the multimodal version of SLED (trained on three different types of satellite data) showed significant improvements. Interestingly, while adding radar data (Sentinel-1) didn't always boost performance on every single task—likely because radar images have fewer "colors" or bands than optical images—it did help the model perform better on specific challenges like the SustainBench tasks, which measure things like sanitation and water access.

The authors suggest that this approach opens the door to a more flexible future for geospatial AI. By using location as a "binding modality," SLED eliminates the need for the costly and subjective process of aligning samples from different sensors. This means researchers can easily add new types of data in the future without having to rebuild the entire system. The paper concludes that while there is still work to be done, particularly in understanding how to best combine different types of data, SLED proves that distillation is a powerful and scalable strategy for creating high-quality, general-purpose representations of our planet, making advanced Earth observation accessible to more people and more applications than ever before.

Technical Summary: SLED: Scalable Location Encoding via Distillation

Problem Statement

While geospatial foundation models (GeoFMs) offer powerful representations of Earth Observations (EO), their direct application is often hindered by computational and logistical constraints. Generating embeddings for downstream tasks typically requires acquiring and processing massive volumes of raw satellite data (e.g., 1.7 TB/day from Sentinel-2) and running inference through heavy pretrained models.

To address this, location encoders have emerged as a solution. These models learn to generate embeddings directly from latitude and longitude, eliminating the need for input imagery at inference time. However, current state-of-the-art (SOTA) location encoders suffer from three primary limitations:

Computational Cost: They rely on contrastive learning frameworks (e.g., InfoNCE loss) similar to CLIP, necessitating massive batch sizes (16K–32K) to function effectively, creating significant compute bottlenecks.
False Negatives: Contrastive losses treat all non-matching samples in a batch as negatives. For location encoders, this is problematic because geographically proximate locations are inherently similar; treating them as negatives introduces noise and degrades performance.
Multimodal Constraints: Existing multimodal approaches require strict spatiotemporal coregistration of samples across different sensor modalities (e.g., matching Sentinel-1, Sentinel-2, and Landsat images for the exact same location and time). This process is costly, subjective, and excludes vast amounts of data where perfect alignment is unavailable.

Methodology

The authors propose SLED (Scalable Location Encoder via Distillation), a framework that shifts from contrastive learning to a distillation-based approach.

Core Framework

SLED treats geospatial location (latitude and longitude) as a "binding modality." Instead of contrasting location embeddings against image embeddings, SLED uses Mean Squared Error (MSE) loss to distill knowledge from frozen, modality-specific "teacher" GeoFMs into a trainable "student" location encoder.

Single-Modality Training: The student location encoder receives coordinates and is trained to minimize the MSE between its output and the embedding of a corresponding image produced by a frozen teacher model (e.g., TerraFM for Sentinel-2, FG-MAE for Sentinel-1, MoCo for Landsat).
Multimodal Training: Inspired by ImageBind, SLED treats location as the single binding modality. For $K$ $K$ modalities, the framework introduces $K$ $K$ parallel linear layers (alignment heads) on top of the shared position encoder.
- Decoupling: These heads decouple the latent space size of the student from the varying dimensions of the teacher encoders.
- Independence: Crucially, this architecture allows for independent sampling. A single batch can contain a Sentinel-2 image for location A, a Landsat image for location B, and a Sentinel-1 image for location C. There is no requirement for spatiotemporal coregistration between the modalities within a batch.

Training Specifications

Efficiency: SLED achieves strong performance with batch sizes as small as 128, a fraction of the 16K–32K required by contrastive methods.
Position Encoding: The framework utilizes Random Fourier Features (RFF) for positional encoding, though ablations were conducted with Spherical Harmonics and SirenNet.
Data: The authors pretrain on 100K total samples, utilizing Sentinel-2 (S2-100K), Landsat 8/9 (SSL4EO-L), and a custom Sentinel-1 dataset. In multimodal settings, subsets of these datasets are used to maintain a constant total sample count.

Key Contributions

Distillation for Location Encoding: The paper demonstrates that unimodal and multimodal EO data can be distilled into performant location embeddings using MSE loss with small batch sizes (128), eliminating the need for large-batch contrastive training.
Scalability and Speed: The distillation framework accelerates pretraining by up to 47x compared to SOTA location encoders (e.g., SatCLIP, GeoCLIP) by removing the need for massive batches and complex negative sampling.
Modality-Agnostic Multimodal Learning: By using location as a binding modality, SLED eliminates the need for costly spatiotemporal coregistration. It can flexibly incorporate new modalities by simply adding a frozen teacher and an alignment head, without requiring the training data to be perfectly aligned across sensors.
Open Resources: The authors release a pip module (sled-geo), pretrained model weights, and a new benchmark dataset for climate-related variables and a uniform-at-random Sentinel-1 pretraining dataset.

Results

The authors evaluated SLED on 19 human-centric benchmark tasks across four suites: CHELSA (climate regression), TorchSpatial (classification and regression), and SustainBench (social indices).

Performance vs. SOTA:
- CHELSA: SLED models outperformed existing location encoders (SatCLIP, GeoCLIP) on 7 out of 8 climate regression tasks.
- TorchSpatial: SLED achieved performance on par with SOTA for classification tasks. For regression, it showed significant gains in elevation mapping but lagged slightly in forest cover.
- SustainBench: SLED significantly outperformed SatCLIP but fell short of GeoCLIP. The authors attribute GeoCLIP's lead to its training on ground-level imagery, which is more relevant for the survey-based targets in SustainBench, whereas SLED relies on satellite-derived teachers.
Multimodal Benefits:
- Adding Landsat to Sentinel-2 training generally improved metrics, particularly on regression tasks.
- Adding Sentinel-1 (SAR) showed mixed results: it provided gains on SustainBench tasks but sometimes caused a slight performance dip on other tasks. The authors suggest this may be due to the lower channel count of SAR imagery compared to multispectral data or the maturity of SAR encoders.
Efficiency: SLED reduced pretraining time per epoch from ~3–5 hours (for SOTA models) to 0.105 hours while using a batch size of 128.

Significance

The paper claims that SLED represents a paradigm shift in how location encoders are trained and scaled. Its primary significance lies in democratizing and accelerating geospatial representation learning:

Accessibility: By removing the dependency on massive batch sizes and expensive contrastive infrastructure, SLED makes high-quality location encoding accessible to researchers with limited computational resources.
Flexibility: The "binding modality" approach allows for the seamless integration of diverse, unaligned geospatial data sources. This removes the subjective and expensive bottleneck of spatiotemporal coregistration, enabling the use of the vast, heterogeneous archives of Earth Observation data that were previously difficult to leverage jointly.
Future-Proofing: The modular design (frozen teacher + alignment head) suggests a scalable path for future multimodal learning, where new sensors can be added without retraining the entire system or curating perfectly aligned datasets.

The authors conclude that while distillation may not always match the performance of models trained on perfectly aligned, ground-level data for specific survey-based tasks, it offers a superior, scalable, and efficient approach for general-purpose geospatial representation learning.