Earth Embeddings as Products: Taxonomy, Ecosystem, and Standardized Access

This paper addresses the fragmentation of pre-computed Geospatial Foundation Model embeddings by proposing a three-layer taxonomy and extending TorchGeo with a unified API to standardize access, thereby enabling interoperable, reproducible, and accessible Earth observation workflows.

The Gist

Imagine the Earth as a giant, incredibly complex library. For years, scientists have been trying to read every book in this library to understand our planet's climate, forests, and cities.

Recently, a new technology called Geospatial Foundation Models (GFMs) arrived. Think of these models as super-intelligent librarians who can read a book and instantly summarize its entire story into a single, short "ID card" (a vector of numbers). These ID cards are called Earth Embeddings.

However, there's a huge problem: The library is a mess.

The Problem: A Library with No Catalog

Right now, the ecosystem of these "Earth ID cards" is chaotic.

• Different Formats: Some librarians write their ID cards on sticky notes, others on index cards, and some on digital tablets that only work on specific computers.
• Different Resolutions: Some cards describe the whole city, while others describe just a single street corner.
• Hard to Find: To use a card, you often have to build your own special machine to read it. If you want to compare the "City Card" from Librarian A with the "Street Card" from Librarian B, you can't do it easily because they speak different languages.

This makes it very hard for regular people (practitioners) to use these powerful tools. It's like trying to build a house when every brick comes from a different factory with a different shape and no instructions on how to stack them.

The Solution: A Universal Translator

The authors of this paper, a team of researchers from Sweden, Germany, and the US, decided to fix this mess. They did three main things:

1. They Drew a Map (The Taxonomy)

They organized the chaotic library into a neat three-story building:

• The Data Floor (The Cards): They sorted the ID cards by how detailed they are.
  • Location Cards: Just tell you "Where" (like a GPS coordinate).
  • Patch Cards: Summarize a whole photo (like a "vibe check" of a neighborhood).
  • Pixel Cards: Describe every single dot in an image (like a high-definition map of every tree).
• The Tool Floor (The Readers): They looked at the tools people use to test these cards, like scoreboards and competitions.
• The Value Floor (The Uses): They looked at what people actually do with these cards, like finding similar forests or mapping poverty.

2. They Built a Universal Adapter (TorchGeo)

This is the most important part. The team updated a popular software library called TorchGeo.

• Before: If you wanted to use a specific Earth ID card, you had to download 4 different software packages, write 100 lines of code, and hope your computer didn't crash.
• After: With their new update, you can load any of these different ID cards using just one simple command, like plugging a USB drive into a computer. It doesn't matter if the card came from Google, Clay, or a university; the software treats them all the same way.

They turned these complex data products into "first-class citizens," meaning they are now as easy to use as a standard photo or a spreadsheet.

3. They Wrote a User Manual (The Survey)

They created a giant table (Table I and II in the paper) that acts like a menu for a restaurant. It tells you:

• What ingredients (data) were used to make the dish?
• Who owns the recipe (license)?
• Can you cook it yourself (reproducibility)?
• How big is the portion (resolution)?

Why This Matters

Imagine you are a farmer trying to predict crop yields.

• Without this paper: You spend months trying to figure out how to download the data, fix the file formats, and write code just to get the data to load. You might give up.
• With this paper: You open your software, click "Load Earth Embeddings," and instantly see a map of similar fields from around the world. You can immediately start solving your problem.

The Future

The authors are also shouting out a "To-Do List" for the future:

• Look at the Oceans: Most cards only look at land; we need cards for the sea and sky too.
• Be Clearer: We need to know why the computer made a certain ID card (explainability).
• Better File Types: Stop using messy, old file formats; use modern "cloud-native" formats that work better on the internet.

In short: This paper takes a scattered, confusing pile of high-tech Earth data and organizes it into a clean, accessible, and easy-to-use system, allowing anyone to finally unlock the secrets hidden in our planet's data.

Technical Summary

1. Problem Statement

The rapid emergence of Geospatial Foundation Models (GFMs) has produced powerful "Earth Embeddings" (low-dimensional vectors capturing semantic, spatial, and temporal information). However, the ecosystem for these derived products is highly fragmented, creating significant barriers for practitioners:

• Incompatibility: Embeddings exist in a chaotic mix of file formats (GeoParquet, GeoTIFF, NumPy, PyTorch), spatial resolutions, and licensing terms.
• Engineering Bottleneck: There is no unified methodology to load, query, or evaluate these products. Researchers must write custom loaders for each product, often dealing with missing geospatial metadata or incompatible coordinate systems.
• Reproducibility Issues: Many embedding products lack clear provenance regarding the training data, model weights, or inference pipelines used to generate them.
• Conceptual Confusion: There is a lack of distinction between the dynamic foundation models (requiring active inference) and static pre-computed embedding products ("frozen" assets), leading to invalid performance comparisons.

2. Methodology

The authors employ a three-pronged approach to address these challenges:

A. Taxonomy and Landscape Analysis
The authors formalize the Earth Embedding landscape into a three-layer taxonomy (visualized in Fig. 1):

1. Data (Embeddings): Categorized by spatial granularity:
   • Location Embeddings: Encode coordinates $\{x, y, t\}$ independent of imagery (e.g., SatCLIP).
   • Patch-level Embeddings: Single vectors summarizing image chips (e.g., Clay, Major TOM), optimized for retrieval.
   • Pixel-level Embeddings: Spatially preserved feature maps where every pixel has a vector (e.g., Presto, Google Satellite), optimized for segmentation.
2. Tools (Analysis Frameworks): Includes benchmarks (e.g., NeuCo-Bench) and challenges (e.g., Embed2Scale) used to evaluate embedding quality.
3. Value (Downstream Applications): Practical use cases ranging from land cover mapping to poverty mapping and retrieval.

B. Systematic Survey
The authors curated a repository of seven existing embedding products (as of late 2025), analyzing them across:

• Spatial/Temporal Extent: Global vs. regional coverage; snapshot vs. annual time series.
• Technical Specs: Resolution, vector dimensions, data types, and compression strategies.
• Reproducibility: A detailed audit of the model architecture (e.g., ResNet, ViT, DINOv2), self-supervised learning (SSL) techniques (e.g., MAE, Contrastive, Distillation), training data sources, and license compatibility (Code, Weights, Data).

C. Technical Integration (TorchGeo)
To solve the access bottleneck, the authors extended the TorchGeo library. They implemented:

• Unified API: Standardized data loaders for diverse embedding products, treating them as "first-class" geospatial datasets.
• Interoperability: Support for both search/retrieval tasks (patch-based) and dense prediction tasks (pixel-based).
• Code Simplification: Reducing complex, multi-repo workflows into concise, reproducible code blocks (e.g., 20 lines of code for retrieval tasks).

3. Key Contributions

1. Comprehensive Taxonomy & Metadata Atlas: The paper provides the first structured classification of Earth Embeddings, offering a detailed metadata table (Table I) covering resolution, license, and temporal extent for major products like Clay, Major TOM, Earth Index, Copernicus-Embed, Presto, Tessera, and Google Satellite Embeddings.
2. Reproducibility Audit: Table II offers a critical breakdown of the "black box" nature of many products, highlighting which models are open-source, which training data is available, and which licenses apply to weights vs. data.
3. Unified Software Framework: The integration of these embeddings into TorchGeo allows users to access heterogeneous data sources via a single interface. This decouples downstream analysis from model-specific engineering.
4. Design Principles for Future Products: The authors propose four critical guidelines for future embedding releases:
   • Input Diversity: Move beyond Sentinel-1/2 to include oceans, atmosphere, and hyperspectral data.
   • Explainability: Improve uncertainty quantification and provenance tracking.
   • Cloud-Native Formats: Adopt formats like COG, GeoZarr, or GeoParquet that preserve geospatial metadata, avoiding raw .pt/.npy files.
   • Standardized Benchmarks: Release weights and use common benchmarks to enable fair comparison.

4. Results and Demonstrations

The paper demonstrates the efficacy of the unified approach through two code examples (Listings 1 & 2):

• Search/Retrieval: Using the Earth Index Embeddings, the authors successfully implemented a cosine similarity search to find locations with similar semantic features to a query Sentinel-2 image. This was achieved without writing custom loaders for the specific embedding format.
• Land Cover Mapping: Using Google Satellite Embeddings and the EuroCrops dataset, the authors demonstrated a workflow for spatiotemporal intersection and k-NN clustering for crop type prediction.
• Outcome: The integration successfully abstracted away the complexities of file formats (e.g., handling upside-down rasters in Google data, vector vs. raster formats) and licensing constraints, enabling "fair, side-by-side benchmarking."

5. Significance

This work is significant because it bridges the gap between theoretical research (foundation models) and operational application (geospatial analysis).

• Democratization: By standardizing access, it lowers the barrier to entry for practitioners who cannot afford the compute costs of running foundation models from scratch.
• Scientific Rigor: It addresses the "reproducibility crisis" in geospatial AI by formalizing the distinction between dynamic models and static products and providing a framework for consistent evaluation.
• Ecosystem Maturity: The paper signals a shift from a fragmented research landscape to a structured product ecosystem, paving the way for transparent, scalable, and comparable Earth observation workflows.