Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications

Prithvi-EO-2.0 is an open-source, multi-temporal geospatial foundation model trained on 4.2 million global time series samples that significantly outperforms its predecessor and other existing models across diverse Earth observation tasks, from disaster response to ecosystem monitoring, through the integration of temporal and location embeddings and a Trusted Open Science development approach.

The Gist

Imagine the Earth as a giant, living movie screen that has been filming itself for decades. Every day, satellites fly overhead, snapping billions of photos of forests, cities, oceans, and farms. The problem? There is too much footage. It's like having a library with millions of movies but no one to watch them or tell you what's happening in them.

Enter Prithvi-EO-2.0. Think of it not just as a computer program, but as a super-intelligent, super-attentive student who has studied this entire library of Earth movies.

Here is the story of how this student was trained and what it can do, explained simply:

1. The Old Student vs. The New Super-Student

The paper introduces Prithvi-EO-2.0 as the "big brother" of an earlier model called Prithvi-EO-1.0.

• The Old Student (1.0): Was smart, but only studied movies from the United States and didn't pay much attention to time. It looked at a picture of a forest and said, "That's a forest."
• The New Student (2.0): Is a global scholar. It has studied 4.2 million time-series clips from all over the world. Crucially, it learned to watch the changes over time. It doesn't just see a forest; it sees the forest turning green in spring, losing leaves in autumn, and recovering after a fire. It also knows where it is looking (latitude/longitude) and when (season/year).

The Analogy: If the old model was a tourist who took a single photo of a beach and forgot about it, the new model is a lifeguard who has watched that same beach for ten years, knowing exactly how the tides change, when storms hit, and how the sand shifts.

2. How Was It Trained? (The "Self-Taught" Method)

Usually, to teach a computer to recognize things, humans have to label every single photo (e.g., "This is a fire," "This is a flood"). This is slow and expensive.

Prithvi-EO-2.0 used a trick called Self-Supervised Learning (specifically, a method called "Masked Autoencoders").

• The Game: Imagine showing the student a video of a forest, but you cover up 75% of the screen with a black blanket.
• The Challenge: The student has to guess what's under the blanket based on the tiny sliver of the image they can see and the time of year.
• The Result: By playing this game millions of times, the student learned the "grammar" of the Earth. It learned that green usually means plants, that water reflects light differently than soil, and that crops grow in specific patterns.

3. What Can It Do? (The "Swiss Army Knife")

The paper tests this student on three very different types of "exams" to see how versatile it is:

A. The Emergency Responder (Disaster Response)

• The Task: Spotting floods, wildfires, and landslides.
• The Analogy: Imagine a firefighter who can look at a satellite photo and instantly say, "The water is rising here," or "The fire burned this specific patch of trees."
• The Result: Prithvi-EO-2.0 is better at finding the "bad guys" (floods and fires) than the old models, even when the data is messy or cloudy. It's like having a detective who can find a needle in a haystack even if the haystack is on fire.

B. The Farmer and Land Manager (Mapping)

• The Task: Counting crops, identifying land types, and tracking how cities grow.
• The Analogy: Think of a farmer who needs to know exactly which field has corn and which has soybeans, even if they haven't visited the farm in months.
• The Result: The model can distinguish between different types of crops and land use with high accuracy, even when trained on very little data. It's like a botanist who can identify a plant just by looking at a blurry photo of its shadow.

C. The Climate Scientist (Ecosystems)

• The Task: Measuring how much carbon trees are absorbing (Biomass) and how much food plants are making (Gross Primary Productivity).
• The Analogy: This is like a doctor taking an X-ray of the Earth to see how "healthy" the lungs (forests) are and how much oxygen they are producing.
• The Result: The model can estimate these complex numbers using just satellite photos, doing a better job than traditional methods that require expensive ground sensors.

4. Why Is This a Big Deal?

• It's Fast and Cheap: Because the model learned so much during its "training" phase, you don't need thousands of labeled examples to teach it a new task. You can teach it a new job with just a few examples (like teaching a human a new language after they already know the grammar).
• It's Open Source: The creators (a team from NASA, IBM, universities, and more) didn't hide this student. They released the "brain" (the code and weights) for free on the internet so anyone can use it.
• It's a Team Effort: Real-world experts (like actual firefighters and farmers) helped design the tests. This ensures the model isn't just smart in a lab, but actually useful in the real world.

The Bottom Line

Prithvi-EO-2.0 is a universal translator for Earth's data. It takes the chaotic, massive stream of satellite images and turns it into clear, actionable knowledge. Whether you are trying to save a city from a flood, help a farmer grow more food, or track climate change, this model acts as a powerful, all-seeing assistant that helps us understand our planet better than ever before.

Technical Summary

Here is a detailed technical summary of the paper "Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications."

1. Problem Statement

Earth Observation (EO) has entered a new era with Geospatial Foundation Models (GFMs), yet current models face three critical limitations:

• Lack of Multi-Temporal Modeling: Most GFMs treat satellite imagery as static 2D images or process only point data/short sequences, failing to capture long-term trends and seasonal dynamics inherent in EO data.
• Insufficient Validation: There is a lack of rigorous, standardized benchmarking across diverse tasks and resolutions, making it difficult for users to assess model suitability for specific use cases.
• Adoption Barriers: Adapting state-of-the-art GFMs to new applications often requires significant AI expertise. The lack of streamlined tools, clear instructions, and accessible codebases hinders broader community adoption.

2. Methodology

A. Dataset Construction (HLS Global)

• Source: The model was pre-trained on NASA's Harmonized Landsat and Sentinel-2 (HLS) archive, which unifies data from Landsat 8/9 and Sentinel-2A/B at 30m resolution.
• Scale: The dataset comprises 4.2 million global time-series samples spanning a decade (2014–2023).
• Sampling Strategy:
  • Stratification: Samples were selected to ensure diverse representation of Land Use/Land Cover (LULC) classes and ecoregions, with specific oversampling of urban areas and heterogeneous landscapes.
  • Temporal Sequences: Unlike previous models using single timestamps, Prithvi-EO-2.0 uses sequences of 4 images with intervals of 1–6 months to capture seasonal patterns and long-term changes.
  • Quality Control: Rigorous filtering removed cloudy pixels (>20%) and missing data, with interpolation applied for minor gaps.
• Metadata: The dataset includes geolocation (latitude/longitude) and temporal metadata (year, day-of-year) for every sample.

B. Model Architecture

Prithvi-EO-2.0 is based on the Masked Autoencoder (MAE) framework using a Vision Transformer (ViT) backbone, with two key architectural innovations:

1. 3D Spatiotemporal Embeddings:
   • Replaced standard 2D patch embeddings with 3D convolutions to process sequences of images ($T \times H \times W$).
   • Utilized 3D positional encodings (combining time, height, and width) to learn spatial and temporal relationships simultaneously.
2. Metadata Integration:
   • Explicitly incorporates location (lat/lon) and time (year/day-of-year) embeddings.
   • These are added to the token embeddings via learned weighted sums.
   • Drop Mechanism: During pretraining, metadata is randomly dropped (10% probability) to ensure the model can function effectively even when metadata is unavailable in downstream tasks.

C. Model Variants

The authors trained two model sizes based on ViT architectures:

• Prithvi-EO-2.0-300M (ViT-L)
• Prithvi-EO-2.0-600M (ViT-H)
• Variants were trained with (TL) and without temporal and location embeddings to isolate the impact of these features.

D. Training Infrastructure

• Hardware: Trained on the JUWELS supercomputer (Booster partition) using A100 GPUs.
• Scale: The 600M model utilized ~58,000 GPU-hours.
• Hyperparameters: Cosine learning rate scheduler, weight decay of 0.05, and global batch size of 3,840.

3. Key Contributions

• Multi-Temporal Foundation Model: Introduction of the first large-scale GFM explicitly designed for global, multi-temporal EO data with integrated spatiotemporal and metadata embeddings.
• Comprehensive Benchmarking: Extensive evaluation using GEO-Bench, a standardized framework covering 12 datasets (6 classification, 6 segmentation) across varying resolutions (0.1m to 15m) and domains.
• Real-World Validation: Collaboration with Subject Matter Experts (SMEs) to validate the model on three critical application domains:
  1. Disaster Response: Flood mapping, wildfire scar/intensity mapping, and landslide detection.
  2. Land Cover & Crop Mapping: Multi-temporal crop segmentation (US/Europe) and classification.
  3. Ecosystem Dynamics: Above Ground Biomass (AGB) estimation and Gross Primary Productivity (GPP) prediction.
• Open Science & Tooling:
  • Release of models on Hugging Face.
  • Integration into TerraTorch (a PyTorch Lightning toolkit) to simplify fine-tuning and deployment for non-experts.
  • Provision of data loaders and code on GitHub.

4. Results

A. Benchmarking (GEO-Bench)

• Overall Performance: Prithvi-EO-2.0 outperformed its predecessor (Prithvi-EO-1.0) by 8% on average across all GEO-Bench tasks.
• Comparison: It surpassed six other leading GFMs (including DOFA, ScaleMAE, Satlas, and DeCUR) in both classification and segmentation tasks.
• Impact of Architecture: Models with temporal and location embeddings (TL variants) consistently outperformed non-TL variants, proving the value of explicit spatiotemporal modeling.
• Resolution Versatility: Despite being pre-trained on 30m data, the model achieved state-of-the-art results on high-resolution tasks (e.g., 0.1m drone imagery), demonstrating strong generalization.

B. Downstream Task Performance

• Disaster Response:
  • Flood/Wildfire: Improved IoU for water and burn scar classes by 3.5–5.6 points over the previous version.
  • Landslide Detection: Demonstrated superior few-shot learning capabilities. When trained on only ~1% of the data, Prithvi-EO-2.0 maintained high performance (mIoU ~67%), whereas baseline U-Nets suffered significant degradation.
• Land Cover & Crops:
  • Achieved the highest mIoU (50.7%) in US crop segmentation and outperformed baselines in European land cover classification, even with reduced data ratios (down to 6.25% of training data).
  • Successfully handled long time-series inputs (up to 61 frames in PASTIS dataset) despite being pre-trained on 4-frame sequences.
• Ecosystem Dynamics:
  • AGB Estimation: Approached the performance of multimodal baselines using only multispectral data, though SAR integration remains a challenge.
  • GPP Estimation: Achieved the highest $R^2$ scores (up to 0.88) compared to Random Forest, XGBoost, and ResNet baselines, effectively utilizing spatio-spectral information without needing manual vegetation indices.

5. Significance

• Paradigm Shift: Prithvi-EO-2.0 establishes a new standard for EO foundation models by moving beyond static image analysis to dynamic, multi-temporal understanding.
• Scalability & Efficiency: The model demonstrates that large-scale pre-training on global data enables robust performance even with limited labeled data for downstream tasks (few-shot learning), reducing the cost of data annotation.
• Community Impact: By integrating with TerraTorch and releasing under a permissive license, the project lowers the barrier to entry for researchers and practitioners, fostering a "Trusted Open Science" approach.
• Versatility: The model's ability to generalize across resolutions (from 0.1m to 30m) and domains (disaster, agriculture, ecology) highlights its potential as a universal backbone for Earth Observation applications.