ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmark

Technical Summary of ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmark

Problem Statement
Hyperspectral imaging (HSI) offers dense spectral information (hundreds of contiguous bands) essential for material-level characterization of Earth's surface, surpassing the capabilities of multispectral sensors. While recent advances in self-supervised learning (SSL) have leveraged HSI for static spatial representation, existing large-scale datasets remain "temporally shallow." They are predominantly designed for static scene understanding, lacking the continuous, long-term temporal coverage required to model surface dynamics, such as deforestation, land-use transitions, and ecosystem changes. Consequently, the development and systematic evaluation of long-horizon spatiotemporal hyperspectral foundation models are currently hindered by a lack of suitable data and standardized benchmarks.

Methodology
The authors address this gap through the construction of ChronoEarth-492K and the ChronoEarth-Benchmark.

• Data Source and Curation: ChronoEarth-492K is built upon NASA's EO-1 Hyperion mission (2001–2017), the world's longest continuous spaceborne hyperspectral archive. The authors curated 492,354 radiometrically harmonized patches across 185,398 global locations.
• Preprocessing:
  • Spectral Harmonization: From the original 242 bands, unstable and low-signal bands (e.g., water absorption regions) were removed, retaining 155 spectrally consistent bands aligned to a unified wavelength grid.
  • Global Patching: Scenes were resampled to a uniform 30m resolution and divided into non-overlapping $128 \times 128$ pixel patches. A deterministic global gridding system based on UTM zones was used to assign unique location identifiers (UIDs) to each patch.
  • Spatiotemporal Alignment: Patches sharing the same UID across different timestamps were aggregated to form temporal sequences. The dataset includes 28,786 locations with multi-temporal sequences (≥3 observations), enabling both short- and long-horizon analysis.
• Benchmark Construction: The ChronoEarth-Benchmark integrates six open-source geospatial products (covering land cover, crop types, forest dynamics, and soil properties) aligned with the hyperspectral observations. It defines three task categories:
  1. Static Tasks: Predicting labels from a single hyperspectral observation.
  2. Short-Horizon (SH) Tasks: Aggregating temporally adjacent observations (e.g., $T \le 4$) for prediction.
  3. Long-Horizon (LH) Tasks: Predicting future states (e.g., year $n+1$) based on historical sequences (e.g., years 2001 to $n$).
  • Generalization Settings: The benchmark includes spatial-temporal and continental generalization splits to evaluate model robustness under domain shifts (e.g., training on Europe/North America and testing on Africa/Latin America).
• Experimental Protocol: The authors evaluated state-of-the-art hyperspectral foundation models (e.g., SpectralViT, LESSViT, DOFA, HyperSigma). To handle temporal inputs, they employed a two-stage approach: (1) static pretraining on the full dataset, and (2) stage-two temporal pretraining on multi-temporal sequences using a causal masking objective (predicting the next frame). Temporal adaptation strategies included max pooling, attention-based aggregation, and fine-tuning with temporal SSL.

Key Contributions

1. ChronoEarth-492K: The first large-scale, temporally calibrated hyperspectral SSL dataset, offering 17 years of global observations with 492,354 patches and 28,786 multi-temporal sequences.
2. ChronoEarth-Benchmark: A unified evaluation suite spanning static, short-horizon, and long-horizon temporal tasks, along with controlled spatial and temporal generalization settings.
3. Empirical Baselines: Extensive evaluations of hyperspectral foundation models, establishing reference baselines and demonstrating the efficacy of temporal pretraining and modeling under diverse settings.

Results

• Static Performance: Self-supervised pretraining on ChronoEarth consistently improved downstream performance over supervised training from random initialization. Architectures explicitly modeling spatial-spectral interactions (e.g., LESSViT) generally outperformed others on static tasks.
• Generalization: Models exhibited performance degradation under spatial and temporal distribution shifts, with geographic variability identified as a dominant factor limiting generalization. However, models pretrained on ChronoEarth demonstrated effective cross-satellite transfer to EnMAP-based datasets, despite the spectral differences between EO-1 Hyperion and EnMAP.
• Temporal Modeling:
  • Short-Horizon: Incorporating additional temporal observations generally improved performance. Temporally pretrained models (using stage-two SSL) outperformed both max-pooling and supervised attention-based variants, indicating that learned temporal aggregation is superior to naive pooling.
  • Long-Horizon: Extending temporal history generally improved performance on land-cover transition tasks (CLCD, NLCD-S). The temporally pretrained model consistently outperformed other strategies, confirming the necessity of learned temporal modeling for capturing extended dependencies. However, for time-sensitive targets like crop type (CDL), performance became unstable with very long horizons ($T \le 8$), suggesting that longer historical context may introduce noise for certain tasks.

Significance
The paper claims that ChronoEarth and its benchmark provide the first large-scale, temporally grounded platform for systematic spatiotemporal hyperspectral representation learning. By moving beyond static scene understanding, this work enables the study of long-term environmental dynamics and establishes a standardized framework for evaluating temporal generalization and forecasting capabilities. The authors note that while their current approach extends static backbones with temporal modules, future research should focus on developing native spatiotemporal foundation models that integrate spatial, spectral, and temporal learning in a unified architecture to better handle long-term, irregularly sampled observations.