Technical Summary: BELDE – Building a Large-scale Earth-observation Land-cover Dataset for Europe

Problem Statement

Accurate land-cover semantic segmentation is essential for environmental monitoring, urban planning, and disaster response. While multi-spectral sensors are increasingly available, true-color (RGB) imagery remains the most widely accessible modality due to lower storage, transmission, and deployment constraints across various satellite and aerial platforms. However, existing RGB-based semantic segmentation datasets suffer from significant limitations: restricted geographic coverage, small scale, narrow environmental diversity, or lack of public accessibility. Consequently, models trained on these datasets often fail to generalize to regions with different climates, land-cover distributions, or atmospheric conditions. There is a critical need for a large-scale, geographically diverse, and publicly accessible RGB benchmark to evaluate the robustness and transferability of Earth observation models.

Methodology

Dataset Construction (BELDE)

The authors introduce BELDE (Building a Large-scale Earth-observation Land-cover Dataset for Europe), a publicly available dataset constructed via a fully automated and reproducible pipeline.

• Data Sources: The dataset combines Sentinel-2 True Color Imagery (TCI) and ESA WorldCover 2021 land-cover annotations. Sentinel-2 provides annual median composites at 10m resolution, facilitating cloud removal and normalization.
• Geographic Scope: The primary dataset covers Europe (13°W–50°E longitude, 33°N–60°N latitude), encompassing diverse climates from Mediterranean to alpine environments.
• Preprocessing Pipeline:
  1. Spatial Alignment: Co-registered RGB tiles and land-cover maps are aligned using geospatial identifiers.
  2. Patch Extraction: A sliding-window strategy generates non-overlapping 256 × 256 pixel patches at 10m resolution to prevent data leakage.
  3. Label Harmonization: The original 11 ESA WorldCover classes are aggregated into 7 semantic classes (Tree, Shrub, Grass, Crop, Built-up, Barren, Water) to address class imbalance and semantic overlap in RGB observations (e.g., merging mangroves into trees, snow into barren).
  4. Quality Filtering: Patches with missing data or >90% water coverage are excluded.
• Final Statistics: The curated dataset contains 1,088,385 image-mask pairs.

Out-of-Distribution Benchmarks

To evaluate geographic domain shift, two additional subsets were generated using the same pipeline:

• BELDE-K: Covers the Republic of Korea (16,607 pairs).
• BELDE-CA-NV: Covers California and Nevada, USA (88,155 pairs).

Evaluation Protocol

The authors evaluated 17 semantic segmentation architectures across three families:

1. Encoder-Decoder CNNs: UNet, UNet++, DeepLabV3/V3+, FPN, LinkNet, PSPNet.
2. Transformer-based Models: SegFormer, DPT variants (DeiT3, MaxViT).
3. Lightweight/Hybrid Architectures: EfficientFormer, FastViT, LALE.

Models were trained using the AdamW optimizer with a multiclass Dice loss, ImageNet normalization, and standard augmentations. Performance was measured using Accuracy, Precision, Recall, F1-score, and IoU on in-domain (Europe) and zero-shot cross-domain (Korea, CA-NV) test sets.

Key Contributions

1. BELDE Dataset: A large-scale, publicly available RGB semantic segmentation dataset containing over 1.08 million geo-aligned image-mask pairs across Europe at 10m resolution.
2. Cross-Region Benchmarks: The release of BELDE-K and BELDE-CA-NV to enable systematic evaluation of geographic domain shift and model transferability.
3. Baseline Results: Comprehensive benchmarking of 17 state-of-the-art architectures, establishing performance baselines for both in-domain and cross-domain settings.
4. Reproducible Pipeline: An open-source, automated framework for dataset generation, promoting transparency and future expansion.

Results

In-Domain Performance (Europe)

• Top Performers: Transformer-based and hybrid architectures achieved the highest accuracy. EfficientFormer-L7 and MaxViT reached F1-scores of 83.0% and 82.9%, respectively.
• CNN Performance: CNN-based models like UNet remained competitive, achieving an F1-score of 81.8% with 31M parameters.
• Lightweight Models: The LALE-S2 model demonstrated a favorable efficiency-performance trade-off, achieving 78.2% F1-score with only 2.6M parameters.
• Class-Specific Analysis: Models performed strongly on spectrally homogeneous classes (Tree, Water) but struggled with structurally heterogeneous and rare classes, particularly Shrub, which consistently yielded the lowest F1-scores across all architectures.

Cross-Region Generalization

• Performance Drop: Models trained on BELDE (Europe) exhibited significant performance degradation when applied to out-of-distribution regions, highlighting the challenge of geographic domain shift.
  • BELDE-K (Korea): F1-scores dropped to 58.3% (EfficientFormer-L7) and 58.2% (MaxViT).
  • BELDE-CA-NV (USA): F1-scores dropped to 66.4% (MaxViT) and 66.2% (EfficientFormer-L7).
• Observations: The performance decrease is attributed to distinct land-cover compositions (e.g., higher tree density in Korea, higher barren land in CA-NV) and ecological differences. While transformer models generally maintained higher absolute performance, all architectural families showed consistent degradation, suggesting that geographic variability is a dominant limiting factor rather than model capacity alone.

Significance and Claims

The paper posits that BELDE addresses a critical gap in Earth observation research by providing a continental-scale RGB benchmark. Its primary significance lies in:

• Enabling Robustness Studies: By offering geographically diverse training and evaluation sets, BELDE allows researchers to systematically study and quantify the challenges of domain shift in land-cover mapping.
• Real-World Relevance: Focusing exclusively on RGB imagery ensures the dataset is applicable to a broad range of operational and resource-constrained sensing systems.
• Baseline Establishment: The provided benchmarks serve as a reference for evaluating the transferability of segmentation models, revealing that current state-of-the-art models struggle to generalize across continents without adaptation.

The authors conclude that while transformer-based models offer superior accuracy, lightweight architectures provide viable alternatives for resource-constrained deployment. They emphasize that generalization across continents remains a challenging open problem, underscoring the necessity of geographically diverse benchmarks for developing robust Earth observation systems.