Advancing Earth Observation Through Machine Learning: A TorchGeo Tutorial

This paper introduces a tutorial for the PyTorch-based library TorchGeo that demonstrates its core abstractions and guides users through an end-to-end workflow for training a semantic segmentation model on Sentinel-2 imagery to perform multispectral water segmentation.

The Gist

Imagine you are a chef trying to cook a gourmet meal using ingredients from a massive, global warehouse.

The Problem: The "Standard Kitchen" vs. The "Satellite Warehouse"
Usually, when people teach computers to "see" (like recognizing cats in photos), they use standard recipes. They take a neat, square photo, chop it into tiny, uniform pieces, and feed it to the computer.

But Earth Observation (looking at Earth from space) is different. It's like trying to cook with ingredients from a warehouse that is:

1. Huge: The "photos" are entire continents, not just a single picture. They are too big to fit on your kitchen counter (computer memory).
2. Messy: The ingredients come in different units. One pile is in meters, another in feet. One is a photo, another is a map drawn with lines.
3. Tricky: If you just grab a random handful of ingredients, you might accidentally grab the same spot twice (which ruins the learning) or miss the whole picture.

The Solution: TorchGeo (The "Smart Kitchen Assistant")
The authors of this paper introduced a tool called TorchGeo. Think of it as a super-smart kitchen assistant designed specifically for this messy satellite warehouse. Instead of forcing you to manually measure, re-draw, and chop every single ingredient before you start cooking, TorchGeo handles the heavy lifting.

Here is how the paper explains their new "Cooking Class" (Tutorial) using simple analogies:

1. The Magic Mixing Bowl (Composable Datasets)

In a normal kitchen, if you have a bowl of flour and a bowl of sugar, you mix them. In the satellite world, you might have a photo of a forest and a separate map of where the trees are.

• The Analogy: TorchGeo has magic operators (like & and |) that act like a smart mixing bowl.
  • The Union (|) says, "Mix everything together!" (Creating a giant mosaic of all available photos).
  • The Intersection (&) says, "Only keep the parts where both the photo and the map exist."
• Why it matters: You don't have to manually cut out the matching pieces. The assistant does it instantly, only grabbing the specific slice you need right now, saving you from trying to carry the whole warehouse into your kitchen.

2. The GPS Cutter (Spatiotemporal Indexing)

Usually, you pick a photo by its filename. With satellites, you pick a photo by its location (latitude/longitude) and time.

• The Analogy: Imagine a giant, infinite pizza. Instead of asking for "the top-left slice," you tell the assistant, "Give me a 256-inch square slice starting at these exact GPS coordinates."
• Why it matters: The assistant instantly cuts that exact square out of the massive pizza, ensuring the "top" of your slice matches the "top" of the map, even if the map was drawn in a different language (coordinate system).

3. The Smart Tasting Spoon (Geographic Samplers)

When training a computer, you need to show it many examples.

• The Analogy:
  • Random Sampling (Training): Imagine a blindfolded chef throwing darts at the pizza to pick random spots to taste. This helps the chef learn the general flavor of the whole pizza.
  • Grid Sampling (Testing): Imagine the chef carefully cutting the pizza into a perfect grid to taste every single inch to make sure the whole thing is cooked.
• Why it matters: TorchGeo handles this "dart throwing" and "grid cutting" automatically, ensuring the computer learns without cheating (like seeing the same spot twice) and tests thoroughly.

4. The Real-World Test: The "Rio Water Hunt"

The second half of the paper is a live demonstration. They built a model to find water in satellite photos of Rio de Janeiro, Brazil.

• The Challenge: Satellite photos have many "colors" (bands) that human eyes can't see, like infrared. The model needed to be taught how to handle these extra colors.
• The Fix: They taught the model to look at the "special ingredients" (like calculating how wet a pixel looks using math formulas called spectral indices) and added them to the mix.
• The Result: They trained the model on a computer, then sent it to look at a real, massive photo of Rio.
• The Output: Instead of just saying "I got 80% right," the model produced a new map (GeoTIFF) showing exactly where the water is in Rio, pixel-by-pixel. You can zoom in and see if it correctly identified the water in the rivers and the ocean.

The Big Takeaway

This paper isn't just about code; it's about removing the friction.

Before, if you wanted to use AI to study Earth, you had to spend 90% of your time fixing messy data and 10% actually building the AI. TorchGeo flips that ratio. It lets scientists and developers spend 90% of their time solving real problems (like tracking water, forests, or cities) and only 10% worrying about the messy data.

It turns the "hard mode" of satellite data processing into a smooth, standard workflow, making it easier for anyone to use AI to protect and understand our planet.

Technical Summary

Based on the provided paper from the ICLR 2026 Machine Learning for Remote Sensing (ML4RS) Workshop, here is a detailed technical summary of the work.

Paper Title

Advancing Earth Observation Through Machine Learning: A TorchGeo Tutorial

1. Problem Statement

The paper addresses the fundamental disconnect between standard computer vision (CV) workflows and Earth Observation (EO) machine learning pipelines. While CV typically deals with curated, natural-image datasets, EO data presents unique challenges:

• Data Structure: Imagery arrives as large, georeferenced scenes (often multi-band) rather than fixed-size images.
• Label Complexity: Labels exist as raster masks or vector geometries in distinct Coordinate Reference Systems (CRS) and spatial resolutions, often separate from the imagery.
• Workflow Friction: Standard CV pipelines lack native support for critical EO operations, including:
  • Reprojecting and resampling layers into a common grid.
  • Sampling geographically aligned "chips" from scenes too large to fit in memory.
  • Constructing spatially separated train/validation/test splits to prevent label leakage.
  • Exporting predictions back to georeferenced formats (e.g., GeoTIFF) for downstream analysis.
• The Gap: Using satellite imagery is not merely "computer vision with larger images"; it requires domain-specific handling of metadata and spatial alignment that standard libraries (like torchvision) do not provide.

2. Methodology: TorchGeo Framework

The paper introduces TorchGeo, a PyTorch-based domain library designed to bridge this gap. The methodology focuses on two executable Python notebooks that demonstrate the library's core abstractions:

A. Core Abstractions (Introduction to TorchGeo)

• Composable Datasets: Utilizes set-like operators (& for intersection, | for union) to combine partially overlapping rasters (e.g., multi-sensor imagery) and labels. This allows for lazy loading, where mosaicking and reprojection occur on-demand during windowed reads rather than pre-processing entire scenes.
• Spatiotemporal Indexing: Enables slicing large rasters directly by projected coordinates and timestamps (e.g., dataset[xmin:xmax, ymin:ymax]), ensuring consistent CRS and resolution alignment at read time without manual preprocessing.
• Geographic Samplers: Integrates with standard PyTorch DataLoaders using specialized samplers:
  • RandomGeoSampler: For training, generating random chips to avoid spatial autocorrelation.
  • GridGeoSampler: For inference/evaluation, generating a grid of overlapping patches.
  • These samplers bypass the storage-heavy "pre-tiling" stage by performing windowed reads of only required pixels.

B. Case Study: Earth Surface Water Segmentation

The second notebook demonstrates an end-to-end workflow using the Earth Surface Water dataset (Sentinel-2 imagery and binary water masks):

• Data Preparation:
  • Constructs separate RasterDataset objects for imagery and masks.
  • Applies a global CRS (World Mercator, EPSG:3395) to align globally distributed patches.
  • Uses the intersection operator to pair inputs and labels.
• Multispectral Preprocessing:
  • Computes dataset-specific mean/std statistics.
  • Appends spectral indices (two NDWI variants and NDVI) as additional input channels.
  • Implements a normalization strategy that scales original sensor bands while leaving computed indices unchanged to preserve physical meaning.
• Model Architecture & Adaptation:
  • Uses DeepLabV3 with a ResNet-50 backbone.
  • Critical Adaptation: Re-initializes the first convolutional layer to accept the specific number of input channels (e.g., 6 spectral bands + 3 indices = 9 channels) while preserving the output feature dimension. Pre-trained weights are omitted as they target RGB natural images.
• Training Protocol:
  • Trained from scratch for 10 epochs using AdamW (lr=0.0001).
  • Uses RandomGeoSampler for training chips (512x512 px) to ensure spatial generalization.

3. Key Contributions

1. Domain-Specific Library: Provides a robust PyTorch ecosystem (TorchGeo) with 140+ datasets and 60+ datamodules specifically designed for geospatial metadata handling.
2. Lazy Composition & Indexing: Introduces a paradigm where complex data fusion (mosaicking, reprojection) is handled lazily via set operators and coordinate-based slicing, eliminating the need for massive pre-processing pipelines.
3. End-to-End Workflow Demonstration: The tutorial code (distributed as two notebooks) serves as a reproducible template for:
   • Handling multi-sensor, multi-resolution data fusion.
   • Adapting standard CV architectures to multispectral inputs.
   • Performing spatially aware sampling to reduce label leakage.
4. Inference to GeoTIFF: Demonstrates the full lifecycle from training to running gridded inference on a real-world scene (Rio de Janeiro) and exporting predictions as a pixel-aligned GeoTIFF.

4. Results

• Performance: On the validation set of the Earth Surface Water dataset, the trained model achieved:
  • Overall Accuracy: 0.977
  • Intersection over Union (IoU): 0.824
• Practical Application: The model was successfully deployed on a Sentinel-2 scene over Rio de Janeiro (captured Feb 1, 2026). The inference process utilized overlapping patches to generate a seamless, georeferenced prediction map (visualized in Figure 1), allowing for the analysis of model behavior in complex coastal and riverine environments.

5. Significance

This work is significant because it lowers the barrier to entry for applying deep learning to remote sensing. By abstracting away the complex geospatial mechanics (CRS alignment, tiling, sampling strategies), TorchGeo allows researchers to focus on model architecture and scientific inquiry rather than data engineering. The tutorial establishes a reproducible standard for:

• Spatial Integrity: Ensuring that training splits and evaluation metrics are spatially meaningful.
• Scalability: Enabling the processing of massive, global-scale datasets without requiring them to be fully loaded into memory.
• Operationalization: Providing a direct path from model training to actionable geospatial products (GeoTIFFs) for decision-making and scientific monitoring.