💻 computer science

Scaling Remote Sensing Foundation Models: Data Domain Tradeoffs at the Peta-Scale

This paper investigates the scaling behaviors of remote sensing foundation models trained on over a quadrillion pixels of satellite data, revealing that performance remains data-limited rather than model-parameter-limited and offering practical insights for optimizing future large-scale Earth observation AI development.

Original authors: Charith Wickrema, Eliza Mace, Hunter Brown, Heidys Cabrera, Nick Krall, Matthew O'Neill, Shivangi Sarkar, Lowell Weissman, Eric Hughes, Guido Zarrella

Published 2026-01-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Charith Wickrema, Eliza Mace, Hunter Brown, Heidys Cabrera, Nick Krall, Matthew O'Neill, Shivangi Sarkar, Lowell Weissman, Eric Hughes, Guido Zarrella

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot to understand the world from above, looking down at cities, forests, and oceans through satellite cameras. This paper is like a massive field report from a team of engineers who tried to build the biggest, most powerful "brain" (a foundation model) for this job, using a dataset so huge it contains over one quadrillion pixels of satellite images.

Here is the story of what they found, explained simply:

1. The Goal: Building a "Satellite Brain"

In the world of regular photos (like selfies or landscapes), we know that if you give a computer more data and a bigger brain, it usually gets smarter. But for satellite images, things are different. The world looks very different from space, and we didn't know if just making the computer bigger would actually help, or if we were just hitting a wall.

The team built a massive dataset called Akupara, which is like a giant library of satellite photos covering almost the entire Earth (except the US, due to privacy rules). They used this to train different sizes of AI models to see how they learned.

2. The Big Surprises (What Worked and What Didn't)

The "More Data" Rule (It Works, But Slowly)

The Analogy: Imagine you are trying to learn to identify every type of tree in the world.

The Finding: If you show the robot 5,000 pictures of trees, it learns a bit. If you show it 1 million pictures, it learns a lot more.
The Catch: The paper found that while adding more data always helps, the improvement gets smaller and smaller the more you add. It's like studying for a test: reading the first 10 pages of a textbook helps you a lot. Reading the next 10 pages helps, but maybe not quite as much. Reading the next 100 pages helps even less.
The Lesson: To make these AI models better, you need more variety in your data (different seasons, different places, different weather), not just more of the same thing.

The "Bigger Brain" Myth (It Didn't Help Much)

The Analogy: Imagine you have a student who is very smart but is only allowed to read a short, specific story.

The Finding: The team tried using a tiny brain (86 million "neurons") and a giant brain (1.8 billion "neurons"). They fed them the exact same amount of satellite photos.
The Result: The giant brain didn't do any better than the tiny one. It was like giving a PhD student a 5-page comic book to study; they couldn't use their extra brainpower because there wasn't enough information to learn from.
The Lesson: In remote sensing, data is the bottleneck, not the computer size. Making the model bigger without giving it more diverse data is a waste of money and electricity.

The "Speed vs. Stability" Trap

The Analogy: Think of training the AI like driving a car up a steep hill.

Batch Size (The Engine): The team tried driving with a huge engine (large batches of data processed at once) to go faster. They found that for the early part of the trip, a huge engine didn't make the car climb the hill any faster. It just burned more gas.
Learning Rate (The Gas Pedal): They also tried pressing the gas pedal very hard (high learning rates). This caused the car to shake, spin out, or crash (the AI "diverged" or stopped learning).
The Lesson: It's better to drive steadily with a moderate gas pedal and a steady stream of new scenery (data) than to try to speed through the process.

3. The "Recipe" for Success

Based on their experiments, the authors suggest a specific recipe for anyone trying to build these satellite AI models:

Don't Rush: Start with a gentle learning pace. If you go too fast, the model gets confused and crashes.
Focus on Variety: Don't just collect 1 million photos of New York City. Collect photos of New York, a desert, a rainforest, a snowy mountain, and a city at night. The AI needs to see everything to get smart.
Don't Oversize the Brain: If you have a limited amount of data, a medium-sized brain is just as good as a giant one. Save your money for more data.
Test Small First: Before spending millions of dollars on a massive computer run, test your settings on a small scale. If the model starts acting crazy, stop immediately. This saves time and money.

Summary

The paper concludes that for satellite AI, more data is the key, but only if it's diverse data. Simply making the computer model bigger or trying to train it faster doesn't work well. The "limit" isn't how smart the computer is; it's how much of the world we can show it. To build the next generation of satellite AI, we need to keep collecting more varied pictures of Earth, not just bigger computers.

Technical Summary: Scaling Remote Sensing Foundation Models

Problem Statement

While scaling laws for foundation models (FMs) are well-established in natural language processing and natural image domains, their applicability to remote sensing (RS) remains poorly understood. RS data presents unique challenges, including diverse spectral and spatial properties, limited labeled samples, and a potential upper bound on representational capacity due to the finite variety of observable objects in overhead imagery. A critical gap exists in understanding the relationship between training dataset size, model parameters, and performance at the petascale. Specifically, it is unclear whether current frontier-scale RS models are constrained by data availability or model capacity, and whether increasing model size or batch size yields diminishing returns in this domain.

Methodology

The authors conducted a systematic study using Akupara, a new global remote sensing dataset comprising over 1 petapixel (1 × 10¹⁵ pixels) of high-resolution Maxar satellite imagery (0.3m–1.2m Ground Sample Distance). The dataset includes over 1 million full satellite images covering 415 million km².

To generate training labels at this scale, the team developed G-DAUG (Geospatial Data Augmentation), a pipeline that pairs image chips with OpenStreetMap (OSM) vector data to create automated, weak supervision labels (semantic segmentation, instance segmentation, and rotated bounding boxes) for 37 object classes.

The experimental design focused on four guiding questions using two model architectures:

MTP (Multi-Task Pretraining): A fully supervised framework using a shared ViT backbone with task-specific decoders for segmentation and detection.
ScaleMAE: A self-supervised masked autoencoder that incorporates real-world scale (GSD) into positional embeddings.

Key experiments included:

Batch Size Sensitivity: Sweeping batch sizes (128–512) to determine critical batch sizes for convergence efficiency.
Learning Rate (LR) Stability: Implementing a "Fail Fast" protocol to triage unstable LRs before committing to full training runs, utilizing a Warmup, Stable, Decay (WSD) schedule.
Data Scaling: Training a fixed, oversized model (ViT-Giant, 1.8B parameters) on nested subsets of Akupara ranging from 5k to 1M images to isolate data effects.
Parameter Scaling: Training models of varying sizes (86M to 1.8B parameters) on a fixed dataset (100k images) to isolate parameter effects.

Key Contributions

The paper provides a process-oriented account of scaling RS foundation models, offering the following empirical insights:

Throughput Scaling Lessons: Early-to-mid-stage training is largely insensitive to batch size increases beyond moderate thresholds. Efficiency is better achieved through data diversity and LR scheduling than aggressive batching.
Stable Optimization Schedules: A WSD regime with a moderate base learning rate (specifically 1 × 10⁻⁴ for MTP with ViT-L) provides robust convergence. The "Fail Fast" warmup protocol effectively identifies and discards unstable high-LR candidates, saving compute resources.
Empirical Data-Scaling Law: Within the range of 5k to 1M full-scene images, validation loss follows a consistent power law ( $L(N) = A + BN^{-a}$ ) with an exponent $a \approx 0.03$ . This indicates diminishing but non-saturated returns, supporting the value of further data expansion.
Empirical Parameter-Scaling Law: Across model sizes from 86M to 1.8B parameters, performance gains were effectively flat ( $b \approx 0$ ) when data and training budgets were held constant. This suggests that in typical data regimes, scaling parameters alone yields negligible benefits.
Scalable Weak Supervision: The G-DAUG pipeline demonstrates a reproducible method for generating global-scale supervision from open vector sources, bridging the gap between GIS data and high-resolution raster training.

Results

Data vs. Parameters: The study confirms that RS foundation models currently operate in a data-limited regime. Increasing dataset size consistently improves performance (following a power law), whereas increasing model parameters without a corresponding increase in data or training tokens per parameter does not reduce loss.
Batch Size: Increasing batch size from 128 to 512 did not accelerate convergence or improve efficiency in early-to-mid training stages.
Learning Rates: High learning rates (e.g., $10^{-3}$ ) caused divergence or oscillation. The optimal stable rate identified was $10^{-4}$ , which allowed for monotonic loss reduction.
Failure Modes: Approximately half of the data-scaling runs became trapped in local minima early in training, highlighting the sensitivity of large-scale RS training to initialization and optimization dynamics.

Significance and Claims

The authors claim that this work establishes the first empirical scaling behaviors for RS foundation models trained at the petascale. The primary significance lies in shifting the paradigm for RS model development from a focus on "bigger models" to "better data diversity and optimization strategies."

The paper argues that:

RS is Data-Limited: Unlike some natural image domains where model capacity can be a bottleneck, RS performance is currently constrained by the availability of diverse, in-domain data.
Diminishing Returns are Manageable: While the data-scaling exponent ( $a=0.03$ ) indicates diminishing returns, the returns are not saturated, implying that continued data collection yields measurable gains.
Engineering over Architecture: Future progress depends on engineering levers such as data stratification (geography, sensor, season), label quality, and stable optimization schedules rather than simply increasing parameter counts or batch sizes.
Practical Guidelines: The study provides a "practical checklist" for the community, emphasizing conservative learning rates, stratified data sampling to avoid content duplication, and rigorous hyperparameter triage to prevent wasted compute.

The authors conclude that sustainable progress in RS foundation modeling hinges on the use of more diverse data (including multi-modal and multi-temporal sources) rather than merely scaling compute or model size.