🤖 AI

GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations

The paper proposes GAIR, a novel location-aware self-supervised learning framework that extends Vision Transformers with Neural Implicit Local Interpolation to generate continuous, high-resolution geospatial representations, enabling superior performance across diverse geospatial tasks compared to existing foundation models.

Original authors: Zeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zeping Liu, Ni Lao, Zhangyu Wang, Junfeng Jiao, Gengchen Mai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand the world. Right now, AI has two very different "eyes":

The Satellite Eye: It looks down from space. It sees the whole neighborhood, the shape of the city, and the layout of the fields. But it's like looking at a map from a plane; you can't see the texture of the sidewalk or the color of the front door.
The Street-View Eye: It walks around on the ground. It sees the cracks in the pavement, the graffiti on the wall, and the trees up close. But it's blind to the big picture; it doesn't know how the whole neighborhood fits together.

The Problem:
For a long time, AI researchers tried to teach these two eyes to talk to each other. But they hit a wall. A single satellite photo covers a huge area (like a whole city block), while a street photo covers just one tiny spot. It's like trying to match a single pixel from a giant map to a whole room in a house. The scales are so different that the AI gets confused. It tries to guess where the street photo belongs on the map, but it often gets it wrong because the "pixels" don't line up perfectly.

The Solution: GAIR (The "Magic Zoom Lens")
The authors of this paper created a new system called GAIR. Think of GAIR as a magic, liquid zoom lens that bridges the gap between the sky and the street.

Here is how it works, using simple analogies:

1. The "Liquid Map" (Neural Implicit Representations)

Usually, a satellite image is like a grid of Lego bricks. If you want to know what's in the middle of a brick, you have to guess.
GAIR turns that grid into liquid. Instead of being stuck to a rigid grid, the satellite image becomes a smooth, continuous fluid.

The Analogy: Imagine a satellite photo is a painting made of pixels. GAIR turns that painting into a high-resolution hologram. No matter how close you zoom in (even to a microscopic point), the hologram can tell you exactly what the satellite sees at that specific coordinate.
The Result: Now, the AI can take a street-level photo and ask the "liquid map," "What does the satellite see right here?" It gets a perfect, fine-grained answer, even if the street photo is just a tiny dot on the big satellite image.

2. The "Three-Way Handshake" (Contrastive Learning)

To teach the AI, they use a method called "contrastive learning." Imagine a game of "Find the Match."

The Players:
- Player A: The Street View photo (what the ground sees).
- Player B: The Satellite photo (what the sky sees).
- Player C: The GPS Coordinates (the address).
The Game: The AI is shown a street photo and its GPS address. It then looks at the "liquid map" and tries to find the exact spot on the satellite image that matches.
The Goal: The AI learns that "This street view of a bakery" + "This GPS address" = "This specific patch of satellite image."
The Magic: Because the "liquid map" (the NILI module) can zoom in perfectly, the AI learns to match the street view to the exact pixel on the satellite image, not just a blurry guess.

3. The "Universal Translator"

Once the AI learns this matching game on millions of examples (using a dataset they built called Streetscapes1M, which is like a giant library of 1 million matched street/satellite pairs), it becomes a super-smart geographer.

It doesn't just know how to match pictures. It learns the meaning of places.

Example: If you show it a street photo of a run-down neighborhood, it now knows exactly what that looks like from space.
Example: If you show it a satellite photo of a dense forest, it can predict what the trees look like from the ground.

Why is this a Big Deal?

Before GAIR, AI models were like tourists who could only see the world from a helicopter or only from the sidewalk. They couldn't connect the two.

GAIR allows the AI to:

Predict the Economy: Look at a street view and guess the income level of the neighborhood (useful for banks or city planners).
Track Crops: Look at a satellite image and know exactly what kind of crops are growing, even if the view is slightly different.
Find Species: Identify what kind of birds or plants are in an area just by looking at the satellite map and the GPS location.

The "De-Biasing" Trick

The authors also noticed a problem: Most street view data is in cities. There is very little data for rural farms or forests. If you only train on cities, the AI becomes "city-biased" and fails in the countryside.

To fix this, they used a clever trick. Since the "liquid map" (satellite data) is available everywhere (even in the middle of nowhere), they taught the AI to rely on the satellite and GPS data when street data is missing.

The Analogy: If the AI is in a city, it uses all three senses (Sky, Street, GPS). If it's in a remote forest where there are no street cameras, it gracefully switches to just using the Sky and GPS senses, without losing its intelligence. This makes the AI work well everywhere, not just in New York or London.

Summary

GAIR is a new AI system that uses a "liquid" mathematical trick to perfectly align satellite photos with street-level photos. It teaches the computer to see the world from both the sky and the ground simultaneously, making it much smarter at understanding geography, economics, and nature, whether it's in a busy city or a quiet farm.

1. Problem Statement

Current Geo-Foundation Models (GeoFMs) face significant limitations in geospatial representation learning, primarily due to the scale gap and lack of explicit spatial alignment between different data modalities:

Scale Discrepancy: Overhead Remote Sensing (RS) images (e.g., Sentinel-2) provide broad, low-resolution views, while Street View (SV) images offer fine-grained, ground-level details. A single RS patch often corresponds to hundreds of SV images, creating a "one-to-many" spatial relationship.
Inadequate Alignment: Existing methods often rely on constructing discrete, co-located image pairs. However, in medium-resolution RS data, an SV image's viewshed may collapse to a sub-pixel scale, making precise pairing impossible without losing fine-grained correspondences.
Missing Modality Integration: Most GeoFMs focus on fusing different overhead sensors (e.g., Optical + SAR) or RS + Text, neglecting the integration of ground-level imagery and precise geolocation metadata, which limits their spatial reasoning capabilities.
Data Scarcity: Labeled geospatial data is scarce and imbalanced, necessitating robust self-supervised learning (SSL) frameworks that can leverage vast amounts of unlabeled multimodal data.

2. Methodology

The authors propose GAIR (Geo-Aligned Implicit Representations), a location-aware SSL framework designed to bridge the scale gap and align heterogeneous modalities.

A. Architecture Overview

GAIR utilizes a factorized encoder architecture with three distinct branches:

Remote Sensing Encoder ( $f$ ): A Vision Transformer (ViT) that extracts patch-wise feature maps from overhead RS images, preserving spatial structure.
Street View Encoder ( $g$ ): A ViT that extracts global image-level embeddings from ground-level SV images.
Location Encoder ( $e$ ): A neural network (using Random Fourier Features) that maps geocoordinates (latitude/longitude) into a dense vector representation.

B. Core Innovation: Neural Implicit Local Interpolation (NILI)

To solve the scale mismatch, GAIR introduces the NILI module, which acts as a continuous, resolution-agnostic decoder for the RS features.

Mechanism: Instead of treating RS features as discrete patches, NILI learns a continuous function that maps arbitrary spatial coordinates within the RS image to a localized feature embedding.
Process: For a specific SV image location $x_i$ , the module interpolates the RS feature map $f(r_i)$ using the four nearest neighboring patch embeddings. It employs a shared Multi-Layer Perceptron (MLP) modulated by coordinate offsets to generate a geo-aligned localized RS embedding ( $z^{(q)}_i$ ) that corresponds exactly to the SV image's location.
Benefit: This allows the model to extract RS features at any arbitrary coordinate (even sub-pixel), effectively bridging the resolution gap without requiring high-resolution RS data or rigid image pairing.

C. Self-Supervised Learning Objectives

GAIR is trained using two contrastive learning objectives on unlabeled data:

Implicit Neural Contrastive Learning (INCL): Aligns the localized RS embedding ( $z^{(q)}_i$ ) with the corresponding SV embedding ( $g(s_i)$ ). This forces the model to learn semantic consistency between the overhead view and the ground view at the exact same location.
Spatially Explicit Contrastive Learning (SECL): Incorporates the location embedding ( $e(x_i)$ ) into the contrastive loop. It aligns both the RS and SV embeddings with the location vector, reinforcing geospatial consistency across the entire dataset. A memory bank is used to store location embeddings from previous batches to increase negative sample diversity.

3. Key Contributions

Novel Framework (GAIR): The first SSL framework to explicitly integrate overhead RS, street view, and geolocation metadata using a geo-aligned implicit representation approach.
NILI Module: A novel Neural Implicit Local Interpolation module that enables continuous, coordinate-level alignment between RS and SV modalities, eliminating the need for discrete, scale-matched image pairs.
Streetscapes1M Dataset: The authors constructed a new global-scale pretraining dataset containing 1 million multimodal tuples (RS image, SV image, geolocation) sampled from 688 cities.
Comprehensive Evaluation: Extensive benchmarking across 9 geospatial tasks and 22 datasets, covering socio-economic regression, species recognition, land cover segmentation, and image geolocalization.

4. Experimental Results

GAIR was evaluated against state-of-the-art GeoFMs (e.g., SatMAE, CROMA, SpectralGPT, DOFA) and standard SSL baselines (MoCo V3, MAE).

Overall Performance: GAIR achieved State-of-the-Art (SOTA) performance across all 9 downstream tasks.
Street View Tasks:
- Outperformed baselines in Socio-economic Indicator Regression (e.g., population density, income) and Human Perception tasks.
- Demonstrated superior View Direction Classification accuracy, proving its ability to capture spatial context.
Remote Sensing Tasks:
- Achieved the highest mIoU in Burn Scars Segmentation (83.26%) and Cropland Polygon Delineation (43.35%) in linear probing settings, outperforming larger models like CROMA (ViT-L) and Scale-MAE.
- Showed strong performance in Multi-temporal Crop Type Mapping, validating the model's ability to handle temporal sequences.
Location-Based Tasks:
- Significantly improved Geo-Aware Species Recognition and Image Regression (e.g., Nightlight Luminosity, Elevation) compared to models without location priors.
- Demonstrated that pretraining with GAIR provides better initialization than GeoCLIP or TaxaBind, even when the pretraining data (Streetscapes1M) contained no species images.
Ablation Studies:
- NILI is Critical: Removing NILI (replacing it with global pooling) caused a significant performance drop, confirming that continuous spatial alignment is superior to discrete patch matching.
- Contrastive vs. Reconstruction: Contrastive learning (GAIR) vastly outperformed reconstruction-based objectives (MAE), indicating that semantic alignment is more important than pixel-level reconstruction for cross-view tasks.
- Geolocation Impact: Including the location encoder further boosted performance, particularly for abstract tasks like socio-economic prediction.

5. Significance and Implications

Bridging the Scale Gap: GAIR successfully solves the fundamental challenge of aligning overhead and ground-level views by treating the RS image as a continuous field rather than a grid of discrete patches.
Generalizability: The model learns generalizable geospatial representations that transfer well across diverse tasks (regression, classification, segmentation) and spatial scales, even with limited labeled data (few-shot learning).
Bias Mitigation: The authors demonstrated that the factorized architecture allows for "de-biasing" by pretraining the RS and location encoders on rural data where street view is unavailable, reducing geographic bias without sacrificing performance.
Efficiency: Despite the complex pretraining, the NILI module is discarded during inference for standard tasks, meaning GAIR retains the same computational efficiency (GFLOPs and latency) as standard ViT-based GeoFMs.

In conclusion, GAIR represents a paradigm shift in geospatial foundation models by moving from discrete patch alignment to continuous, location-aware implicit representation, enabling more robust and generalizable geospatial AI.