🤖 AI

A Proxy Consistency Loss for Grounded Fusion of Earth Observation and Location Encoders

This paper introduces a Proxy Consistency Loss (PCL) formulation that integrates abundant proxy variables into a trainable location encoder to overcome the scarcity of high-quality labeled Earth observation data, demonstrating superior performance in air quality prediction and poverty mapping compared to existing fusion strategies.

Original authors: Zhongying Wang, Kevin Lane, Levi Cai, Morteza Karimzadeh, Esther Rolf

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhongying Wang, Kevin Lane, Levi Cai, Morteza Karimzadeh, Esther Rolf

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the air quality in a city or the wealth of a neighborhood. You have a powerful camera (satellite images) and a map, but you are missing the most important ingredient: ground truth.

In the real world, getting accurate data (like measuring actual air pollution with a sensor or surveying people about their income) is expensive, slow, and sparse. You might only have sensors in a few scattered parks, leaving huge gaps in between. If you try to teach the robot using only these few scattered points, it gets confused. It might memorize the exact locations of the sensors but fail to understand the general pattern, much like a student who memorizes the answers to a specific test but fails when the questions are slightly different.

This paper proposes a clever solution called Proxy Consistency Loss (PCL). Here is how it works, explained through simple analogies.

The Problem: The "Sparse Sensor" Dilemma

Think of the robot as a student trying to learn geography.

The Goal: Predict air quality (or poverty) everywhere.
The Problem: The teacher (the dataset) only gives the student answers for 50 specific cities. The student tries to guess the air quality for the 10,000 other towns.
The Mistake: Without extra help, the student just guesses based on the 50 cities. If a new town is slightly different, the student gets it wrong because they didn't learn the rules of the world, just the locations of the answers.

The Solution: The "Proxy" Helper

The researchers realized that while we don't have sensors everywhere, we have other data everywhere.

The Proxy: Imagine you don't have a thermometer in every house, but you do have a satellite that sees how bright the streetlights are at night. Brighter lights usually mean richer neighborhoods.
The Catch: Streetlights aren't a perfect measure of wealth, but they are related. They are a "proxy."

The Innovation: The "Location Coach"

Most previous methods tried to just feed the streetlight data into the robot's brain along with the satellite photos. The researchers found this didn't work well. Instead, they created a special "Location Coach" (a trainable location encoder).

Here is the magic trick:

The Main Job: The robot looks at satellite photos and the "Location Coach" to predict the real answer (e.g., Air Quality).
The Side Job (The Proxy Consistency Loss): The "Location Coach" is also forced to look at the proxy data (e.g., the streetlights) and try to predict that as well.

The Analogy:
Imagine a chef (the robot) trying to cook a perfect stew (the prediction).

Old Way: The chef only tastes the stew at 50 specific spots in the pot. They guess the rest.
New Way (PCL): The chef has a "Location Coach." The coach tells the chef, "Hey, not only do you need to cook the stew, but you also need to be able to predict the temperature of the kitchen based on where you are standing."
Why it works: To be good at predicting the kitchen temperature everywhere, the chef has to learn the general rules of how heat moves through the room. Once the chef understands the physics of the room (the geography), they become much better at cooking the stew, even in spots where they never tasted it before.

The "Proxy Consistency Loss" (PCL)

This is the mathematical rule that forces the "Location Coach" to do its side job.

It says: "Your internal map of the world (the location embedding) must be so good that it can explain both the real answer (Air Quality) AND the proxy answer (Streetlights)."
This prevents the robot from just memorizing the sensor locations. It forces the robot to learn a smooth, logical map of the world that makes sense for both tasks.

What Did They Find?

They tested this on two big problems:

Air Quality: Predicting pollution across the US using sparse sensors and a global weather model as the proxy.
Poverty Mapping: Predicting wealth in Africa using survey data and satellite night-light data as the proxy.

The Results:

Better Guesses: The robot using PCL was much better at guessing the answer for places it had never seen before (out-of-sample).
Smarter Maps: When they visualized the robot's internal "map," the PCL version was smooth and logical. The old versions were "spotty" and messy, like a map with random dots.
Robustness: Even when the test data was very different from the training data (like testing in a completely different region), the PCL model held up better.

The Takeaway

In a world where perfect data is rare, we can use "imperfect but abundant" data (proxies) to train our AI to understand the world better. By forcing the AI to learn a consistent map that explains both the perfect data (where we have it) and the imperfect data (where we have plenty), we get a much smarter, more reliable predictor.

It's like teaching a student not just the answers to a test, but the underlying logic of the subject, by giving them a second, related subject to study at the same time.

1. Problem Statement

Machine learning models for Earth Observation (EO) often face a critical bottleneck: the sparsity of high-quality labeled data (e.g., in-situ air quality sensors or poverty survey clusters). While EO data (satellite imagery) is abundant, the ground-truth labels required for supervised learning are often limited to specific points, making it difficult to train models that generalize well across large geographic areas or to unseen regions (out-of-sample).

Existing approaches attempt to mitigate this by:

Transfer Learning: Pre-training on abundant proxy data (e.g., nighttime lights for poverty).
Proxy Stacking: Adding proxy data as extra input channels to the observation encoder.
Frozen Location Encoders: Using pre-trained, general-purpose location embeddings (e.g., GeoCLIP) as a geographic prior.

However, these methods have limitations. Simply stacking proxy data often yields marginal gains. Frozen location encoders may be too general for specific tasks, while training task-specific location encoders from scratch on sparse labels often leads to overfitting to the specific training points, failing to capture broader spatial patterns.

2. Methodology

The authors propose a novel framework called Proxy Consistency Loss (PCL) to train a trainable location-time encoder that is regularized by abundant proxy data.

Core Architecture

The model fuses two encoders at the embedding level:

Observation Encoder: Processes EO data (e.g., satellite image time series) to produce an embedding $e_{obs}$ .
Location-Time Encoder: Processes geospatial coordinates (longitude, latitude) and time to produce an embedding $e_{loc}$ .
Fusion: The embeddings are concatenated and passed through a prediction head (MLP) to predict the target variable $y$ .

The Proxy Consistency Loss (PCL)

The key innovation is a multi-task learning objective where the location encoder is jointly optimized for two tasks:

Primary Task: Predicting the sparse target label $y$ (e.g., PM2.5 concentration).
Auxiliary Proxy Task: Reconstructing a continuous proxy variable $z$ (e.g., reanalysis PM2.5 or VIIRS night lights) using only the location encoder output.

The total loss function is:
$\mathcal{L} = \mathcal{L}_{pred} + \lambda \mathcal{L}_{proxy-consistency}$
Where:

$\mathcal{L}_{pred}$ is the standard supervised loss (e.g., MSE) back-propagating through both encoders.
$\mathcal{L}_{proxy-consistency}$ measures the error between the predicted proxy $\hat{z}$ (derived from $e_{loc}$ ) and the actual proxy data $z$ . Crucially, this loss only updates the location encoder, acting as a spatial regularizer.

Sampling Strategy

To prevent overfitting to sparse labeled points, the proxy data is sampled independently of the labeled data. During training, for every batch of labeled samples, an additional batch of proxy samples is drawn uniformly at random (UAR) across the entire study domain. This forces the location encoder to learn a robust, globally consistent representation rather than memorizing specific sensor locations.

3. Key Contributions

Proxy Consistency Loss (PCL): A novel loss formulation that uses abundant, continuous proxy variables to regularize trainable location encoders, preventing overfitting to sparse labels.
Agile Location Encoding: Demonstrates that location encoders can be effectively trained from scratch (rather than relying on frozen pre-trained models) if properly regularized by proxy data.
Separation of Sampling: A training strategy that decouples the sampling of proxy data from labeled data, allowing the model to learn geographic priors from the entire domain, not just where labels exist.
Empirical Validation: Comprehensive experiments showing that PCL outperforms naive proxy stacking, frozen encoders, and standard multi-task pretraining approaches.

4. Experimental Results

The authors evaluated the method on two distinct tasks: Air Quality Prediction (PM2.5) in the contiguous US and Poverty Mapping (Asset Index) across Africa.

Air Quality Prediction (PM2.5)

In-Sample (Interpolation): PCL achieved the best performance, improving $R^2$ by 12.2% and reducing RMSE by 9.5% compared to an observation-only baseline. It outperformed both frozen encoders (GeoCLIP, Climplicit) and a two-stage proxy pretraining approach.
Out-of-Sample (Extrapolation): Using a checkerboard split to test spatial generalization, PCL maintained superior performance, showing a 17.8% gain in $R^2$ over the baseline. It demonstrated significantly better robustness when extrapolating to unseen regions compared to models trained without PCL.
Qualitative Analysis: Visualizations of the latent space showed that PCL-trained encoders produced spatially smoother embeddings with stronger temporal structures, whereas models trained only on sparse EPA stations exhibited artifacts and overfitting to station locations.

Poverty Mapping

PCL significantly outperformed all baselines, including proxy-stacked models and frozen GeoCLIP.
The method successfully utilized VIIRS night light data as a proxy to regularize the location encoder, leading to better generalization across 30 African countries.

Ablation Studies

Sampling Ratio: Performance improved rapidly as the ratio of proxy samples ( $\rho$ ) increased, saturating around $\rho \approx 8-16$ . Uniform random sampling of proxy points was found to be sufficient; aligning proxy samples with label locations provided no additional benefit.
Temporal Encoding: Including "Day of Year" (DOY) encoding in the location encoder significantly improved performance for air quality, highlighting the importance of spatiotemporal consistency.
Auxiliary Proxies: Adding auxiliary physical variables (e.g., humidity, wind) alongside the main proxy further improved extrapolation performance.

5. Significance and Conclusion

This paper addresses a fundamental challenge in geospatial AI: how to leverage abundant, imperfect proxy data to improve models trained on sparse, high-quality labels.

Robustness: The PCL framework provides a mechanism to create task-specific geographic priors that are robust to spatial domain shifts, a common failure point for standard deep learning models in EO.
Efficiency: Unlike two-stage pretraining, PCL integrates the proxy signal directly into the training loop, avoiding the need for separate pretraining phases.
Generalizability: While tested on PM2.5 and poverty, the framework is applicable to any EO task where sparse point observations can be complemented by spatially continuous proxy variables (e.g., soil moisture, water quality).

The authors conclude that while proxy data alone is insufficient and naive stacking is suboptimal, grounding the location encoder with a consistency loss allows the model to learn a latent space that captures both the specific task requirements and the broader geospatial context, leading to superior generalization.