🤖 machine learning

Beyond Point Predictions: Uncertainty-Aware Satellite Poverty Mapping for Public Policy

This paper introduces an uncertainty-aware machine learning framework that combines spatiotemporal transformers with conformal prediction to generate statistically guaranteed poverty estimates across Africa, demonstrating that while Earth observation data alone cannot guide policy due to inherent uncertainty, it can reliably supplement traditional surveys to optimize aid allocation while strictly controlling exclusion risks.

Original authors: Markus B. Pettersson, James Bailie, Mohammad Kakooei, Eagon Meng, Adel Daoud

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Markus B. Pettersson, James Bailie, Mohammad Kakooei, Eagon Meng, Adel Daoud

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

For decades, the map of global poverty has been drawn with a rough, uneven hand. In many parts of Africa, the most detailed information about who is poor and who is not comes from household surveys, where teams of interviewers visit villages to ask families about their assets, from televisions to flush toilets. These surveys are the gold standard for truth, but they are expensive, slow to update, and often leave vast stretches of the continent unmeasured. In the last few years, a new tool has emerged to fill these gaps: machine learning models that scan satellite images to guess a neighborhood's wealth. By analyzing patterns in the light of cities at night or the texture of the land during the day, these computer programs can now produce poverty maps for entire countries. They are fast, cheap, and surprisingly accurate on average, offering a glimpse into places that were previously invisible to policymakers.

However, a new study reveals a critical flaw in relying on these maps alone. While the models are good at predicting the general wealth of a region, they are often too uncertain to make life-or-death decisions about who gets aid. A model might predict that a village is poor, but if the computer's guess could easily be wrong, handing out money based solely on that guess risks leaving the truly needy behind or wasting funds on those who do not qualify. The researchers behind this work, based in Sweden and the United States, set out to solve this problem not by making the models more accurate, but by teaching them to admit when they are unsure. They developed a method that wraps every prediction in a statistical safety net, ensuring that decision-makers know exactly how much they can trust the data before they act.

The researchers began by training a sophisticated computer model on a massive dataset of satellite images and real-world survey results from thirty-eight African countries. They fed the model sequences of images from the Landsat satellites and nighttime light sensors, teaching it to estimate an "International Wealth Index" for roughly seventy thousand neighborhoods. This index is a score from zero to one hundred based on what a household owns. The model learned to predict this score with impressive skill, explaining about seventy-five percent of the differences in wealth across the continent. Yet, when the team asked the model to provide a range of possible values rather than a single number, a different picture emerged. Even for the best-performing models, the range of uncertainty was so wide that it covered nearly forty percent of all possible wealth levels. In practical terms, a single prediction could not distinguish between a family with a basic dirt floor and one with a refrigerator and a car. This finding challenges the assumption that high accuracy in a general sense means the data is ready for specific policy use.

To fix this, the team introduced a technique called "conformal prediction," which acts like a calibration tool for the model's confidence. Instead of just guessing a number, the model now produces a range of values, and the researchers adjusted this range until they could mathematically guarantee that the true wealth of a neighborhood would fall inside it a specific percentage of the time, such as ninety percent. This process ensures that the model never lies about its own uncertainty. When the researchers tested this approach, they found that while the model was still accurate on average, the "safety zones" around its predictions were often too wide to be useful for deciding who gets help. A neighborhood might be predicted to be poor, but if the safety zone includes both very poor and moderately wealthy households, the model cannot safely say who qualifies for aid.

Recognizing that a wide safety zone is a sign of genuine uncertainty, the researchers developed a new strategy for using these maps in the real world. They created a system called "Conformalized Thresholding," which allows the model to say "I don't know" when the evidence is not strong enough. If the model is confident that a neighborhood is poor, it flags it for aid. If it is confident the neighborhood is wealthy, it leaves it alone. But if the prediction falls into a gray area where the model cannot be sure, the system automatically triggers a follow-up survey to get the real answer. This approach treats the satellite data and the ground surveys not as rivals, but as partners. The satellite data handles the easy cases at a low cost, while the expensive surveys are reserved only for the difficult cases where the computer is unsure.

The team tested this hybrid approach in simulations across seventeen African countries, comparing it against traditional methods that rely entirely on surveys or entirely on satellite guesses. They found that their new method could deliver significantly more aid to the right people for the same amount of money. By using the satellite model to screen out the clearly wealthy and the clearly poor, they saved enough money on surveys to fund more cash transfers for the eligible families. In these simulations, the method successfully kept the rate of missed eligible families below a strict limit, proving that it is possible to use fast, automated data without sacrificing reliability. The results showed that when the system was allowed to be slightly less strict about excluding people, it could rely even more on the satellite data, further increasing the efficiency of aid distribution.

The study concludes that the future of poverty mapping lies not in chasing perfect predictions, but in building systems that know their own limits. The researchers demonstrated that even with the best available satellite data, the uncertainty in the predictions is too high to rely on them blindly for policy. However, by combining these powerful tools with a method that explicitly measures and manages that uncertainty, governments and aid organizations can make faster, cheaper, and more reliable decisions. The work suggests that the most effective way to help the poor is to let the machines do the heavy lifting of scanning the continent, while keeping human surveys ready to step in exactly where the machines hesitate. This balance allows for the scale of satellite technology without the risk of leaving the most vulnerable behind due to a computer's guess.

Technical Summary: Beyond Point Predictions: Uncertainty-Aware Satellite Poverty Mapping for Public Policy

Problem Statement
High-resolution poverty data are critical for targeting development resources and evaluating aid programs, yet such data remain scarce across much of Africa due to the high cost and logistical difficulty of household surveys. While Earth Observation (EO) combined with Machine Learning (ML) has emerged as a scalable alternative for estimating poverty using satellite imagery, current approaches rely heavily on point predictions (e.g., $R^2$ metrics). The authors argue that strong aggregate performance does not guarantee utility for decision-making. Specifically, a model with a high $R^2$ (e.g., 0.75) can still produce residuals large enough to misclassify neighborhoods relative to a poverty threshold, leading to the exclusion of eligible recipients or the misallocation of funds. The core problem is the lack of statistically valid uncertainty quantification in existing EO-ML poverty maps, which prevents their reliable use in policy contexts where error rates must be strictly controlled.

Methodology
The paper proposes a framework that moves beyond point estimates to provide statistically guaranteed prediction intervals and decision rules. The methodology consists of four main components:

Spatiotemporal Transformer with Simultaneous Quantile Regression (EO-SQR):
The authors train a transformer-based model on sequences of multispectral Landsat and nighttime-light images to predict the International Wealth Index (IWI), a proxy for asset wealth. Unlike standard regression, the model employs simultaneous quantile regression to predict specific quantiles (e.g., the 5th and 95th percentiles) directly, generating initial prediction intervals for neighborhood-level wealth.
Conformalized Quantile Regression (CQR):
To ensure the prediction intervals achieve a desired coverage rate (e.g., 90%), the authors apply Conformal Prediction. Using a held-out calibration set, they measure the residuals of the initial intervals, sort them, and select a quantile corresponding to the target coverage. This quantile serves as a uniform correction term, widening the intervals to guarantee that the true wealth value falls within the interval at the specified rate, assuming exchangeability between calibration and deployment data.
Conformalized Thresholding (CT):
For binary policy decisions (e.g., "Is this neighborhood below the poverty line $\beta$ ?"), the authors introduce CT. This method classifies a neighborhood as "Below" or "Above" only if the entire calibrated prediction interval lies strictly on one side of the threshold. If the interval overlaps the threshold, the model abstains, returning "Indeterminate." Crucially, CT calibrates the intervals separately for neighborhoods below and above the threshold, allowing for independent control of the error rates for both classes (e.g., capping the false negative rate at 5% and the false positive rate at 5%).
Screen And Follow up with Error control (SAFE):
To address aid allocation, the authors extend CT into the SAFE procedure. Given a fixed budget and an exclusion rate constraint (the maximum fraction of eligible neighborhoods that can be left without aid), SAFE operates in three steps:
- Screen: Confidently exclude neighborhoods predicted to be above the poverty line (controlled by the exclusion rate).
- Direct Aid: Provide aid directly to neighborhoods confidently predicted to be below the line.
- Follow-up: Conduct targeted surveys for "indeterminate" neighborhoods to resolve uncertainty before allocating aid.
  SAFE iteratively selects the coverage parameter ( $\alpha_2$ ) that maximizes the utility metric: aid dollars delivered per truly eligible recipient.

Key Results

Prediction Intervals vs. Point Accuracy: The EO-SQR model achieves an $R^2$ of 0.75 on held-out data, matching state-of-the-art performance. However, the resulting 90% conformalized prediction intervals are wide, with a median width of 28.7 IWI points. This width encompasses a significant portion of the observed wealth distribution (from basic floor materials to households with TVs and refrigerators), indicating that point predictions alone are insufficient for local-level targeting.
Classification Performance: In threshold classification tasks (poverty line $\beta = 20$ ), standard point predictions violate error constraints (20% false negatives). Standard intervals satisfy error constraints but classify fewer than 50% of samples. The proposed CT method achieves the target error rates (5% for both classes) while maintaining a decision coverage of 69.3%, significantly outperforming standard conformal intervals (34.0% coverage).
Aid Allocation Efficiency: In simulations across 17 African countries, SAFE outperforms baseline strategies ("Give all," "Survey all," and standard CQR). SAFE delivers substantially more aid per eligible recipient by leveraging EO-ML for confident cases and reserving costly surveys for uncertain cases. As the permitted exclusion rate increases, SAFE's performance approaches that of an "Oracle" (perfect knowledge) strategy.
Spatial Patterns: The generated maps reveal that uncertainty is not strictly correlated with wealth; for instance, some wealthy regions exhibit wide prediction intervals, while some poorer regions do not.

Significance and Claims
The paper claims to break the "either-or" dichotomy of poverty measurement, which forces a choice between costly, complete surveys or potentially unreliable EO-ML predictions. The authors assert that:

Uncertainty is Inherent: Even with high $R^2$ , EO-ML models possess inherent uncertainty that must be quantified to be used responsibly. High accuracy does not imply low absolute error sufficient for threshold-based policy.
Complementarity: EO-ML and surveys are not competitors but complements. By using uncertainty-aware methods, policymakers can act on model outputs where the signal is decisive and reserve survey resources for cases where the model is uncertain.
Statistical Guarantees: The proposed methods (CT and SAFE) provide marginal, distribution-free guarantees on error rates (misclassification and exclusion), ensuring that the risk of excluding eligible populations remains below a prespecified level on average across the population, rather than for every individual neighborhood.
Generalizability: The calibration and decision procedures are agnostic to the underlying model or data source (e.g., they can apply to mobile phone data or administrative records), provided the data is exchangeable.

The work establishes that while EO-ML cannot naively replace surveys for all policy tasks, it can serve as a reliable, efficient supplement when integrated with uncertainty quantification and targeted follow-up mechanisms.

Technical Summary: Beyond Point Predictions: Uncertainty-Aware Satellite Poverty Mapping for Public Policy

More like this