💻 computer science

Global Building Area Estimation Products: How Accurate Are They?

This paper presents a comprehensive, independent evaluation of four major global building footprint datasets (GHSL, TEMPO, GBA, and Overture) using the ORBITaL-Net ground truth, revealing that while TEMPO and GBA generally achieve the highest accuracy, all products exhibit significant performance variations based on geographic region, population density, and income levels, with notably lower accuracy in Africa, Asia, and high-density urban areas.

Original authors: Saad Lahrichi, Doa'a Allabadi, Kyle Bradbury, Jordan Malof

Published 2026-07-23

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Saad Lahrichi, Doa'a Allabadi, Kyle Bradbury, Jordan Malof

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Earth as a giant, living puzzle. For decades, scientists and city planners have been trying to map every single piece of that puzzle: where the houses are, how big they are, and how many people live in them. This isn't just about drawing pretty maps; it's about understanding how our world is growing. If we know exactly where buildings are, we can figure out how much energy they use, how to prepare for disasters, and even track how much pollution cities are spewing into the air. To do this, researchers use "satellite eyes" to look down from space and create digital maps of building footprints. But here's the catch: just like different artists might draw the same tree in slightly different ways, different computer programs create these building maps using different rules and tools. Until now, no one had really sat down to compare them fairly to see which artist was actually the most accurate.

This paper is like a giant, global "taste test" for four of the most popular building-mapping tools available today. The researchers didn't just guess which one was best; they brought in a "gold standard" referee: a massive dataset called ORBITaL-Net. Think of ORBITaL-Net as a team of super-observant humans who spent years looking at incredibly sharp, high-resolution satellite photos and manually drawing the outlines of every building they saw, one by one. Because these humans did the work with such care, their drawings are treated as the "truth." The researchers then took the four computer-generated maps—GHSL, GBA, TEMPO, and Overture—and compared them against this human-made truth to see which one got the numbers right.

The results of this taste test were a bit surprising and nuanced. There wasn't a single "winner" that was perfect in every way. Instead, it depends on what you are looking for. If you want the map that is closest to the actual total number of square meters of buildings, the Global Building Atlas (GBA) seems to be the most precise, making the smallest average mistakes. However, if you care more about getting the pattern right—knowing exactly which neighborhoods are crowded and which are empty—the TEMPO product did the best job. It was the most consistent at spotting where buildings actually are, even if its total count was slightly off.

The study also found that these maps aren't equally good everywhere. It's like how a GPS might work perfectly in a city but get confused in a dense forest. All the products struggled significantly when looking at Africa and Asia, often getting the size or location of buildings wrong. They also had a harder time in super-dense cities, where buildings are packed tight together. Interestingly, the GHSL product tended to "hallucinate" too many buildings, consistently overestimating how much space was covered, while the others tended to be a bit too shy, underestimating the total area.

Perhaps the most important lesson from this research is that the way you measure accuracy matters. The researchers tested the maps on different grid sizes, like looking at the world through a coarse net versus a fine mesh. They found that the results changed depending on the grid used, proving that you can't just pick one number to declare a winner. Instead, the best tool depends on your specific needs: do you need the most accurate total area, or the most reliable map of where buildings are located? By using this rigorous, human-checked comparison, the paper gives us a clear guide on which digital map to trust for different jobs, helping us build a more accurate picture of our planet's future.

Problem Statement
Geo-spatial rasters of building footprint area are critical for downstream tasks such as monitoring urbanization, improving energy efficiency, tracking greenhouse gas emissions, and disaster preparedness. While multiple global building raster datasets have emerged (e.g., GHSL, Microsoft TEMPO, Global Building Atlas, Overture), there is a lack of an independent, comprehensive, and fair assessment of their accuracy. Existing comparisons suffer from several limitations: ground truth data often relies on lower-resolution imagery (4m or less) with limited geographic diversity; assessments are frequently conducted on a single reference grid, which can systematically bias results in favor of products with similar resolutions; and recent products are often excluded from comparative analyses.

Methodology
The authors conduct a comprehensive accuracy evaluation of four major global building products: Global Human Settlement Layer (GHSL), Microsoft's TEMPO, The Global Building Atlas (GBA), and Overture Maps.

Ground Truth: The study utilizes ORBITaL-Net, a globally diverse dataset of manually labeled building footprints derived from very high-resolution (0.45 m) Maxar satellite imagery (2010–2020). This dataset includes 1.49 million hand-labeled buildings across 128,000 image chips, covering both urban and rural areas. The labeling process involved a rigorous three-tier validation (self-review, peer-review, expert validation) to ensure high accuracy.
Experimental Design: To ensure fairness and avoid bias introduced by reprojection, the authors evaluate products on multiple reference grids corresponding to the native grids of the products being tested:
1. GHSL WGS84 (3 arcsec)
2. GHSL Mollweide (100 m)
3. TEMPO EPSG:3857 (77 m)
  For each grid, they compute ground truth building areas within fully covered cells and reproject the other products onto these grids. Vector products (GBA, Overture) are intersected with grid cells, while raster products (GHSL, TEMPO) undergo area-weighted density aggregation.
Metrics: Performance is measured using five complementary metrics: Total building area, Aggregate difference ( $\Delta$ ), Percentage difference ( $\Delta\%$ ), Mean Absolute Error (MAE), Weighted Mean Absolute Percentage Error (WMAPE), and Coefficient of Determination ( $R^2$ ).
Stratification: Results are stratified by geographic location (continent), population density (ranging from very low-density rural to high-density urban), and World Bank income groups to identify regional and socioeconomic biases.

Key Contributions

Unprecedented Ground Truth: The study leverages ORBITaL-Net, which offers higher resolution (0.45 m) and greater geographic diversity than previous ground truth sources used in building product comparisons.
Multi-Grid Evaluation: Unlike prior work that often uses a single reference grid, this work evaluates products on multiple grids (including the native grids of the products themselves) to provide a fairer comparison and reveal how reprojection affects accuracy.
Comprehensive Benchmark: The study includes a broad set of recent products (GHSL, TEMPO, GBA, Overture) and provides a detailed breakdown of performance across various global contexts.

Results

Overall Accuracy: No single product is uniformly the most accurate across all metrics.
- GBA consistently achieves the lowest MAE and WMAPE across all reference grids, indicating superior precision in total area estimation.
- TEMPO consistently achieves the smallest aggregate bias ( $\Delta$ ) and the highest $R^2$ , suggesting it better captures spatial variation and makes fewer large-magnitude errors, though it may have more small-magnitude errors.
- Overture generally performs below GBA and TEMPO.
- GHSL consistently exhibits the highest error rates and a systematic tendency to overestimate building area, particularly in sparse rural settlements.
Reprojection Effects: Reprojecting raster products onto grids with different resolutions introduces additional error. For instance, TEMPO's error increases when evaluated on GHSL grids. Vector products (GBA, Overture) theoretically avoid this specific error source.
Regional and Socioeconomic Biases:
- Geography: All products perform significantly worse in Africa and Asia compared to the Americas and Oceania. GHSL shows severe failure in Africa ( $R^2 = -0.37$ ).
- Population Density: Accuracy generally degrades in high-density urban areas for most products (except GHSL, which performs relatively better there but still worse than others). TEMPO maintains the most stable $R^2$ across density strata.
- Income: Non-GHSL products perform best in high-income regions. GHSL shows extreme instability, with near-zero $R^2$ in lower-middle-income regions and massive overestimation (+106%) in those same areas.

Significance and Claims
The paper claims to provide the first global, independent comparison of heterogeneous building area products using manually labeled ground truth. The authors state that their work fills an important gap in the literature by establishing a rigorous benchmark. They conclude that while no single product is perfect, GBA and TEMPO generally offer the highest overall accuracy depending on the specific evaluation criteria (precision vs. bias/spatial correlation). The study highlights that product accuracy is not uniform globally; it varies significantly by region, population density, and income level. These findings provide essential context for researchers and practitioners to select the most appropriate dataset for their specific application and guide future development of building area estimation models.

More like this