Inferring Height from Earth Embeddings: First insights using Google AlphaEarth

This study demonstrates that AlphaEarth Embeddings effectively encode topographic information that, when processed by U-Net++ architectures, enables accurate regional surface height mapping with strong generalization, despite challenges related to distribution shifts and residual bias.

The Gist

The Big Idea: Reading the "DNA" of the Earth

Imagine you have a massive, high-resolution 3D map of a region in France. It's perfect, but it's expensive and hard to make. Now, imagine you have a different tool: a "smart summary" of that same region. This summary isn't a picture; it's a list of numbers (a vector) that describes everything about that spot on the ground—the trees, the soil, the weather history, and the satellite signals.

This paper asks a simple question: Can we use these "smart summaries" to rebuild the 3D map?

The researchers used a new technology called AlphaEarth Embeddings. Think of these embeddings as the "DNA" of a specific location on Earth. They are compact, digital fingerprints created by Google's AI, which has studied billions of satellite images and data points.

The Experiment: Teaching a Robot to "Guess" the Height

The team wanted to see if they could train a computer (specifically, a Deep Learning model) to look at these "DNA fingerprints" and guess the height of the ground (like a mountain, a hill, or a flat field).

1. The Teacher (The Data): They had a "Gold Standard" map (a Digital Surface Model) from the French government. This was the answer key.
2. The Student (The AI): They used two types of AI architectures, named U-Net and U-Net++.
   • Analogy: Imagine U-Net as a standard student who studies hard. U-Net++ is like that same student, but with a special "cheat sheet" (nested skip connections) that helps them remember details from earlier in the lesson, making them better at connecting the dots.
3. The Test: They taught the AI using 70% of the land. Then, they tested it on the remaining 30% (a completely different area) to see if it could generalize or if it just memorized the answers.

The Results: Who Passed the Test?

1. The Linear Baseline (The "Old School" Method)
They first tried a simple linear math model (Ridge Regression).

• Result: It failed miserably. It tried to draw a straight line through a curved world. It predicted negative heights (holes in the ground where there are mountains) and got confused easily.
• Metaphor: It's like trying to predict the shape of a rollercoaster by drawing a straight line. It just doesn't work.

2. The Deep Learning Models (The "Smart" Methods)
Both U-Net and U-Net++ did much better. They understood that the "DNA" of the land contains clues about its shape.

• U-Net: Did a great job on the training data (learning the rules) but stumbled a bit when tested on new land.
• U-Net++: Was the champion. It handled the new land much better.
  • Analogy: If U-Net is a tourist who memorizes the map of one city, U-Net++ is a local guide who understands how cities are built, so they can navigate a new city they've never seen before.

The Challenges: The "Accent" Problem

Even the best model (U-Net++) wasn't perfect. On the test set, the error was about 16 meters (roughly the height of a 5-story building).

Why?

• The Distribution Shift: The training area was mostly flat or low hills. The testing area had higher, steeper mountains.
• Metaphor: Imagine teaching a student to speak English using only a book about London. Then, you take them to Scotland. They know the words, but the "accent" and the slang are different. The AI knew the concept of height, but it got confused by the specific "accent" of the new terrain.

Why This Matters

This study is a big deal for three reasons:

1. It Works: It proves that these "Earth Embeddings" (the DNA) actually contain enough information to guess the shape of the land. You don't always need a new satellite photo; sometimes the "summary" is enough.
2. It's Efficient: Instead of building a massive, heavy computer model from scratch, you can use these pre-made summaries and a lightweight AI decoder. It's like using a pre-cooked meal kit instead of farming your own wheat.
3. The Future: While the AI isn't perfect yet (it still has a bias), it shows that we are moving toward a world where we can map the entire Earth's surface using these smart, compressed data summaries, saving time and money.

The Bottom Line

The researchers successfully taught an AI to look at a digital "fingerprint" of the Earth and guess the height of the ground. While the AI isn't perfect yet (it gets a little confused when the terrain changes drastically), it proved that these new "Earth Embeddings" are a powerful, promising tool for mapping our world. The U-Net++ model was the star of the show, showing that with the right architecture, AI can learn to "see" the shape of the land from data that isn't even a picture.

Technical Summary

1. Problem Statement

The study addresses the challenge of regional surface height mapping (generating Digital Surface Models, or DSMs) in the context of the "big data" era of Earth Observation (EO). While EO data is abundant, high-quality labeled data (ground truth) for training deep learning (DL) models is scarce and expensive to acquire.

• The Gap: Traditional supervised DL approaches require massive, task-specific labeled datasets.
• The Opportunity: Geospatial Foundation Models (GeoFMs), specifically Google's AlphaEarth, generate "Earth Embeddings"—compact, multimodal vector representations of geographic locations that encode spatial, temporal, and spectral information without needing task-specific labels.
• The Research Question: Can these pre-computed Earth Embeddings effectively guide lightweight deep learning regression models to infer terrain surface heights from a high-quality reference DSM, particularly when transferring between regions with different height distributions?

2. Methodology

Data Sources

• Input: AlphaEarth Embeddings (10m spatial resolution, 64 bands) derived from Sentinel-1/2, GEDI, Copernicus DEM, and GRACE data. The study used the 2020 annual dataset (uint8 format) to reduce storage/computational load.
• Target (Ground Truth): A high-quality Digital Surface Model (DSM) from the French National Geographic Institute (IGN) at 5m resolution, resampled to 10m.
• Study Area: Nouvelle-Aquitaine, France (~8,000 km²).
  • Training/Validation: 70% of the area (5,550 km²).
  • Testing: A contiguous 30% of the area (2,315 km²) to ensure spatial independence.
• Preprocessing: Embeddings were normalized to [0, 1] range; invalid pixels (-128) were set to 0.

Model Architecture

The authors employed lightweight convolutional encoder-decoder architectures to act as "task-specific heads" that map the 64-channel embedding vectors to a single-channel height map.

1. U-Net: Standard convolutional architecture.
2. U-Net++: An enhanced version with nested skip connections to better preserve multi-scale spatial context.

• Backbone: ResNet-18 encoder (initialized with ImageNet weights).
• Training Strategy: Patch-based training (512x512), AdamW optimizer, Mean Squared Error (MSE) loss, and early stopping.
• Baseline: A Ridge Regression model was used to compare non-linear DL performance against a linear baseline.

Evaluation Metrics

Performance was assessed using:

• Correlation coefficients ($R^2$, Pearson, Spearman).
• Error statistics: RMSE, Mean/Median Difference, Standard Deviation (SD), Normalized Median Absolute Deviation (NMAD).
• Physical plausibility: Checking for negative height values.

3. Key Contributions

1. First Evaluation of Earth Embeddings for Surface Height: This is the first study to investigate AlphaEarth Embeddings specifically for regional surface height mapping (as opposed to canopy height or land cover).
2. Scalable Raster Processing: Unlike previous studies that processed embeddings at point locations or small patches, this study processed the embeddings in their native raster domain (10m resolution) over a large regional scale (7,865 km²), demonstrating the feasibility of regional deployment.
3. Architectural Comparison: It provides a systematic comparison between standard U-Net and U-Net++ architectures for decoding geospatial embeddings, highlighting the importance of spatial context in regression tasks.
4. Bias Analysis: The study identifies and quantifies the impact of distribution shifts (differences in height frequency between training and testing areas) on model generalization.

4. Results

Training Performance

Both DL models achieved excellent training performance, indicating the embeddings contain decodable height signals:

• U-Net: $R^2 = 0.97$.
• U-Net++: $R^2 = 0.97$.
• Baseline (Ridge): $R^2 = 0.64$.
• Observation: DL models closely followed the diagonal reference line, while the linear baseline showed systematic over/underestimation.

Testing Performance (Generalization)

Performance dropped on the independent test set due to a shift in height distribution (test set had more pixels in the 110–160m range), but DL models significantly outperformed the linear baseline.

• U-Net++ (Best Performer):
  • $R^2 = 0.84$
  • RMSE $\approx$ 16.4 m
  • Median Difference = -2.62 m
• U-Net:
  • $R^2 = 0.78$
  • RMSE $\approx$ 19.3 m
  • Median Difference = -7.22 m
• Ridge Regression:
  • $R^2 = 0.38$
  • RMSE $\approx$ 32.2 m
  • Critical Failure: Produced physically implausible negative heights (down to -90m) and exhibited massive bias (-16.42m mean difference).

Key Findings

• U-Net++ Superiority: The nested skip connections in U-Net++ provided better generalization and robustness to outliers compared to standard U-Net, evidenced by lower RMSE and a smaller gap between mean and median errors.
• Physical Consistency: DL models maintained predictions within realistic positive height ranges, whereas the linear model failed to extrapolate correctly under distribution shifts.
• Distribution Shift Impact: The performance drop from training to testing was primarily attributed to the mismatch in height frequency distributions between the two regions, not a lack of signal in the embeddings.

5. Significance and Conclusion

• Feasibility: The study proves that AlphaEarth Embeddings contain sufficient geospatial information to guide DL models for regional terrain mapping without requiring task-specific pre-training on labeled height data.
• Efficiency: Using pre-computed embeddings with lightweight decoders (U-Net++) offers a computationally efficient alternative to training massive end-to-end models from raw satellite imagery.
• Transferability: While the models show strong correlation ($R^2 > 0.8$) across regions, the residual bias and RMSE (~16m) highlight that distribution shifts remain a challenge. The embeddings capture transferable topographic patterns, but accuracy degrades when the target region's morphological statistics differ significantly from the training set.
• Future Outlook: The authors suggest that expanding training data to include diverse morphological conditions and benchmarking different embedding datasets are necessary steps to improve regional transferability and reduce bias.

In summary, this paper establishes a promising workflow for embedding-driven height mapping, demonstrating that Geospatial Foundation Models can serve as powerful, transferable feature extractors for terrain inference, provided that spatially aware architectures (like U-Net++) are used to decode the information.