💻 computer science

AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World

This paper introduces AerialMetric, a comprehensive benchmark dataset comprising 68,000 real-world and synthetic image-depth pairs with metric ground truth, to address the domain gap in UAV monocular metric depth estimation by systematically evaluating existing models and establishing state-of-the-art performance through fine-tuning.

Original authors: Zhongqiang Song, Guanying Chen, Yuqi Zhang, Yin Zou, Chuanyu Fu, Zhiyuan Yuan, Chuan Huang, Shuguang Cui, Xiaochun Cao

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhongqiang Song, Guanying Chen, Yuqi Zhang, Yin Zou, Chuanyu Fu, Zhiyuan Yuan, Chuan Huang, Shuguang Cui, Xiaochun Cao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot camera that has spent its entire life walking on the ground. It has learned to guess how far away things are just by looking at pictures of streets, living rooms, and forests. It's a pro at this. But now, you strap this camera onto a drone and send it flying high above the world. Suddenly, the robot is confused. It sees the world from a "bird's-eye view" it has never experienced before, and its guesses about distance become wildly inaccurate.

This paper, titled "AerialMetric," is about fixing that confusion. The authors built a new "school" (a dataset) specifically to teach these camera brains how to judge distance while flying.

Here is a breakdown of what they did, using simple analogies:

1. The Problem: The "Ground-Level" Bias

Think of existing AI depth models like a person who has only ever looked at the world from eye level. They know that a car is about 4 meters away because they've seen cars up close. But if you put them on a mountain top and ask, "How far is that car down there?" they might guess it's 10 meters away or 100 meters away. They are lost because the angle and the scale are totally different.

The paper found that when you take the best "ground-level" depth models and try to use them on drone footage, they fail miserably. It's like trying to use a map of a subway system to navigate a city skyline; the perspective is just wrong.

2. The Solution: The "AerialMetric" Dataset

To fix this, the team created AerialMetric, a massive new training library. Instead of just one type of data, they built it like a four-course meal to ensure the AI gets a balanced education:

The "Real World" Buffet (AerialMetric-Oblique): They took thousands of existing photos from real cities and rural areas where the distance was already measured accurately (using lasers or 3D mapping). This is like giving the student real homework from actual flights.
The "Science Lab" (AerialMetric-Decoupled): This is the most unique part. They flew a drone in a very controlled way, changing only one thing at a time. They tested:
- Different heights (80m vs. 120m).
- Different angles (looking straight down vs. looking sideways).
- Different lenses (wide vs. narrow).
- Analogy: Imagine a chef tasting a soup and changing only the salt, then only the pepper, to see exactly how each ingredient changes the flavor. This helps the AI understand exactly how height and angle affect distance.
The "Virtual Reality" Simulator (AerialMetric-Synthetic): They used computer graphics (like a high-end video game engine) to create fake drone flights. This allows them to create thousands of tricky scenarios (like flying over a factory or a park) that might be hard or dangerous to film in real life.
The "Wild Card" (AerialMetric-Wild): They grabbed random drone videos from the internet. These are messy, uncontrolled, and represent the real world where you don't have perfect data. This is the "final exam" to see if the AI can handle surprises.

3. The Results: Teaching the AI to Fly

The authors took a top-performing AI model (called MoGe2) and "fine-tuned" it using their new AerialMetric dataset.

Before Training: The AI was terrible at guessing distances from the sky. It was like a student failing a test with a score near zero.
After Training: The AI became a master. It learned to adjust its guesses based on how high the drone was and what angle it was looking.
- The Analogy: It's like taking a driver who only knows how to drive on flat highways and teaching them to drive on steep mountain roads. After the training, they can handle the turns and the elevation changes without crashing.

4. Did They Break Anything? (The "Ground" Test)

A common fear in AI is that if you teach a model something new, it might forget what it already knew. The authors checked this. They tested their newly trained "Drone Expert" on ground-level pictures (like street views).

The Result: The AI didn't forget how to drive on the ground. It actually got slightly better at some ground tasks, proving that learning to fly didn't ruin its ability to walk.

Summary

In short, the paper says: "Drone cameras are currently bad at guessing distance because they were trained on ground photos. We built a new, huge, and diverse training set specifically for the sky. When we taught our AI with this new set, it became the best at guessing distances from drones, and it didn't lose its ability to work on the ground."

They have made all their data, code, and the trained model available for others to use, essentially handing the keys to the "Drone Driving School" to the rest of the world.

Technical Summary: AerialMetric

Problem Statement

While data-driven monocular metric depth estimation has achieved remarkable progress in ground-level scenarios (street-view and indoor), these models exhibit a significant domain gap when applied to Unmanned Aerial Vehicle (UAV) imagery. State-of-the-art models trained primarily on terrestrial datasets fail to generalize to aerial viewpoints due to drastically different viewing geometries, bird's-eye perspectives, and altitude variations. Currently, the field lacks a real-world, diverse, and variable-controlled aerial dataset with reliable metric ground truth. This absence hinders effective model adaptation, prevents the reliable learning of absolute scene scale in aerial contexts, and makes systematic evaluation of existing methods under diverse UAV imaging conditions difficult. Consequently, aerial metric depth estimation remains substantially behind its ground-level counterpart.

Methodology

To address these challenges, the authors introduce AerialMetric, a large-scale benchmark dataset designed specifically for monocular metric depth estimation in aerial scenarios. The dataset is constructed through a hybrid pipeline comprising four complementary subsets:

AerialMetric-Oblique: Curated from six publicly available urban-scale photogrammetry datasets (e.g., UrbanBIS, GauU-SceneV2). It aggregates over 47K image-depth pairs from 25 sub-datasets covering City, Rural, and Natural scenes. Ground truth is derived via active LiDAR projection (with ray penetration artifact mitigation) and passive photogrammetry mesh rasterization.
AerialMetric-Decoupled: A strictly controlled benchmark collected using an RTK-equipped DJI Matrice 300 UAV. It employs orthogonal sampling to disentangle the effects of four core variables: Pitch Angle (nadir to oblique), Relative Altitude (80m and 120m), Field of View (FOV), and Scene Types (Building, Lawn, Farmland, Factory). This subset contains ~4,600 image-depth pairs designed exclusively for testing to evaluate the impact of specific imaging configurations.
AerialMetric-Synthetic: A training augmentation set containing 16K photorealistic image-depth pairs generated using Google Earth Studio and Unreal Engine (with AirSim and Cesium). This subset covers long-tail scenarios, extreme viewpoints, and complex lighting conditions that are difficult to capture in the real world, providing controlled multi-angle views.
AerialMetric-Wild: A collection of 100 high-fidelity sequences from unlabeled internet drone videos. To resolve monocular scale ambiguity, the authors employ a human-in-the-loop scale recovery framework. This involves using multi-view relative depth estimation (COLMAP + Depth Anything 3) followed by metric scale calibration using human-annotated landmark pairs and geographic tools (Google Earth Pro).

Model Adaptation and Evaluation:
The authors conduct systematic evaluations of state-of-the-art metric depth models (including ZoeDepth, DepthPro, UniDepth, MoGe2, and Metric3Dv2) on the AerialMetric benchmarks. They demonstrate that zero-shot transfer from ground pretraining yields poor performance. To establish a strong baseline, they fine-tune the representative model MoGe2 using Low-Rank Adaptation (LoRA) on a mixture of AerialMetric-Oblique, AerialMetric-Synthetic, and a small fraction of ground-domain data (Hypersim, MVS-Synth, TartanAir) to preserve generalization.

Key Contributions

AerialMetric Dataset: The introduction of a large-scale, diverse aerial metric depth dataset comprising four complementary subsets. This significantly expands the scale, diversity, and geometric coverage of available aerial depth data, addressing the scarcity of dedicated real-world aerial datasets.
Comprehensive Benchmark: The establishment of a rigorous evaluation protocol for monocular aerial metric depth estimation. This includes systematic analyses of how viewing angle, flight altitude, and camera field-of-view impact prediction accuracy, utilizing the variable-decoupled subset to isolate these factors.
State-of-the-Art Performance: By adapting representative models on the AerialMetric dataset, the authors achieve state-of-the-art performance across diverse UAV deployment scenarios. They demonstrate that the dataset effectively improves the generalization capability of monocular metric depth estimation methods in aerial settings.

Results

Domain Gap: Zero-shot evaluation reveals a severe domain gap. For instance, on the AerialMetric-Oblique City subset without ground-truth intrinsics, the baseline MoGe2 achieves a $\delta_1$ accuracy of only 5.1%, whereas the fine-tuned MoGe2-Aerial achieves 89.3%.
Robustness to Parameters: The fine-tuned model exhibits high isotropic performance across varying pitch angles, altitudes, and FOVs. In contrast, zero-shot foundation models show catastrophic failures at specific angles (e.g., strictly nadir or oblique views) and suffer precipitous drops in accuracy when altitude increases (e.g., from 80m to 120m).
Generalization: The adapted model maintains competitive performance on ground-domain benchmarks (NYUv2, KITTI, ETH3D, etc.), indicating that the aerial adaptation does not lead to catastrophic forgetting of terrestrial capabilities.
Wild Evaluation: On the challenging AerialMetric-Wild dataset, the adapted model reduces the AbsRel error from 55.26 to 21.34 and improves $\delta_1$ from 9.5 to 53.7 compared to the zero-shot baseline.

Significance

The paper claims that AerialMetric bridges the critical gap between ground-level and aerial depth estimation. By providing a standardized benchmark with reliable metric ground truth and a controlled variable-decoupled subset, the work enables the community to systematically assess and improve aerial perception. The results demonstrate that while existing models struggle with aerial viewpoints, targeted adaptation using high-quality aerial data can yield robust, metric-scale depth estimation suitable for real-world UAV applications such as infrastructure inspection, environmental monitoring, and autonomous delivery. The authors position this work as a foundational step toward reliable 3D perception for autonomous aerial robotics.