💻 computer science

MapAnything: Evaluating Monocular Metric Depth Models for 3D Urban Asset Localization

This paper introduces MapAnything, a novel framework that automates the 3D geo-localization of urban objects and incidents from single monocular images by leveraging metric depth estimation models, achieving high accuracy validated against LiDAR data for scalable urban asset management.

Original authors: Miriam Louise Carnot, Jonas Kunze, Erik Quinten Fastermann, Eric Peukert, André Ludwig, Bogdan Franczyk

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Miriam Louise Carnot, Jonas Kunze, Erik Quinten Fastermann, Eric Peukert, André Ludwig, Bogdan Franczyk

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a city manager trying to keep a perfect, up-to-date map of everything in your city: every traffic sign, every tree, every pothole, and even every piece of graffiti. Traditionally, doing this is like trying to measure the ocean with a teaspoon. You need expensive trucks with laser scanners (LiDAR) or teams of people walking around with clipboards. It's slow, costly, and by the time you finish, the data is already outdated because a sign was stolen or a new pothole appeared.

This paper introduces MapAnything, a new tool that tries to solve this problem using something we all have: a single photo taken from a regular camera (like on a bus, a taxi, or even a smartphone).

Here is how it works, broken down into simple concepts:

The Magic Trick: Turning a Flat Photo into a 3D Map

Think of a regular photo as a flat painting. It tells you what something looks like, but not how far away it is. If you see a traffic sign in a photo, you don't know if it's 5 meters away or 50 meters away just by looking at the picture.

The researchers used a special type of AI called a "Monocular Metric Depth Estimation" model. You can think of this AI as a super-estimator. When you feed it a single photo, it doesn't just see a picture; it "hallucinates" a 3D depth map. It assigns a specific distance in meters to every single pixel in the image.

The Analogy: Imagine looking at a flat drawing of a forest. A normal person sees trees. This AI sees the drawing and instantly knows, "That tree is 10 meters away, that bush is 3 meters away, and that mountain is 100 meters away," all without ever having visited the forest.

The Math: From "How Far" to "Where"

Once the AI knows how far away an object is, the system uses some basic geometry (like the rules of a triangle) to figure out exactly where that object is on the globe.

The Camera's Position: We know where the camera was (from GPS) and which way it was facing.
The Angle: We know where the object is in the photo (left, right, up, down).
The Distance: The AI tells us how far the object is.

By combining these three pieces of information, the system draws a line from the camera to the object and calculates the exact GPS coordinates. It's like a navigator saying, "I am here, I am looking North, and that sign is 15 meters away to my right; therefore, the sign is at this specific street corner."

The Experiment: Testing the "Super-Estimators"

The researchers didn't just guess; they tested four of the newest and smartest AI models (named DepthAnything, DepthPro, UniDepth, and Metric3D) to see which one was the best at guessing distances.

They compared the AI's guesses against LiDAR point clouds.

The Analogy: Imagine LiDAR is a high-tech laser scanner that measures the city with laser precision, creating a perfect 3D model. The researchers took photos of the same spots, ran them through the AI, and checked: "Did the AI guess the distance correctly compared to the laser?"

The Results:

The Winners: UniDepth and Metric3D were the champions. They were the most accurate at guessing distances, especially for things like flat roads and buildings.
The Distance Factor: The AI was very good at guessing distances for objects close to the camera (like a pothole right in front of a car). However, as objects got farther away (beyond 20 meters), the guesses got a bit fuzzier, much like how it's hard to judge the distance of a mountain compared to a tree in your yard.
The "Nature" Problem: The AI struggled a bit with trees and vegetation. It's hard to guess the distance of a leafy tree because it's not a solid, flat surface like a wall or a road.

Real-World Tests: Signs and Potholes

The team tested their system on two real-world tasks:

Traffic Signs: They tried to find and map traffic signs using photos from professional street cameras and crowd-sourced photos (from apps like Mapillary).
- Success: Using the best AI (UniDepth), they found 87% of the signs that should have been visible in the photos.
- Bonus: They actually found more signs than the city's official database had recorded, including temporary signs that the city had forgotten to log.
- Accuracy: For signs within 20 meters, the location was very accurate (usually within a few meters).
Road Damage (Potholes): They looked for cracks and holes in the road.
- Success: Because potholes are usually close to the camera (at the bottom of the photo), the system was very accurate. The errors were smaller here than with the traffic signs.

Why This Matters

The paper concludes that this method is a game-changer for city maintenance.

No Expensive Hardware: You don't need a $100,000 laser truck. You just need a camera.
Crowd-Sourcing: Since it works with single images, cities could potentially use photos taken by garbage trucks, buses, or even citizens reporting issues (like illegal dumping or broken signs) to update their maps automatically.
Keeping the "Digital Twin" Alive: Cities are building "Digital Twins" (virtual copies of the real city). This tool allows them to keep that virtual copy updated continuously and cheaply, rather than waiting years for the next expensive survey.

The Bottom Line:
The paper proves that we can turn a simple 2D photo into a 3D map of the city with surprising accuracy. While it's not perfect for things very far away or hidden in trees, it is accurate enough to help cities find missing signs, map new potholes, and keep their digital maps up to date without breaking the bank.

Technical Summary: MapAnything

Problem Statement
City administrations face significant challenges in maintaining comprehensive, up-to-date spatial databases and urban digital twins for infrastructure assets (e.g., traffic signs, trees) and incidents (e.g., graffiti, road damage). Current data acquisition methods rely heavily on expensive, resource-intensive technologies like LiDAR sensors, aerial photogrammetry, or manual surveys, which limit the frequency of data collection and scalability. Existing alternatives, such as using only camera coordinates or requiring multi-view triangulation, often lack the precision needed for accurate mapping or fail when objects are not visible across multiple images. Furthermore, many approaches depend on prior knowledge of object dimensions, which is not always available. There is a need for a scalable, low-cost method to extract 3D spatial data from single 2D street-view images without requiring prior object knowledge or overlapping viewpoints.

Methodology
The paper introduces MapAnything, a geo-localization framework that automates the spatial mapping of urban objects from a single monocular image. The methodology consists of three primary steps:

Object Detection/Segmentation: The system identifies the target object within the image using manual annotation or deep learning models (e.g., U-Net, Vision Transformers).
Metric Depth Estimation and Geometric Calculation:
- The framework employs Monocular Metric Depth Estimation models to predict a dense depth map (in meters) for every pixel in the image.
- Using the estimated depth ( $d$ ) at the object's location (center of a bounding box or weighted average of a segmentation mask) and the camera's intrinsic and extrinsic parameters, the system calculates the object's 3D position.
- The process involves normalizing pixel coordinates, calculating angular offsets relative to the camera's optical axis, and adjusting for the camera's yaw and pitch.
- Horizontal distances and coordinate deltas ( $\Delta x, \Delta y$ ) are derived using trigonometric relationships, which are then converted into latitude and longitude changes using the Earth's radius.
De-duplication and Matching: The framework filters duplicate detections from multiple images by calculating the minimum distance between objects of the same class and merges predictions within a defined radius. It also matches predictions against existing database entries using angular offsets and distance thresholds.

The authors evaluate four state-of-the-art depth estimation models: DepthAnything, DepthPro, UniDepth, and Metric3D.

Key Contributions

MapAnything Framework: A scalable pipeline for extracting geo-coordinates of urban assets from single 2D street-view images, eliminating the need for multi-view data or prior object size knowledge.
Comprehensive Model Evaluation: A detailed benchmarking of four recent monocular metric depth models against high-precision LiDAR point clouds. The evaluation analyzes performance across various distance intervals (<5m to >20m) and semantic categories (flat surfaces, construction, objects, nature).
Practical Case Studies: Validation of the framework's utility through two real-world applications:
- Traffic Sign Mapping: Comparing predicted locations against a city council database and manual annotations.
- Road Pavement Damage Detection: Assessing localization accuracy for damage instances.

Results

Depth Estimation Accuracy:
- UniDepth and Metric3D (ViT backbone) generally produced the most accurate results when compared to LiDAR ground truth.
- UniDepth performed best for shorter distances (<20m) and large man-made structures (flat/construction groups).
- Metric3D showed superior performance for objects at greater distances and in "object" and "nature" categories, though all models struggled with the fine details of vegetation.
- DepthPro produced sharp contours but exhibited higher absolute errors in certain distance ranges.
- Inference time varied significantly, with the small UniDepth model being the fastest and DepthPro the slowest.
Case Study 1 (Traffic Signs):
- Using high-quality Cyclomedia imagery, the system equipped with UniDepth successfully localized 87% of theoretically visible signs.
- The system detected more signs than currently recorded in the city database, including many temporarily mounted signs that were missing from official records.
- Localization accuracy was highest for signs within 20 meters; error increased significantly beyond this range.
- Crowd-sourced Mapillary data presented greater challenges due to GPS inaccuracies, but the majority of signs were still detectable.
Case Study 2 (Road Damage):
- Localization errors for road damage were generally lower than for traffic signs, attributed to the damage being closer to the camera (average 7.55m).
- DepthAnything yielded the smallest deviation overall, while UniDepth performed best for damage within six meters.
Error Analysis: An ablation study indicated that the segmentation step contributed minimally to the overall coordinate error, with depth estimation being the primary source of deviation.

Significance and Claims
The paper claims that MapAnything offers a highly scalable, low-cost alternative to traditional LiDAR-based mapping, enabling city administrations to efficiently update urban digital twins using standard imagery. The framework is particularly effective for mapping objects within an estimated 20-meter distance, where spatial deviations consistently remain below a five-meter operational threshold. This threshold is deemed sufficient for practical municipal inventory management, as even manually measured coordinates in existing databases can exhibit deviations of up to 5 meters while retaining operational value.

The authors emphasize the framework's potential for continuous maintenance and temporal change detection. By processing street-view imagery from different dates, the system can automatically flag missing or damaged assets without commissioning new surveys. Additionally, the approach is positioned as a tool for integrating citizen reporting (e.g., user-submitted photos of anomalies) into automated urban management systems, providing geo-coordinates for anomalies like illegal dumping or severe road damage without requiring specialized hardware.