💻 computer science

Simultaneous Localization and 3D-Semi Dense Mapping for Micro Drones Using Monocular Camera and Inertial Sensors

This paper proposes a real-time, edge-aware lightweight monocular SLAM system for micro drones that fuses deep learning-based depth prediction with inertial data via an extended Kalman filter to achieve accurate 3D semi-dense mapping and autonomous navigation on resource-constrained platforms without relying on global loop closure.

Original authors: Jeryes Danial, Yosi Ben Asher, Itzik Klein

Published 2026-09-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jeryes Danial, Yosi Ben Asher, Itzik Klein

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a tiny drone, no larger than a hand, trying to navigate a cluttered hallway. To move safely without crashing, it must instantly understand the shape of the world around it. It needs to know where it is, where the walls are, and how far away the obstacles lie. For decades, engineers have struggled to give these small machines this kind of vision. The cameras they carry are often too weak to build a detailed picture of the room, while the powerful computers needed to process complex images are too heavy and drain the battery too quickly. This creates a difficult trade-off: either the drone sees the world as a sparse collection of dots, missing most of the structure, or it tries to build a rich, detailed map but runs out of power before it can finish.

The solution to this problem lies in a new approach developed by researchers at the University of Haifa. They created a system that allows a micro-drone to build a semi-detailed 3D map of its surroundings in real time, using only a single camera and basic motion sensors. Instead of trying to map every single point in a room, which is computationally expensive, their method focuses on the edges—the sharp lines where walls meet floors or where objects stand out against the background. By combining these sharp visual lines with a lightweight prediction of how deep the scene is, the drone can construct a clear, usable map of the environment without needing a supercomputer.

The researchers tested this system on a small, commercially available drone equipped with a single camera and inertial sensors that measure movement. In the past, similar systems relied on matching specific points, like corners or textures, across different images to figure out where the drone was. While this works well for knowing the general direction, it often fails to capture the full shape of a room, leaving large gaps in the map. Other modern methods use complex artificial intelligence to fill in those gaps, creating dense, realistic 3D models. However, these AI-heavy approaches require massive amounts of computing power and energy, making them impossible to run on the tiny processors found in micro-drones. The new system bridges this gap by using a clever mix of old and new techniques. It uses a lightweight neural network, a type of simple artificial intelligence, to guess the depth of the scene based on the image, and then uses the sharp edges detected in the photo to lock that guess into place.

To make sure the drone knows exactly how far away things are, the system fuses the camera data with information from the drone's motion sensors. Monocular cameras, which use only one lens, have a natural blind spot: they cannot tell the difference between a small object that is close and a large object that is far away. This is known as a scale ambiguity. The researchers solved this by feeding the motion data from the drone's internal sensors into a mathematical filter that constantly corrects the camera's guesses. This process ensures that the map grows at the correct size, preventing the drone from thinking a hallway is ten times longer than it actually is.

The results of this approach are striking. When tested against standard navigation systems on a well-known dataset of indoor environments, the new system proved to be significantly more accurate. The researchers found that the error in the drone's estimated path was reduced by nearly 75 percent compared to the leading traditional method. More importantly, the map produced was not just a collection of scattered points but a coherent structure of edges and surfaces that clearly defined the room's layout. The system operated smoothly on low-power hardware, capable of running on devices as small as a Raspberry Pi Zero, which is a tiny, inexpensive computer board. This means the technology is not just a theoretical concept but a practical tool that could be deployed on real-world drones today.

In practical terms, this means a micro-drone can now fly through a complex indoor space, such as a warehouse or an indoor corridor, and build a reliable 3D map of the obstacles in its path without needing to send data back to a powerful server or carry a heavy battery. The system successfully navigated indoor corridors and avoided obstacles in tests, demonstrating that it can handle the challenges of real-world navigation. By focusing on the edges of objects and using a streamlined method to estimate depth, the researchers have shown that high-quality mapping does not require heavy machinery. Instead, it can be achieved through a smart combination of simple sensors and efficient algorithms, opening the door for a new generation of autonomous drones that can operate independently in the most resource-constrained environments.

Technical Summary: Simultaneous Localization and 3D-Semi Dense Mapping for Micro Drones

Problem Statement
Autonomous navigation for resource-constrained micro-drones requires real-time, accurate 3D mapping and pose estimation. Existing monocular Simultaneous Localization and Mapping (SLAM) systems face significant trade-offs:

Sparse Methods: Classical approaches like ORB-SLAM rely on sparse feature matching and global optimization (loop closure, bundle adjustment). While effective for pose estimation, they produce sparse 3D point maps lacking the geometric richness required for dense perception and interaction. Furthermore, their reliance on computationally intensive global optimization makes real-time deployment on embedded hardware difficult.
Dense/Learning-Driven Methods: Learning-based approaches (e.g., DeepTAM, D3VO, NeRFs) can generate dense maps but are often computationally prohibitive for low-power platforms due to high parameter counts and the need for hardware acceleration (GPUs).
Scale Ambiguity: Monocular SLAM inherently suffers from scale ambiguity, affecting accuracy.
Edge SLAM Limitations: Previous edge-based methods attempt to generate semi-dense maps but often remain computationally heavy due to global optimization dependencies and suffer from noise in tracking loss functions based on optical flow.

The core challenge is developing a system that provides accurate, dense (or semi-dense) mapping and robust pose estimation on low-power platforms without relying on global loop closure or heavy neural computations.

Proposed Methodology
The authors propose a lightweight, hybrid, edge-aware monocular SLAM system designed for real-time operation on platforms like the DJI Tello drone. The system integrates classical geometric vision with lightweight deep learning modules, avoiding global loop closure in favor of local geometric consistency.

The pipeline consists of four main threads:

Image Pre-processing and Feature Extraction:
- ORB keypoints and descriptors are extracted and matched across frames using Hamming distance and KD-trees for speed.
- Canny edge detection is performed on each frame.
- A lightweight, pre-trained FastDepth CNN (based on MobileNet-NNConv5 with depth-wise separable layers) predicts per-frame dense depth maps. Edge pixels are associated with these depth values.
Pose Estimation and Sensor Fusion:
- Relative motion between frames is estimated using epipolar geometry (Essential Matrix decomposition via RANSAC and SVD) based on matched ORB features.
- An Extended Kalman Filter (EKF) fuses the monocular relative pose with inertial data (linear velocity and Euler angles) from the drone's API.
- To address the drift inherent in the drone's API (which worsens as battery levels drop), the process noise covariance in the EKF is dynamically modeled as a function of battery level and time since the last monocular update.
Semi-Dense Edge Map and 3D Anchors Generation:
- Using the predicted depth maps and estimated poses, the system triangulates corresponding features to generate a set of 3D "anchors."
- This thread enforces geometric constraints through three specific local loss functions:
  - Reprojection Loss ( $L_{reproj}$ ): Minimizes the error between observed and projected 3D points.
  - Cycle Consistency Loss ( $L_{cycle}$ ): Enforces that a round-trip transformation of edge pixels (via depth and relative pose) returns to the original pixel location, regularizing pose and depth consistency.
  - Shape-Preserving Loss ( $L_{shape}$ ): A structural constraint based on "L-shape" geometry (corner angles and collinearity of edge segments) to maintain structural coherence in indoor environments.
Edge-Aware Local Optimization:
- A local Bundle Adjustment (BA) framework simultaneously refines camera poses, 3D point locations, and depth edge maps within a sliding temporal window.
- The optimization minimizes a weighted sum of the three loss functions using the Levenberg–Marquardt (LM) algorithm.
- The system addresses non-linearity, singularities, and rank deficiency through regularization, diagonal regularization, and adaptive weighting based on uncertainty.

Key Contributions

Lightweight Hybrid Pipeline: Integration of sparse epipolar geometry with dense depth prediction and edge extraction for edge-anchored mapping, avoiding heavy global optimization.
Novel Loss Functions: Introduction of an edge-cycle consistency loss and a shape-aware structural constraint (L-shape) that maintain view coherence without requiring explicit 2D-2D edge matching.
Multi-Objective Optimization: A local bundle adjustment process that jointly refines poses, keypoints, and edge segments under local geometric constraints, eliminating the need for global loop closure or dense volumetric fusion.
Sensor Fusion for Scale: A method to resolve monocular scale ambiguity by fusing visual relative transformations with inertial measurements via an EKF, utilizing dynamic noise modeling based on battery status.

Experimental Results
The system was evaluated on the TUM RGB-D dataset and demonstrated on a DJI Tello drone.

Quantitative Accuracy: Compared to the ORB-SLAM2 baseline, the proposed Edge-Aware SLAM achieved a 74.7% reduction in Root Mean Square Error (RMSE) for the Absolute Trajectory Error (ATE). The mean error decreased from 0.17m to 0.040m, and the standard deviation dropped significantly from 0.71m to 0.011m.
Qualitative Mapping: Visualizations show that the system produces clear, accurate, and coherent 3D edge maps early in the sequence, whereas ORB-SLAM2's point cloud is less interpretable at the same stage. The proposed method maintains high accuracy and shows no drift over time, unlike the baseline which exhibits less clarity and interpretability.
Performance: The system operates in real-time on resource-constrained platforms. The authors note that their ORB-based edge map creation runs approximately 100 times faster than ORB-SLAM on a standard laptop, facilitating deployment on small hardware like the Raspberry Pi Zero.

Significance and Claims
The paper claims to offer an effective, practical solution for real-time mapping and navigation in resource-constrained environments. By combining sparse keypoints with dense edge-aware depth, the system enhances map detail and robustness without the computational burden of global loop closure or large neural networks. The approach enables robust autonomous navigation and obstacle avoidance in indoor corridors, providing a viable alternative for embedded autonomous systems that require dense 3D understanding but lack high-end GPU capabilities. The work demonstrates that classical geometric methods, when augmented with lightweight deep learning for depth and edge estimation, can overcome the limitations of both purely sparse and purely learning-driven SLAM approaches.

Technical Summary: Simultaneous Localization and 3D-Semi Dense Mapping for Micro Drones

More like this