💻 computer science

Cross-Session 3D LiDAR and Camera Fusion for Robust Localization of Unmanned Aerial Vehicles in GPS-Denied Environments

This paper proposes Cross-Fusion, a novel real-time localization method for UAVs in GPS-denied environments that integrates 3D LiDAR odometry with monocular camera features and employs a cross-session fusion strategy to achieve GPS-level accuracy and robustness in challenging, feature-sparse settings.

Original authors: Cong Hoang Quach, Chi Thanh Vo, Dong LT. Tran, Truong Son Nguyen, Manh Duong Phung, Thuan Hoang Tran

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cong Hoang Quach, Chi Thanh Vo, Dong LT. Tran, Truong Son Nguyen, Manh Duong Phung, Thuan Hoang Tran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are flying a drone to inspect a bridge, but you are flying inside a long tunnel or under a massive overpass. In these places, your GPS signal is like a radio station that suddenly goes silent; you can't tell where you are. If the drone loses its sense of direction, it might crash or get lost.

This paper introduces a new way to help drones find their way in these "GPS-denied" zones. The authors call their method Cross-Fusion. Here is how it works, broken down into simple concepts:

The Problem: The "Drunk Walk"

Most drones rely on GPS, but when that's gone, they have to guess their position using sensors.

The Camera: It's like a human eye. It's great at recognizing patterns, but if the walls are plain white or the lights are dim, it gets confused.
The LiDAR: This is like a bat using echolocation. It shoots laser beams to measure distance. It works in the dark, but if the tunnel is a long, flat, empty hallway, the laser beams don't hit anything interesting, and the drone starts to "drift" (get lost) over time.
The Drift: If a drone just guesses its movement step-by-step without a map, small errors add up. It's like walking blindfolded in a straight line; after a few minutes, you might think you are facing North when you are actually facing East.

The Solution: The "Memory Book" and the "Flashlight"

The authors' solution, Cross-Fusion, is a team effort between a camera and a LiDAR sensor, but with a special twist: it uses a pre-made memory book.

1. The Offline Phase (Writing the Memory Book)

Before the drone ever flies the mission, humans or other robots take a "baseline survey" of the area. They take hundreds of photos and scan the area with lasers.

Think of this as creating a detailed photo album and a 3D model of the tunnel or bridge.
The computer studies these photos and learns what the place looks like from every angle. It builds a "Global Map" that acts as a reference guide.

2. The Online Phase (The Flight)

Now, the drone flies the mission. It has two main tools:

The LiDAR (The Flashlight): It constantly scans the ground and walls to figure out how it is moving right now. This is fast and reliable for short distances.
The Camera (The Recognizer): As the drone flies, it takes pictures. Instead of just looking at the picture to guess where it is, the drone compares its current photo against the Memory Book it built earlier.

3. The "Cross-Fusion" Magic

Here is the clever part. When the drone takes a picture, it doesn't just look for a matching photo.

It finds a matching photo in the Memory Book.
It then uses its LiDAR to look at the 3D shape of the object in that photo.
It matches the 3D shape it sees now with the 3D shape stored in the book.

The Analogy: Imagine you are walking through a dark forest. You have a flashlight (LiDAR) to see the path immediately in front of you, but you are starting to lose your way. Suddenly, you see a unique rock formation. You pull out a map (the Memory Book) and realize, "Ah! That rock is on page 42 of my map." You instantly know exactly where you are, correcting your mistake.

Why is this special?

No GPS Needed: It works perfectly in tunnels, under bridges, or inside buildings.
Simple Hardware: It doesn't need expensive, heavy, or complex setups (like multiple cameras or high-end inertial sensors). It just needs one 3D laser scanner and one regular camera.
Staying Power: Because it keeps checking its position against the "Memory Book," it doesn't drift away like other methods do. Even if the drone flies fast or the camera frame rate drops, it can still find its way.

The Results

The authors tested this on a drone flying around a building.

LiDAR alone: The drone slowly drifted off course, like a car with a broken steering wheel.
Cross-Fusion: The drone stayed on the correct path, matching the GPS reference almost perfectly, even though it wasn't using GPS.
Comparison: They compared it to a very advanced, complex system called FAST-LIVO. While that system is slightly more accurate in perfect conditions, it fails completely if the camera slows down or the drone flies too fast. Cross-Fusion is more robust; it keeps working even when conditions get tough.

The Bottom Line

This paper presents a practical way to keep drones safe and on track in places where GPS doesn't work. By combining a laser scanner with a camera that "remembers" a pre-scanned map, the drone can navigate complex, dark, or featureless environments without getting lost. The code for this system is also open-source, meaning other researchers can use and improve it.

Technical Summary: Cross-Session 3D LiDAR and Camera Fusion for Robust UAV Localization

Problem Statement
Accurate localization is critical for Unmanned Aerial Vehicles (UAVs) performing infrastructure inspection, such as monitoring bridge undersides, tunnels, and urban canyons. In these GPS-denied environments, standard satellite navigation is unavailable or unreliable. While existing solutions exist, they face significant limitations:

Visual-only systems degrade in low-light or feature-sparse environments and suffer from drift without loop closure.
LiDAR-only systems struggle in geometrically degenerate scenes (e.g., flat surfaces or long tunnels) and accumulate drift over time due to the lack of global constraints.
Tightly coupled Visual-Inertial-LiDAR (VIL) systems (e.g., FAST-LIVO, LVI-SAM) offer high accuracy but often require complex sensor setups (stereo cameras, IMUs) and struggle when frame rates drop or visual overlap is limited.
Learning-based methods often require extensive training data and may fail under domain shifts (lighting, viewpoint changes).

Furthermore, single-session data is often insufficient for consistent localization in complex structures where lighting varies and features are unevenly distributed.

Methodology: Cross-Fusion
The authors propose Cross-Fusion, a hybrid localization framework that integrates 3D LiDAR and a monocular camera. The system operates in two distinct phases:

Offline Phase (Database Construction):
- A baseline visual-geometric database is constructed using data from prior inspection sessions (e.g., ground robots or manual drone flights).
- Feature Extraction: Local keypoints and descriptors are extracted using CNN-based methods (SuperPoint or DISK).
- Global Retrieval: Global descriptors (EigenPlaces) are computed for each image to enable rapid retrieval of candidate images from large databases.
- Map Generation: Overlapping image pairs are matched (SuperGlue) and processed through a Structure-from-Motion (SfM) pipeline (COLMAP) to generate a sparse 3D point cloud map ( $X_i$ ) with known camera poses.
Online Phase (Real-time Localization):
- LiDAR Odometry: The UAV estimates its coarse pose ( $P_t$ ) by aligning consecutive LiDAR scans using Iterative Closest Point (ICP) or similar methods. This provides continuous motion tracking but is prone to drift.
- Visual Correction:
  - Coarse Retrieval: The current camera image is compared against the offline database using global descriptors to identify the top- $k$ most similar images.
  - Fine Matching: Local features are matched between the current image and the candidate database images using SuperGlue.
  - Cross-Session 2D-3D Transfer: This is the core innovation. Matched 2D keypoints are projected onto the online LiDAR point cloud ( $L_i$ ) and the offline 3D map ( $X_i$ ). This creates 3D-3D correspondences between the current LiDAR scan and the pre-built map.
  - Pose Refinement: A truncated point-cloud registration problem is solved to minimize the distance error between the LiDAR points and the map points, yielding a corrected 6-DoF pose relative to the global map.

Key Contributions
The paper identifies four primary contributions:

Cross-Session Fusion Strategy: A novel pipeline that associates online UAV observations with a pre-built visual-geometric database. This provides global pose constraints to correct drift in feature-sparse environments without requiring GPS.
LiDAR-Guided 2D-3D Correspondence Transfer: A mechanism that lifts 2D-2D image matches into 3D-3D point-cloud correspondences by projecting matched keypoints through the online LiDAR stream. This enables 6-DoF pose correction without relying on an IMU or stereo camera.
Robust and Lightweight Sensor Setup: The system achieves competitive accuracy using only a 3D LiDAR and a monocular camera, avoiding the complexity of tightly coupled visual-inertial systems. It demonstrates greater robustness under low frame rates or degraded visual conditions.
Open-Source Implementation: The authors provide the code to support reproducibility.

Experimental Results
Experiments were conducted in a 150m x 150m building area using a quadcopter equipped with a Quanergy M8 LiDAR and an Intel RealSense D435 camera. GPS data was used solely as a ground truth reference.

Drift Reduction: The proposed method significantly reduced localization error compared to LiDAR-only odometry. While LiDAR-only trajectories accumulated drift over time, the Cross-Fusion method maintained alignment with the GPS reference by leveraging visual corrections from the offline database.
Robustness vs. State-of-the-Art: When compared to FAST-LIVO (a tightly coupled LIVO SLAM method), Cross-Fusion showed slightly lower accuracy under normal conditions. However, it demonstrated superior robustness in extreme scenarios. Specifically, when the camera frame rate was reduced to below 1 fps, FAST-LIVO failed to estimate the trajectory due to its reliance on tight coupling and continuous overlap, whereas Cross-Fusion maintained stable localization.

Significance and Claims
The paper claims that Cross-Fusion offers a practical solution for the "low-altitude economy," specifically for routine infrastructure inspection in GPS-denied environments. By leveraging pre-existing baseline surveys (common in structural health monitoring), the system bridges the gap between the simplicity of single-sensor odometry and the complexity of tightly coupled multi-sensor systems.

The authors modestly acknowledge limitations: the system currently depends on a pre-collected image database, which limits its use in dynamic scenarios where pre-mapping is impractical. Future work is proposed to integrate data from multiple online sessions (e.g., coordinating ground and UAV agents) to enable real-time map construction without relying solely on prior static data. The work does not claim to replace all existing SLAM methods but rather provides a balanced trade-off between accuracy, robustness, and hardware simplicity for specific inspection use cases.