💻 computer science

An Open-Source LiDAR and Monocular Off-Road Autonomous Navigation Stack

This paper presents an open-source off-road navigation stack that achieves LiDAR-comparable robustness using a lightweight monocular setup combining zero-shot foundation models with sparse SLAM-based metric rescaling, temporal smoothing, and edge-masking for reliable obstacle detection in unstructured terrain.

Original authors: Rémi Marsal, Quentin Picard, Adrien Poiré, Sébastien Kerbourc'h, Thibault Toralba, Clément Yver, Alexandre Chapoutot, David Filliat

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rémi Marsal, Quentin Picard, Adrien Poiré, Sébastien Kerbourc'h, Thibault Toralba, Clément Yver, Alexandre Chapoutot, David Filliat

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot dog to run through a dense, messy forest. To do this safely, the robot needs to "see" the world in 3D so it doesn't trip over roots, bump into rocks, or get stuck in tall grass.

Traditionally, robots use LiDAR for this. Think of LiDAR as a high-tech, super-expensive flashlight that shoots out invisible laser beams to map the forest. It's incredibly accurate, but it's heavy, eats up a lot of battery power, and costs as much as a luxury car.

This paper introduces a clever new way to do the same job using just a single camera (like the one on your smartphone) and some very smart AI. Here is the story of how they did it, explained simply:

1. The Problem: The "Guessing Game"

If you just show a robot a photo and ask, "How far away is that tree?", the robot has to guess. Modern AI (called "Foundation Models") is great at guessing depth from a single picture, but it's not perfect.

The Issue: Sometimes the AI gets confused. It might think a shadow is a hole, or a blurry bush is a giant wall. If the robot believes these "ghost obstacles" are real, it will stop moving or crash.
The Cost: The AI is also very hungry for computer power, which slows the robot down.

2. The Solution: The "Smart Glasses" Upgrade

The authors built a complete navigation system (a "stack") that can switch between the expensive laser (LiDAR) and the cheap camera. When using the camera, they added two special "safety nets" to fix the AI's guessing game:

The "Blur Eraser" (Edge Masking):
Imagine looking at a painting of a tree. The edges are often blurry. The AI might think that blur is a solid wall. The researchers taught the system to ignore the fuzzy edges of objects. It's like telling the robot, "Don't worry about the fuzzy outline; just look at the clear center of the object." This stops the robot from seeing "ghost walls" that don't exist.
The "Steady Hand" (Temporal Smoothing):
The robot uses a separate system (SLAM) to figure out where it is in the world. Sometimes, this system gets a little jittery, like a shaky hand holding a camera. If the robot reacts to every tiny shake, it will panic. The researchers added a "smoothing" filter that averages out these shakes, telling the robot, "Take a breath, don't overreact to that one wobble."

3. The Map: From 3D to a "Top-Down" View

Once the robot has a 3D point cloud (a digital cloud of dots representing the world), it needs to plan a path.

The Cloth Trick: The forest floor isn't flat; it has bumps and hills. To separate the ground from the obstacles (like rocks), the system uses a "Cloth Simulation Filter." Imagine dropping a stiff sheet of fabric over the 3D dots. The fabric will drape over the ground but get stuck on top of the rocks. The space between the fabric and the rocks is where the obstacles are. This creates a clean map of what is walkable and what isn't.

4. The Test: The Forest Run

They tested this system in two places:

A Super-Realistic Video Game: They built three levels of difficulty (Easy, Medium, Hard) in a simulator.
The Real World: They put the system on a real robot (a wheeled rover) and drove it through actual off-road terrain.

The Results:

The Camera vs. The Laser: In most situations, the camera-only setup performed just as well as the expensive laser system. It successfully navigated through trees and rocks.
The Weak Spot: The system struggled with tall, swaying grass. Because grass doesn't have hard edges, the "Blur Eraser" couldn't tell the difference between "walkable grass" and "blocking obstacle." The robot treated the grass like a wall and stopped.
Speed: The camera system was a bit slower than the laser system because the AI had to work harder to process the images.

The Big Takeaway

This paper proves that you don't need a $20,000 laser scanner to build a robot that can navigate rough terrain. With a standard camera and some clever software tricks (the "Blur Eraser" and "Steady Hand"), you can build a robot that is almost as good, much cheaper, and uses less battery.

They even made all their code and their "video game" test environments free for everyone to use, so other scientists can build on their work. It's like giving the whole robotics community a free, open-source blueprint for a smart, off-road explorer.

1. Problem Statement

Off-road autonomous navigation requires robust 3D perception to detect obstacles in unstructured, challenging terrain. While LiDAR is the standard for high accuracy, it suffers from high cost, high power consumption (approx. 20W vs. 2W for cameras), and detectability issues. Stereo vision offers a cheaper alternative but struggles with low-textured regions, transparency, glare, and limited depth range.

Existing monocular depth estimation methods based on foundation models (zero-shot learning) show promise but face significant hurdles in outdoor navigation:

Metric Depth Recovery: Foundation models typically output normalized (relative) depth, not metric depth.
Hallucinations: Blurry object borders in depth maps can create "phantom obstacles" that block the robot.
Temporal Instability: Scaling parameters derived from SLAM can fluctuate, causing jitter in the point cloud.
Lack of Integration: There is a scarcity of complete, open-source navigation stacks that integrate these monocular methods for off-road use without requiring task-specific retraining.

2. Methodology

The authors propose a unified, open-source navigation pipeline compatible with both LiDAR and Monocular Camera inputs. The system is built on ROS Noetic and consists of three main modules:

A. 3D Perception Module

LiDAR Path: Directly ingests LiDAR point clouds.
Monocular Path:
1. Zero-Shot Depth Estimation: Uses Depth Anything V2 to generate a normalized disparity map from a single camera image.
2. Metric Rescaling: Combines the normalized map with sparse metric depth measurements from a Visual-Inertial SLAM (VINS-Mono) to recover metric depth.
3. Edge Masking (Post-hoc Adaptation): To prevent "phantom obstacles" caused by blurry depth boundaries, a Sobel filter detects edges in the disparity map. Pixels within a 5-pixel radius of these edges are masked out before 3D back-projection.
4. Temporal Smoothing: To mitigate SLAM instability, the scaling and shift parameters ( $\hat{s}_t$ ) are smoothed using an Exponential Moving Average (EMA) with $\alpha = 0.8$ .

B. Ground Segmentation & Obstacle Detection

Cloth Simulation Filter (CSF): The resulting point cloud (from LiDAR or Monocular) is processed using CSF to separate ground points from obstacles. This is crucial for non-flat off-road terrain.
Elevation Mapping: The filtered obstacles are converted into a robot-centric 2.5D elevation map. A memory buffer retains obstacle positions even after they leave the sensor's field of view to improve planning robustness.
Costmap Generation: The elevation map is converted into a binary costmap using a height threshold (30cm in experiments).

C. Path Planning

Global Planner: Uses A* on the inflated costmap to generate a high-level path.
Local Planner: Uses Timed-Elastic-Bands (TEB) to generate velocity commands, allowing for flexible path adjustments to handle perception errors and dynamic obstacles.
Localization: Achieved via an Extended Kalman Filter (EKF) fusing GNSS, IMU, and SLAM pose estimates.

3. Key Contributions

Open-Source Navigation Stack: A complete, ready-to-use pipeline for off-road reactive navigation that supports both LiDAR and monocular vision without requiring additional training or fine-tuning for new environments.
Foundation Model Integration: Successfully integrates zero-shot monocular depth estimation (Depth Anything V2) with SLAM-based rescaling for metric 3D perception in unstructured environments.
Robustness Enhancements: Introduces specific post-processing techniques (edge masking and temporal smoothing) to solve common failure modes of monocular depth in navigation (hallucinations and jitter).
Comprehensive Benchmark: Provides extensive evaluation in both photorealistic simulations (Isaac Sim) and real-world unstructured environments, along with the release of the simulation environments for reproducibility.

4. Experimental Results

The system was evaluated on a Barakuda robot (equipped with Ouster LiDAR and ZED2i camera) across three difficulty levels (Easy, Medium, Hard) in both simulation and real-world settings.

Performance Metrics: Success Rate (SR), Success weighted by Path Length (SPL), and Distance Ratio (DR).
Simulation Results:
- The monocular configuration (Mono-VINS) achieved competitive results against the LiDAR baseline.
- In the 20m goal scenario, Mono-VINS outperformed LiDAR (93% SR vs. ~67% for LiDAR real-params), largely because the LiDAR baseline suffered from CSF tuning issues in that specific scenario.
- The monocular setup struggled in the Hard 30m scenario (10% SR) due to high grass, which creates large blurry regions in the depth map that edge filtering cannot fully resolve, leading to false obstacle detection.
Real-World Results:
- The monocular configuration achieved a 100% Success Rate and 1.0 Distance Ratio across all scenarios, matching the LiDAR baseline.
- Efficiency Trade-off: The monocular setup had a 22% lower SPL (Success weighted by Path Length), meaning the robot traveled further to reach the goal. This is attributed to:
  - Longer processing chains (depth estimation latency).
  - Limited Field of View (FOV) and depth accuracy causing the robot to turn later or oscillate when avoiding obstacles.
Ablation Study:
- Enabling both Edge Masking and Temporal Smoothing improved the average SPL by significant margins (e.g., 21% improvement over configurations without edge masking), proving their necessity for robust navigation.
Computational Frequency:
- LiDAR configuration runs at 20 Hz for perception.
- Monocular configuration runs at 10 Hz due to the GPU overhead of Depth Anything V2 inference, which also reduces the elevation mapping frequency from 16 Hz to 6 Hz.

5. Significance and Future Work

Viability of Monocular Navigation: The paper demonstrates that foundation-model-based monocular depth estimation is a viable, low-cost, and low-power alternative to LiDAR for off-road navigation, provided specific robustness techniques are applied.
Reproducibility: By open-sourcing the stack and simulation environments, the authors provide a standardized benchmark for future research in off-road autonomy.
Limitations & Future Directions:
- High Grass: The system currently treats tall grass as an obstacle. Future work aims to integrate traversability modules to classify grass as passable.
- Negative Obstacles: The current stack does not handle pits or holes (negative obstacles) well due to lack of visual information.
- ROS2 Migration: The authors plan to migrate the stack from ROS1 to ROS2 for better long-term maintainability.

In conclusion, this work bridges the gap between advanced computer vision foundation models and practical robotic navigation, offering a flexible, training-free solution that significantly reduces hardware costs while maintaining robust performance in complex off-road environments.