Evaluating and Improving the Robustness of LiDAR Odometry and Localization Under Real-World Corruptions

1. Problem Statement

LiDAR odometry and localization are critical for autonomous driving and robotics, yet their robustness against real-world data corruptions remains largely unexplored. While State-of-the-Art (SOTA) systems perform well on clean point cloud data, they are highly sensitive to environmental factors such as adverse weather (rain, snow, fog), sensor noise, and occlusions.

• The Gap: Existing literature focuses on specific aspects (e.g., occlusion or adversarial attacks) but lacks a comprehensive benchmark evaluating how a wide range of realistic corruptions affect both classical and learning-based pose estimation systems.
• The Consequence: Under corruption, odometry position errors can escalate from 0.5% to over 80%, potentially leading to catastrophic navigation failures.

2. Methodology

The authors propose RobustLOL, a systematic framework designed to evaluate and improve the robustness of LiDAR pose estimation.

A. Robustness Benchmark (RobustLOL)

The framework introduces 18 realistic synthetic corruptions categorized into three groups, applied to standard datasets (KITTI and Apollo South-Bay):

1. Weather Perturbations (5 types): Rain, snow, fog, and combinations with wet ground (simulating surface reflections). These use physics-based models (e.g., LISA simulator) to degrade laser pulse returns.
2. Noise Perturbations (8 types): Simulating internal/external sensor errors.
   • Coordinate Systems: Applied in both Cartesian (CCS) and Spherical (SCS) coordinates.
   • Distributions: Gaussian, Uniform, and Impulse noise.
   • Synthesis: Includes background noise (random points in space) and up-sampling (points near existing ones) to mimic dust or light interference.
3. Density Perturbations (5 types): Simulating equipment malfunction or occlusion.
   • Includes local density increase/decrease, beam deletion, layer deletion, and cutout.

B. Defense Strategies

The paper proposes two complementary strategies to mitigate these errors:

1. Detection-and-Filter Pipeline (Algorithm-Agnostic):

   • Stage 1 (Detection): Converts 3D LiDAR scans into 2D range images (64x1024 grid) and uses a lightweight Convolutional Neural Network (CNN) to classify the corruption type (e.g., rain, noise, density change).
   • Stage 2 (Filtering): Applies a specific filter based on the classification. For noise-induced corruptions, a bilateral filter is used to restore point cloud quality before feeding it to the odometry system.

2. Fine-Tuning for Learning-Based Systems:

   • For deep learning models, the authors augment the training dataset with the specific corruptions that degrade performance.
   • Pre-trained models are fine-tuned on this augmented dataset to learn invariant features, thereby improving generalization to corrupted inputs.

3. Key Contributions

• RobustLOL Benchmark: The first comprehensive evaluation framework testing LiDAR odometry/localization against 18 distinct real-world corruption types.
• Lightweight Detection-and-Filter: A novel, real-time capable pipeline that accurately classifies corruptions and restores data quality, effectively defending non-learning systems against noise.
• Fine-Tuning Strategy: Demonstrates that fine-tuning learning-based models on corrupted data significantly boosts robustness, often restoring performance to near-clean levels.
• Open Source: Public release of code and data to ensure reproducibility.

4. Experimental Results

The authors evaluated four odometry systems (MULLS, KISS-ICP, DeLORA, NeRF-LOAM) and one localization system (LocNDF).

• Vulnerability Analysis:

  • Non-Learning Systems: Showed high sensitivity to specific corruptions. For example, KISS-ICP suffered a massive performance drop (error rising from <0.6% to >80%) under background noise due to disruption of direct point-to-point registration. MULLS failed significantly under local density decreases.
  • Learning-Based Systems: Generally more sensitive to geometric distortions but surprisingly robust to certain noise types (e.g., background noise) because their 2D range image projection filters out redundant 3D points. However, NeRF-LOAM showed high instability and variance across runs.
  • Localization: LocNDF remained robust across all tested corruptions.

• Effectiveness of Defense Strategies:

  • Detection-and-Filter: Successfully reduced errors for noise-corrupted data. For KISS-ICP, errors returned to the clean-data baseline. For MULLS, the pipeline caused negligible performance degradation (<0.1%).
  • Fine-Tuning: Fine-tuning DeLORA on augmented data reduced errors by up to 35.79% across all corruption types. Notably, this also improved performance on clean data for the KITTI 09 sequence (2.7% improvement), though it showed a slight decrease on KITTI 10.

5. Significance

This work highlights that even SOTA LiDAR systems are fragile in real-world conditions, with errors potentially exceeding 80%. It provides a standardized benchmark for future research and offers practical, immediate solutions:

1. For Classical Systems: A modular detection-and-filter pipeline can be easily integrated to handle noise.
2. For Learning Systems: Data augmentation via fine-tuning is a highly effective, low-cost method to harden models against environmental corruptions.

The findings underscore the necessity of robustness testing in the development cycle of autonomous systems before deployment in diverse, unstructured environments.