Splatblox: Traversability-Aware Gaussian Splatting for Outdoor Robot Navigation

1. Problem Statement

Autonomous navigation in outdoor environments (e.g., agriculture, forestry, search and rescue) faces significant challenges due to:

• Complex Terrain: The presence of dense vegetation (tall grass, bushes), irregular obstacles (rocks, trees), and deformable surfaces.
• Traversability Ambiguity: Distinguishing between traversable deformable terrain (e.g., tall grass that a robot can drive through) and rigid obstacles (e.g., trees or dense bushes that must be avoided) is difficult for traditional geometric methods.
• Limitations of Existing Methods:
  • Learning-based approaches often require large, environment-specific datasets and struggle with cross-platform generalization.
  • Geometric-only methods (e.g., LiDAR-based occupancy grids) lack semantic understanding, treating all vegetation as obstacles.
  • Neural Radiance Fields (NeRFs) are too slow for real-time planning and require excessive memory.
  • Current 3D Gaussian Splatting (GSplat) pipelines are typically designed for offline rendering, not online, real-time robot navigation on edge devices.

2. Methodology: Splatblox

Splatblox is a real-time system that fuses RGB images and LiDAR point clouds using 3D Gaussian Splatting to create a Traversability-Aware Euclidean Signed Distance Field (ESDF).

A. Core Architecture

The system operates in a closed loop with three main stages:

1. Perception (Semantic Traversability Estimation):
   • Uses CLIPSeg, a promptable segmentation model, to analyze RGB images.
   • Prompts (e.g., "concrete," "sand," "grass," "rocks") classify terrain into four cost categories: Stable, Granular, Poor Foothold, and High Resistance.
   • These 2D semantic costmaps are projected onto LiDAR points to assign traversability costs to 3D points.
2. Mapping (Gaussian Splatting Representation):
   • The system represents the scene using 3D Gaussian primitives. Each primitive is parameterized by a center, covariance (scale/rotation), opacity, and color (spherical harmonics).
   • Innovation: Instead of optimizing for photorealistic color, the system optimizes for traversability costs. The "color" of the Gaussian encodes the semantic cost derived from the segmentation model.
   • Online Update: New Gaussians are spawned from incoming LiDAR points projected into the camera frame. The system maintains a budget of ~100k active Gaussians to fit within edge GPU memory (VRAM < 6GB).
   • Optimization: Uses a combination of L1 pixel-wise loss and SSIM (Structural Similarity) loss to ensure the rendered costmap aligns with the ground truth, preserving boundaries between traversable and non-traversable areas.
3. Planning (ESDF Fusion):
   • GSplat ESDF: A 3D ESDF is derived from the Gaussian volumetric field, providing fine-grained semantic detail in the robot's frontal view.
   • LiDAR ESDF: A standard LiDAR-based ESDF (computed via Nvblox) provides 360° geometric coverage for areas outside the camera's field of view.
   • Fusion Strategy: The system fuses these two fields. The frontal region (critical for semantic reasoning) uses the GSplat ESDF, while the rest of the 360° field uses the LiDAR ESDF.
   • Output: The fused ESDF is fed into the Nav2 planner (using MPPI controller) for collision checking and trajectory generation.

B. Key Technical Features

• Real-time Performance: Runs at approximately 2 Hz on resource-constrained edge hardware (RTX 3070).
• Memory Efficiency: Caps active Gaussians at 100k, discarding those outside the field of view.
• Semantic Reasoning: Directly encodes semantic costs into the 3D volumetric field, allowing the robot to "know" that grass is drivable while a tree is not.

3. Key Contributions

1. First Real-time GSplat Navigation System: The first system to use Gaussian Splatting for on-the-fly mapping and planning in outdoor, unstructured environments without offline training.
2. Traversability-Aware Volumetric Field: Extends GSplat beyond rendering by embedding LiDAR geometry with semantic traversability costs, enabling the distinction between deformable terrain and rigid obstacles.
3. Hybrid ESDF Fusion: A novel fusion strategy combining semantically rich frontal views (GSplat) with full 360° geometric coverage (LiDAR), enabling long-horizon planning up to 100 meters.
4. Edge-Optimized Pipeline: A lightweight implementation capable of running on mobile robots with <6GB VRAM.

4. Experimental Results

The system was validated on a quadruped robot (Ghost Vision 60) and a wheeled platform (Clearpath Jackal) across diverse outdoor scenarios (bushes, rocks, tall grass, pavement).

Performance Metrics vs. State-of-the-Art (Baselines: DWA, Nvblox, GA-Nav, VERN, MIM)

• Success Rate: >50% higher than the best baseline. In long-range (100m) scenarios, Splatblox achieved 100% success, while all baselines failed in at least one scenario.
• Freezing Rate: 40% fewer incidents where the robot got stuck or oscillated.
• Path Efficiency: ~5% shorter trajectory lengths (Normalized Trajectory Length).
• Time to Goal: Up to 13% faster time to reach the destination.
• Specific Scenarios:
  • In narrow corridors between bushes, Splatblox successfully navigated through, while LiDAR-only methods (Nvblox) treated the vegetation as solid obstacles and failed.
  • In dense vegetation, Splatblox correctly identified traversable paths, whereas learning-based methods (GA-Nav) often assigned high costs to vegetation and avoided them unnecessarily.

Ablation Studies

• Representation Density: Using all Gaussian points (dense representation) outperformed using only Gaussian means, reducing collisions. Performance saturated after 50% of active points were used.
• Grid Depth: A critical threshold of 2 meters was identified for the lookahead depth. Below this, the robot experienced collisions due to the ~2Hz update rate; above 2m, success rates improved significantly.

5. Significance and Impact

• Bridging the Gap: Splatblox successfully bridges the gap between high-fidelity 3D reconstruction (GSplat) and real-time robotic control, proving that implicit neural representations can be used for safety-critical navigation.
• Generalization: Unlike learning-based methods that require retraining for new environments, Splatblox generalizes across diverse terrains (mud, rocks, grass, pavement) using promptable segmentation without offline training.
• Scalability: Demonstrates that complex 3D semantic mapping is feasible on edge devices, paving the way for autonomous robots in agriculture, forestry, and disaster response where long-range, unstructured navigation is essential.

6. Limitations and Future Work

• Update Rate: The system runs at ~2 Hz, which is slower than reactive planners (DWA ~10Hz). This limits performance in highly dynamic environments.
• Lighting Dependence: Reliance on vision-based segmentation (CLIPSeg) makes the system sensitive to challenging lighting conditions.
• Occlusions: Partial observability can lead to incomplete map information.
• Future Directions: Improving runtime (e.g., replacing CLIPSeg with faster models), integrating additional sensors, and extending the framework to dynamic environments with motion prediction.