Technical Summary: LiteViLNet

Problem Statement

Road segmentation is a critical perception task for autonomous driving and intelligent robotics, serving as the foundation for trajectory planning and obstacle avoidance. While multi-modal approaches combining RGB images and LiDAR data have achieved state-of-the-art accuracy by leveraging complementary texture and geometric information, they often rely on heavy transformer-based encoders (e.g., Swin Transformer) with hundreds of millions of parameters. These models incur enormous computational costs, making them unsuitable for real-time deployment on resource-constrained edge devices such as embedded platforms in vehicles and robots. Conversely, existing lightweight methods often sacrifice too much accuracy or fail to effectively model complex cross-modal interactions, leaving a gap for an efficient multi-modal network that balances high performance with real-time inference requirements.

Methodology: LiteViLNet

The authors propose LiteViLNet, a lightweight Vision-LiDAR fusion network designed to achieve efficient road segmentation through a carefully architected pipeline that minimizes parameters while maximizing representational capacity.

1. Input Representation

The network accepts an RGB image and a 3D LiDAR point cloud. To facilitate efficient processing, the LiDAR point cloud is pre-processed into an Altitude Difference Image (ADI). This process involves projecting 3D points onto the 2D image plane and calculating weighted height differences within local neighborhoods. The resulting ADI encodes geometric height differences between the ground and obstacles, which is then replicated across three channels to match the RGB input interface.

2. Dual-Stream Lightweight Encoder

The architecture employs a dual-stream encoder to extract features separately:

• RGB Stream: Utilizes a pre-trained MobileNetV3-Large backbone to extract texture features, generating a feature pyramid at five scales.
• LiDAR Stream: Uses a custom tiny encoder constructed entirely from depth-wise separable convolutions (DSConv). This stream mirrors the structure and channel configuration of the RGB stream to ensure feature alignment but introduces negligible overhead (approx. 0.12M parameters).

3. Multi-Scale Feature Fusion Module (MSFM)

To address the challenge of effective cross-modal interaction without heavy computation, the authors propose the MSFM, applied at each of the five feature scales. The module performs:

• Dimension Reduction: Compressing channel dimensions to reduce attention calculation costs.
• Intra-modal Enhancement: Applying Efficient Channel Attention (ECA) to RGB features and Coordinate Attention (CA) to LiDAR features to capture channel dependencies and spatial positions, respectively.
• Cross-modal Attention (CMA): A bidirectional attention mechanism allowing each modality to attend to informative regions of the other.
• Gated Fusion: A learnable gate mechanism dynamically fuses the cross-modal interaction features with the original enhanced features, allowing the network to adaptively adjust the fusion ratio based on input content.

4. Large-Kernel-Bridge Semantic Enhancement

To capture global context without the quadratic complexity of self-attention, a large-kernel-bridge module is inserted after the fusion of the deepest features. This module uses a 7×7 depth-wise separable convolution within a bottleneck structure to enlarge the receptive field with linear complexity. This design replaces the heavy transformer bridges used in other methods, significantly reducing parameters (0.26M vs. ~24M in standard transformers) while maintaining semantic representation capabilities.

5. Decoder and Deep Supervision

A U-Net style decoder with skip connections recovers spatial resolution. The network employs deep supervision, adding auxiliary segmentation heads to intermediate decoder layers to provide additional gradient signals during training. These heads are removed during inference, ensuring no runtime overhead. The loss function combines Binary Cross-Entropy, Lovász-Hinge loss, and Focal loss to optimize segmentation quality and handle class imbalance.

Key Contributions

1. LiteViLNet Architecture: A lightweight multi-modal network specifically designed for efficient road segmentation, achieving a superior balance between accuracy and inference efficiency for edge deployment.
2. Synergistic Module Design: The proposal of a suite of efficient modules, including the dual-stream encoder, the Multi-Scale Feature Fusion Module (MSFM) for effective cross-modal interaction, and the Large-Kernel-Bridge for semantic enhancement, all operating with minimal computational and parameter overhead.
3. ADI Generation Scheme: A compatible ADI generation pipeline that allows the model to utilize RGB-D camera depth data as a substitute for LiDAR, reducing deployment costs and enhancing practical applicability.
4. Comprehensive Validation: Extensive experiments on the KITTI Road dataset and real-world deployments on diverse robotic platforms (Kuafu Delivery Vehicle, Unitree-B2, Unitree-G1) demonstrating the model's robustness and efficiency.

Experimental Results

The authors evaluated LiteViLNet on the KITTI Road dataset and compared it against state-of-the-art methods:

• Accuracy: LiteViLNet achieved a 96.36% MaxF score. This ranks it as the best among all CNN-based methods and makes it comparable to larger transformer-based models (e.g., SNE-RoadSegV2, RoadFormer) which have significantly higher parameter counts.
• Efficiency: The model contains only 14.04M parameters.
  • On an RTX 4060 Ti, it achieves 163.79 FPS (model-only inference).
  • On an NVIDIA Jetson Orin NX (embedded edge device), it achieves 22.18 FPS (model-only), which increases to 68.73 FPS with TensorRT FP16 deployment.
• Comparison: The model outperforms numerous heavy-weight methods in inference speed (e.g., 1.57x faster than USNet and >9.5x faster than PLARD on RTX 4060 Ti) while maintaining highly competitive accuracy.
• Real-World Deployment: The system was successfully deployed on three robotic platforms using an Orbbec Gemini 336L RGB-D camera. The robots demonstrated stable lane-centering navigation at 15 FPS, confirming the system's ability to handle real-world lighting, terrain, and occlusion conditions.

Significance

The paper claims that LiteViLNet resolves the fundamental conflict between high-accuracy perception and real-time computational constraints in autonomous systems. By demonstrating that a lightweight architecture can rival the performance of heavy transformer-based models while running efficiently on embedded hardware, the work validates the potential of such networks for real-time deployment in autonomous driving and intelligent robotics. The successful integration of geometric LiDAR (or depth) data with RGB texture in a low-parameter framework offers a practical solution for safety-critical applications where both precision and low latency are paramount.