Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

1. Problem Statement

Robust 3D object detection in autonomous driving is severely challenged by adverse weather conditions (rain, snow, fog). While LiDAR provides high-precision geometric structure, its performance degrades significantly in precipitation due to signal scattering. Conversely, 4D Radar is robust to weather but lacks spatial precision.

• Limitation of Existing Methods: Current LiDAR-Radar fusion approaches typically rely on fixed or weakly adaptive pipelines. They compute a single, unified fused representation regardless of the environment. This "one-size-fits-all" strategy fails to dynamically adjust modality preferences when environmental conditions change (e.g., relying too heavily on LiDAR during heavy snow).
• The Gap: There is a lack of explicit mechanisms to select the most reliable representation (LiDAR-only, Radar-only, or Fused) based on real-time weather conditions, leading to suboptimal performance and a lack of interpretability in safety-critical scenarios.

2. Methodology

The authors propose a Weather-Conditioned Branch Routing framework that reformulates multi-modal perception as a dynamic routing problem rather than a static fusion problem.

A. Core Architecture

The framework maintains three parallel feature streams throughout the network:

1. LiDAR-only Branch ($L$): Processes LiDAR point clouds independently.
2. Radar-only Branch ($R$): Processes 4D Radar sparse cubes independently.
3. Fusion Branch ($F$): A condition-gated fusion branch that augments LiDAR features with Radar information using Condition-Gated KNN Attention.

B. Weather-Conditioned Router

Instead of a hard switch, the model uses a soft routing mechanism:

• Condition Token Generation: A lightweight router generates a "condition token" ($\hat{c}$) by fusing:
  • Visual cues: Features from a camera image (processed by a weather-classification CNN).
  • Semantic cues: Text prompts (e.g., "A heavy snow driving scene") encoded via a frozen CLIP text encoder.
  • Sensor context: Pooled features from the initial LiDAR and Radar voxels.
• Adaptive Weighting: The condition token is fed into a lightweight MLP to predict soft weights ($w_L, w_R, w_F$) for the three branches.
• Aggregation: The final Bird's Eye View (BEV) feature map is a weighted sum of the three branches. The design ensures that if $w_L \to 1$, the system relies on LiDAR; if $w_R \to 1$, it relies on Radar, etc.

C. Training Strategy (Preventing Branch Collapse)

A major challenge is that the router might collapse to a single dominant branch (usually the fused one) during training. To prevent this, the authors introduce a Weather-Supervised Learning Strategy:

1. Auxiliary Weather Classification Loss: An auxiliary head forces the condition token to retain discriminative weather information by classifying the scene into 7 weather categories (Normal, Overcast, Fog, Rain, Sleet, Light Snow, Heavy Snow).
2. Weather Diversity Regularization:
   • Intra-class consistency: Encourages similar routing weights for samples within the same weather class.
   • Inter-class separation: Penalizes routing weights that are too similar across different weather classes (Hinge loss).
3. Entropy Regularization: Prevents the router from becoming deterministic too early in training, ensuring all branches remain active.
4. Weight Floor: A minimum weight ($\epsilon=0.1$) is enforced for each branch to guarantee they contribute during early training.

3. Key Contributions

1. Novel Formulation: The first work to explicitly formulate LiDAR-Radar fusion as a weather-conditioned branch routing problem, dynamically selecting between LiDAR-only, Radar-only, and Fused representations.
2. Robust Learning Strategy: Introduction of a weather-supervised routing scheme (auxiliary classification + diversity regularization) that successfully prevents branch collapse and ensures the model learns distinct, condition-dependent preferences.
3. Interpretability: The framework provides transparent, interpretable insights into modality reliance. The routing weights explicitly reveal how the model shifts trust from LiDAR to Radar as weather deteriorates.
4. State-of-the-Art Performance: Achieves new SOTA results on the K-Radar benchmark, particularly in adverse weather scenarios where fixed-fusion methods fail.

4. Experimental Results

Evaluated on the K-Radar dataset (Sedan category), the method outperforms strong baselines including 3D-LRF, InterFusion, and single-modality detectors.

• Overall Performance:
  • IoU=0.5: Achieved 71.5% AP_BEV and 43.5% AP_3D, surpassing the previous best (3D-LRF*) by +7.4 AP_BEV and +13.5 AP_3D.
• Adverse Weather Gains:
  • Heavy Snow: AP_3D improved from 24.5 (3D-LRF*) to 33.7.
  • Fog: AP_3D improved from 32.3 to 63.4.
  • Overcast: AP_3D improved from 40.0 to 63.3.
• Ablation Studies:
  • Removing branch routing (forcing fusion only) caused a massive drop (AP_3D @ 0.5 dropped to 22.0), proving the necessity of adaptive selection.
  • Removing diversity regularization caused a significant drop in adverse weather (e.g., Fog AP_3D dropped to 40.2), confirming the importance of enforcing distinct routing behaviors.
  • The lightweight MLP router outperformed complex Transformer or Cross-Attention routers, offering a better trade-off between performance and efficiency.

5. Significance and Impact

• Safety-Critical Interpretability: Unlike "black-box" fusion models, this approach explicitly reveals why a decision is made (e.g., "LiDAR is suppressed because it is snowing"). This transparency is crucial for safety validation in autonomous driving.
• Dynamic Adaptation: The system mimics human intuition by shifting reliance to the most reliable sensor based on environmental context, rather than relying on a static fusion rule.
• Future Direction: This work suggests a paradigm shift from "feature fusion" to "representation selection," offering a practical and robust alternative for multi-sensor perception in unpredictable real-world environments.