Heterogeneous LiDAR Early Fusion and Learned Re-Ranking Strategy for Robust Long-Term Place Recognition in Unstructured Environments

Technical Summary: MinkUNeXt-VINE++ for Robust Long-Term Place Recognition

Problem Statement
Robust localization in unstructured environments, particularly agricultural fields like vineyards, presents a critical challenge for autonomous systems. Unlike urban settings, these environments lack distinctive geometric features and suffer from noise, occlusions, and dynamic changes driven by vegetation growth and weather variations. While LiDAR sensors offer lighting-invariant 3D information, existing state-of-the-art (SOTA) place recognition methods often struggle in these conditions. Furthermore, the increasing prevalence of heterogeneous LiDAR data (e.g., combining different sensor types like Livox and Velodyne) in real-world applications has not been effectively addressed by unified representation methods for place recognition in unstructured settings. Additionally, repetitive environments like vineyards often lead to high false-positive rates in initial retrieval stages, necessitating advanced re-ranking strategies to identify true positives.

Methodology
The paper proposes MinkUNeXt-VINE++, a novel framework that integrates two primary components to enhance LiDAR-based place recognition (LPR):

1. Heterogeneous Early Fusion Strategy:
   The method fuses data from two distinct sensors: the Livox Mid-360 (solid-state, non-repetitive scanning, wide field of view, reliable at short ranges) and the Velodyne VLP-16 (mechanical spinning, repetitive scanning, narrower field of view, better at long ranges).

   • Process: Point clouds from both sensors are downsampled to a common resolution and transformed into a shared coordinate frame.
   • Fusion Logic: A distance-based threshold (set at 10 meters) is applied. Points within the short-range area are selected exclusively from the Livox sensor, while points in the long-range area are selected exclusively from the Velodyne sensor.
   • Outcome: This creates a fused point cloud that leverages the high-density detail of the Livox sensor near the robot and the uniform, long-range coverage of the Velodyne sensor, providing a comprehensive environmental representation without complex computational overhead.

2. Learned Re-Ranking Strategy:
   Following the initial retrieval by the main pipeline (based on MinkUNeXt-VINE), a lightweight learned re-ranking head refines the top-$K$ candidates.

   • Architecture: A Multi-Layer Perceptron (MLP) consisting of three linear layers with ReLU activations.
   • Input: The network takes the concatenation of the query descriptor ($q$) and the database candidate descriptor ($d$). While experiments tested including element-wise products and differences, the final configuration uses only $q$ and $d$ to balance performance and computational efficiency.
   • Training: The head is trained using Binary Cross Entropy (BCE) loss during inference time (25 epochs) to distinguish true matches from false positives among the top candidates.

Key Contributions

• First Unified Early Fusion for Unstructured LPR: MinkUNeXt-VINE++ is the first method to specifically address the early fusion of heterogeneous LiDAR data (Livox Mid-360 and Velodyne VLP-16) for place recognition in unstructured environments, creating a unified representation that leverages the complementary strengths of different scanning patterns.
• Efficient Learned Re-Ranking: The introduction of a lightweight re-ranking head that significantly improves the final ranking of candidate places with minimal computational overhead, specifically targeting the challenge of repetitive environments.
• Extensive Evaluation on TEMPO-VINE: The method is evaluated on the TEMPO-VINE dataset, the only available dataset providing heterogeneous LiDAR data in vineyard environments across different phenological stages (early growth, flowering, pre-harvest).
• Cross-Validation: The approach is tested against SOTA urban-tailored LPR methods (PointNetVLAD, MinkLoc3Dv2, MinkUNeXt) and demonstrates that the proposed preprocessing and fusion techniques improve performance even on these general-purpose models.

Results
Evaluated on the TEMPO-VINE dataset, MinkUNeXt-VINE++ demonstrates significant performance gains over single-sensor approaches and existing SOTA methods:

• Overall Performance: The method achieves a Recall@1 of 89.32% and a Recall@1% of 98.14%.
• Improvement over Single Sensors: Compared to single-sensor approaches, the early fusion strategy alone yields a 20% improvement in Recall@1. When including the re-ranking strategy, the total improvement reaches +30% in Recall@1 compared to the best single-sensor baseline (MinkUNeXt-VINE on Velodyne).
• Specific Gains:
  • Compared to single-sensor MinkUNeXt-VINE (Livox), the fused method improves Recall@1 by 64%.
  • Compared to single-sensor MinkUNeXt-VINE (Velodyne), the fused method improves Recall@1 by 48%.
  • The re-ranking component alone adds a 28% improvement in Recall@1 over the early fusion approach without re-ranking.
• Generalization: The method shows robustness in cross-sensor scenarios (training on one sensor, testing on another) and cross-season scenarios (training on one phenological stage, testing on another), though performance naturally degrades when the environmental differences (e.g., early growth vs. flowering) are most extreme.
• Cross-Dataset Validation: The re-ranking strategy was also validated on the BLT dataset (using a single Ouster sensor), showing a 46% improvement in Recall@1, indicating generalization capabilities beyond the specific TEMPO-VINE dataset.

Significance
The paper positions MinkUNeXt-VINE++ as a significant step forward for long-term autonomy in agricultural settings. By effectively fusing heterogeneous sensor data, the method overcomes the limitations of individual sensors in unstructured environments, providing a more robust environmental representation. The learned re-ranking strategy addresses the specific difficulty of distinguishing true locations in repetitive, occluded environments like vineyards. The authors claim that their approach establishes a new state-of-the-art for place recognition in these challenging conditions, achieving performance levels comparable to those typically seen in urban settings, thereby enabling more reliable navigation and mapping for autonomous agricultural systems. The code and data are made publicly available to facilitate reproduction and further research.