3D Densification for Multi-Map Monocular VSLAM in Endoscopy

Technical Summary: 3D Densification for Multi-Map Monocular VSLAM in Endoscopy

Problem Statement
Monocular Visual Simultaneous Localization and Mapping (VSLAM) applied to endoscopy, specifically using multi-map approaches like CudaSIFT-SLAM, has proven effective for robust camera tracking. These systems successfully recover tracking after frequent losses caused by motion blur, temporal occlusion, tool interaction, or water jets. However, the resulting 3D representations are sparse, noisy, and contain a high percentage of inaccurate points and outliers. Consequently, these sparse maps are inadequate for clinical applications requiring dense environment representation, such as assessing lesion sizes, ensuring complete mucosa coverage, or minimizing the risk of missing critical areas. Furthermore, standard feature-based systems struggle in low-texture regions common in endoscopic environments, while direct methods often lack loop closure and multi-mapping capabilities necessary to recover from tracking failures.

Methodology
The authors propose a post-processing densification method that integrates the robust camera tracking of a feature-based SLAM system with the dense depth prediction of a single-view self-supervised network. The system operates as follows:

1. Input Systems: The pipeline utilizes CudaSIFT-SLAM for generating sparse multi-maps (submaps defined by 3D points and keyframes) and LightDepth, a self-supervised network based on illumination decline, for predicting up-to-scale dense depth maps.
2. Scale Alignment and Outlier Filtering: Since monocular SLAM suffers from scale ambiguity and the sparse points are noisy, the method aligns the dense depth predictions with the sparse 3D point clouds using Least Median of Squares (LMedS).
   • For each keyframe, the system computes a scale proposal for every sparse map point by unprojecting the pixel using the dense depth map.
   • LMedS is employed to determine a robust scale and a dynamic distance threshold to distinguish inliers from spurious outliers. This is preferred over RANSAC because the scale is unknown and 3D point errors depend non-linearly on depth.
   • Inlier points are identified, and a final scale is refined using non-linear optimization with a robust influence function.
3. Dense Fusion: Once the dense depth maps are scaled and aligned with the sparse map points, the system fuses the scaled RGB-D observations using a Truncated Signed Distance Function (TSDF).
4. Surface Extraction: The global surface is explicitly extracted from the TSDF using the Marching Cubes algorithm.

Key Contributions

• Dense 3D Surface Reconstruction: The method achieves dense reconstructions in less than 200 ms per keyframe, making it suitable for real-time applications.
• Robust Outlier Removal: By utilizing LMedS, the system effectively filters spurious points inherent in sparse SLAM maps without requiring pre-defined fixed distance thresholds.
• Domain Agnosticism: The approach removes the need for domain adaptation, as the system components are agnostic to specific domain specifications.
• Validation: The method is experimentally validated on the C3VD phantom colon dataset (including short and screening sequences) and qualitatively tested on real colonoscopy recordings from the Endomapper dataset.

Experimental Results

• Accuracy: On the C3VD dataset, the proposed method achieves an RMS accuracy of 4.15 mm and a Median Absolute (MedA) accuracy of 2.60 mm. This represents a significant improvement over the raw CudaSIFT-SLAM output, which had an RMS accuracy of 99.07 mm (indicating a high volume of outliers) despite a MedA of 2.01 mm.
• Density: The densification process increases the number of map points by a factor greater than 60 compared to the original sparse maps.
• Comparison: The method produces maps with accuracy comparable to LightNeus (a state-of-the-art neural rendering approach) but at a fraction of the computational cost and without requiring ground truth camera poses for training or alignment.
• Performance: The total processing time per keyframe is approximately 186 ms (LightDepth inference: 22 ms, LMedS alignment: 138 ms, TSDF integration: 26 ms, Marching Cubes: 117 ms), fitting within the available 200 ms budget.
• Tracking: In screening sequences with frequent tracking failures, the underlying CudaSIFT-SLAM maintained a camera trajectory RMS ATE of 3.09 mm, successfully locating 83.32% of frames.

Significance and Claims
The paper claims that the proposed combination of CudaSIFT-SLAM and LightDepth successfully cleans sparse multi-maps of spurious points and densifies them, enabling robust camera tracking in real endoscopic procedures. The authors emphasize that their approach addresses the inherent scale ambiguity of monocular depth estimation networks by robustly aligning dense predictions to the single scale of the sparse map. They assert that the method achieves competitive accuracy against state-of-the-art techniques while operating within real-time constraints and without the need for additional depth sensing hardware or explicit ground truth depth annotations. Future work is identified as integrating these NN depth estimates directly into the sequential real-time V-SLAM pipeline for initialization, relocation, and bundle adjustment.