UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environments

1. Problem Statement

Visual Simultaneous Localization and Mapping (SLAM) is critical for robotics and VR, but traditional methods struggle in dynamic environments.

• Limitations of Existing SLAM: Most assume static scenes. When dynamic objects (people, moving vehicles) are present, they cause severe tracking drift and map corruption.
• Limitations of Recent 3DGS SLAM: While 3D Gaussian Splatting (3DGS) offers high-fidelity rendering, existing 3DGS-based SLAM systems often suffer from:
  • Tight Coupling: Tracking and mapping are often sequential or tightly coupled, hindering real-time performance.
  • Dependency on Priors: Many dynamic handling methods rely on semantic segmentation (e.g., Mask R-CNN) or predefined object classes, limiting their ability to handle "open-set" dynamic objects (unknown moving entities).
  • Threshold Sensitivity: Initialization and pruning of Gaussian primitives often rely on hand-crafted thresholds, which are unreliable in dynamic scenarios, leading to memory bloat or incomplete maps.
  • Artifact Generation: Dynamic objects often leave "ghosting" artifacts in the reconstructed static map.

2. Methodology: UP-SLAM

UP-SLAM is a real-time RGB-D SLAM system designed to decouple tracking and mapping while robustly filtering dynamic elements without semantic labels.

A. Parallel Tracking and Mapping Architecture

The system employs a parallelized framework where tracking and mapping run concurrently:

• Tracking Thread: Performs real-time localization and generates keyframes. It uses a training-free uncertainty estimator to generate motion masks that filter out dynamic keypoints before they corrupt the pose estimation.
• Mapping Thread: Optimizes the 3D scene representation and refines the motion mask based on uncertainty predictions.

B. Adaptively Structured 3DGS Representation

Instead of using fixed thresholds to add or remove Gaussians, UP-SLAM introduces a Probabilistic Octree:

• Probabilistic Anchors: Gaussians are managed via anchors equipped with probabilistic attributes representing the degree of motion.
• Bayesian Update: A Bayesian update rule dynamically adjusts the probability of an anchor being occupied. This allows for automatic pruning of Gaussians associated with transient (dynamic) objects and efficient initialization of new ones, eliminating the need for manual threshold tuning.
• Temporal Encoding: To handle the time-dependent nature of SLAM sequences, a temporal encoder (using sinusoidal positional encoding) is integrated into the Multi-Layer Perceptrons (MLPs) that predict Gaussian attributes (color, opacity, rotation, scale). This enhances rendering quality by modeling inter-frame correlations.

C. Training-Free Uncertainty Prediction & Dynamic Filtering

The core innovation for handling dynamics is a training-free, multi-modal uncertainty estimator:

• Multi-Modal Residuals: The system fuses geometric cues (depth/color residuals) with semantic/visual features extracted from DINOv2.
• Uncertainty Modeling: It models uncertainty as a Gaussian distribution per pixel. The loss function minimizes the negative log-likelihood of the residuals, regularized to prevent trivial solutions.
• Open-Set Capability: By relying on appearance and geometric inconsistencies (via DINO features) rather than semantic labels, the system can detect and mask unknown dynamic objects (open-set dynamic handling).
• Motion Mask Refinement: The predicted uncertainty map is binarized to create a motion mask. This mask is further refined by intersecting with YOLOv8-seg results (for known objects) but relies on the residual-guided strategy for unknown objects, ensuring robust exclusion of dynamic regions.

D. Feature Distillation

To maintain efficiency while leveraging powerful visual features:

• High-dimensional DINO features are distilled into low-dimensional Gaussian attributes via a shallow MLP.
• These low-dimensional features are rendered and then mapped back to high-dimensional space via another MLP to supervise the learning, ensuring the map is rich in semantic information without excessive memory overhead.

3. Key Contributions

1. Uncertainty-Aware Parallel Framework: A novel architecture that decouples tracking and mapping, enabling real-time localization and artifact-free static map reconstruction without relying on predefined semantic annotations.
2. Adaptive Structured 3DGS: Introduction of a probabilistic octree and Bayesian anchor updates for automatic Gaussian primitive allocation and pruning, significantly reducing model size and improving robustness in dynamic scenes.
3. Open-Set Dynamic Handling: A training-free uncertainty estimator that fuses DINO features with geometric residuals to detect and filter dynamic objects, even those unseen during training.
4. Comprehensive Evaluation: Integration with ORB-SLAM3 and extensive testing on multiple datasets, establishing a new protocol for assessing rendering quality in dynamic environments.

4. Experimental Results

The authors evaluated UP-SLAM against 16 state-of-the-art methods (including ORB-SLAM3, DynaSLAM, Photo-SLAM, DG-SLAM, and WildGS-SLAM) on TUM, Bonn, MoCap, and ScanNet datasets.

• Localization Accuracy:
  • Outperformed DG-SLAM by 59.8% on average.
  • Achieved an 84.7% improvement in localization accuracy on the MoCap dataset compared to DG-SLAM.
  • Maintained strong performance in static scenes (10.2% improvement over static-only SLAMs), proving it does not degrade performance when dynamics are absent.
• Rendering Quality:
  • Achieved a 4.57 dB (reported in abstract) / 5.47 dB (reported in text) average improvement in PSNR compared to baselines.
  • Successfully constructed high-fidelity, artifact-free static maps in highly dynamic environments where other methods failed (e.g., producing ghosting or incomplete reconstructions).
• Efficiency & Model Size:
  • Achieved 12 Hz processing speed, meeting real-time requirements.
  • Reduced model size significantly (e.g., 4.9 MB vs. 29.9 MB for SplaTAM) due to the probabilistic pruning mechanism.
  • Achieved a 2x speed-up compared to WildGS-SLAM while maintaining higher reconstruction quality.

5. Significance

UP-SLAM represents a significant advancement in robust SLAM for real-world applications. By moving away from rigid semantic priors and hand-crafted thresholds, it offers a generalizable solution for dynamic environments. Its ability to produce reusable, artifact-free static maps is crucial for downstream robotic tasks like navigation and scene understanding, where dynamic objects (people, moving furniture) must be ignored to build a consistent world model. The decoupling of tracking and mapping further ensures the system is viable for deployment on resource-constrained robotic platforms.