Technical Summary: LIT-GS

Problem Statement

Existing LiDAR–Inertial–Visual (LIV) Gaussian Splatting pipelines for 3D mapping rely heavily on RGB photometric cues for initialization and optimization. This dependency renders them fragile in scenarios characterized by significant illumination changes (e.g., extreme sunlight, night) or texture-deficient environments, where photometric correspondence quality degrades, leading to pose estimation errors and structural drift. While thermal imaging offers illumination-invariant supervision, existing thermal 3D Gaussian Splatting methods primarily focus on novel-view synthesis and often lack a tightly coupled pose/structure refinement loop or explicit geometric constraints, resulting in metric-inconsistent maps and surface thickening artifacts.

Methodology

The paper proposes LIT-GS, a LiDAR–Inertial–Thermal Gaussian Splatting framework designed to enforce LiDAR-derived geometry as an explicit, persistent constraint throughout both pose/structure refinement and Gaussian optimization. The system operates on synchronized LiDAR, IMU, and thermal data, utilizing a unified geometry-aware optimization framework.

1. System Architecture and Preprocessing

• Hardware: The system integrates a LiDAR (with built-in IMU), an RGB camera, and a long-wave thermal camera, synchronized via PPS triggers.
• Thermal Preprocessing: Long-wave thermal images undergo CLAHE (Contrast Limited Adaptive Histogram Equalization) to enhance gradients. Duplicate frames caused by sensor non-uniformity correction (NUC) are skipped.
• Feature Matching: SuperPoint and SuperGlue are employed for keypoint detection and context-aware matching. Although trained on RGB, these models are adapted for thermal imagery by focusing on structural gradients rather than color.

2. Confidence-Aware Cross-Modal Anchoring

LIT-GS distinguishes between two types of visual points:

• Anchors: Uncertainty-tagged visual map points derived from an upstream FAST-LIVO2 LiDAR-inertial-visual estimator. These serve as high-confidence geometric anchors due to their tight coupling with LiDAR geometry.
• Non-Anchors: Additional points generated by SuperPoint+SuperGlue matches to improve spatial coverage.
  The system assigns higher weights to anchors and uses a motion-adaptive weighting scheme to balance the contribution of anchors versus non-anchors based on the platform's linear and angular speeds.

3. LiDAR-Plane-Constrained Refinement

Before Gaussian optimization, a LiDAR-plane-constrained Bundle Adjustment (BA) is performed to jointly refine camera poses and triangulated 3D points.

• Objective: The BA minimizes a unified cost function combining thermal reprojection errors and weighted LiDAR point-to-plane residuals.
• Mechanism: The geometric term penalizes the distance between 3D points and local LiDAR-derived planes, effectively reducing scale drift and structural misalignment under weak thermal supervision.

4. Geometry-Regularized Gaussian Optimization

The core rendering optimization introduces a novel objective function that couples thermal photometric fidelity with geometric consistency.

• Representation: 3D Gaussians are initialized from the refined structure, utilizing covariance estimates from the anchor points to define anisotropic primitives.
• Radiance Modeling: View-dependent thermal radiance is modeled using third-order spherical harmonics (SH).
• Loss Function: The total loss $L$ is a weighted sum of photometric loss ($Loss_{pho}$) and a geometric loss ($Loss_{geo}$):
  $$L = (1 - \lambda)Loss_{pho} + \lambda Loss_{geo}$$
  The geometric loss specifically penalizes the point-to-plane distance between rendered 3D points (back-projected from rendered depth) and the associated LiDAR surface normals.
• Adaptive Control: A pruning and densification strategy removes Gaussians that deviate significantly from local LiDAR planes (geometric outliers) or exhibit low opacity, while splitting primitives in high-residual regions.

Key Contributions

1. LIT-GS Framework: A novel LiDAR-inertial-thermal Gaussian Splatting system that injects LiDAR plane constraints into both the pose/structure refinement stage and the Gaussian optimization stage, improving metric consistency in challenging lighting and texture conditions.
2. Cross-Modal Anchoring & Refinement: A confidence-aware scheme that utilizes uncertainty-annotated LIV map points as geometric anchors. This approach fuses learned thermal correspondences with motion-adaptive weighting and LiDAR point-to-plane residuals within a bundle adjustment framework to stabilize thermal-LiDAR alignment.
3. LiDAR-Plane-Regularized Splatting: A differentiable splatting objective that enforces local LiDAR plane consistency during training. This suppresses surface thickening and structural drift, which are common issues in low-contrast thermal imagery, while maintaining high rendering fidelity.

Experimental Results

The authors evaluated LIT-GS on five proprietary sequences and five public sequences from the M2DGR benchmark, comparing against LIV-GaussMap (RGB-LiDAR) and Thermal3D-GS (Thermal-only).

• Qualitative Performance:

  • LIV-GaussMap: Suffered from structural loss and blurring in low-light or high-glare conditions due to unstable RGB photometric cues.
  • Thermal3D-GS: Maintained structure under illumination changes but exhibited blurred edges, distorted surfaces, and "surface thickening" in thermally homogeneous regions due to a lack of explicit geometric regularization.
  • LIT-GS: Consistently produced sharper boundaries, clearer structural details, and geometrically consistent planar regions across all lighting conditions (day, night, strong sunlight).

• Quantitative Performance:

  • Geometric Accuracy: LIT-GS achieved the lowest Earth Mover's Distance (EMD) across most datasets, indicating superior geometric consistency compared to baselines.
  • Rendering Quality: LIT-GS achieved higher SSIM and lower LPIPS scores, reflecting better perceptual quality and structural fidelity.
  • Ablation Studies:
    • Removing the data refinement module significantly degraded PSNR, SSIM, and EMD, confirming the necessity of the LiDAR-plane-constrained BA.
    • Using learned features (SuperPoint+SuperGlue) with CLAHE preprocessing outperformed hand-crafted features (ORB, SIFT) in terms of inlier ratio and pose accuracy (ATE).

Significance and Claims

The paper claims that LIT-GS bridges the gap between robust thermal supervision and metric geometric consistency. By treating LiDAR geometry not just as an initialization tool but as a persistent constraint during Gaussian learning, the method overcomes the brittleness of RGB-driven mapping in extreme lighting and the structural ambiguity of pure thermal mapping. The authors position LIT-GS as a lightweight, geometry-aware solution tailored for robotics applications requiring reliable 3D reconstruction in dynamic and visually challenging environments. The work suggests that while RGB-LiDAR mapping excels in good lighting, a LiDAR-regularized Gaussian framework integrating thermal cues is essential for robustness across the full spectrum of lighting conditions.