Technical Summary: SurfFill - Completion of LiDAR Point Clouds via Gaussian Surfel Splatting

1. Problem Statement

LiDAR point clouds are widely regarded as the gold standard for active 3D reconstruction due to their high global consistency and precision. However, they suffer from systematic gaps, particularly at thin structures, silhouettes, and edges. The primary cause identified is beam divergence: as laser beams travel, their cross-section expands, causing a single beam to illuminate both foreground and background surfaces near geometric discontinuities. This results in "mixed pixels" (spurious range measurements) which are subsequently discarded by pre-processing filters as outliers.

Consequently, LiDAR scans often lack fine geometric details. While image-based reconstruction methods (e.g., Multi-view Stereo, Neural Radiance Fields, Gaussian Splatting) can infer these details from feature-rich regions, they often fail in feature-sparse areas and lack the global precision of LiDAR. Furthermore, standard shape completion methods (often deep learning-based) are typically designed for small, object-centric scenes (thousands of points) and struggle to generalize to large-scale, building-sized LiDAR datasets containing millions of points.

The core challenge is to integrate photometric reconstruction with LiDAR data to fill these gaps without introducing outliers or compromising the accuracy of well-scanned regions.

2. Methodology: SurfFill

The authors propose SurfFill, a Gaussian surfel-based LiDAR completion scheme. The pipeline operates through four main stages:

2.1. Ambiguity Heuristic and Uncertainty Mapping

To identify where LiDAR data is missing, the method analyzes the transition regions between well-scanned areas and gaps.

• 3D Ambiguity Heuristic: The authors observe that while gaps have zero density and well-scanned areas have high density, the transition zones (caused by beam divergence and filtering) exhibit a characteristic low but non-zero point density. They compute an ambiguity score $p(x)$ for each point based on the inverse of its local k-nearest neighbor (KNN) density. High ambiguity scores indicate transition regions adjacent to missing structures.
• 2D Uncertainty Maps: To complement the 3D signal, the method estimates monocular surface normals from RGB images. Large gradients in these normals indicate geometric boundaries where LiDAR data is likely incomplete. This generates an uncertainty map $U$ to guide optimization.

2.2. Pre-processing and Initialization

• Non-uniform Downsampling: The input LiDAR point cloud is downsampled non-uniformly. Points with high ambiguity scores (transition regions) are retained at high density to serve as seeds for reconstruction. Well-scanned regions are aggressively thinned out to save computational resources.
• Gaussian Surfel Initialization: A Gaussian surfel model is initialized using the downsampled points. The initial scales of the Gaussians are set based on the local ambiguity score to encourage finer reconstruction in ambiguous areas.

2.3. Focused Gaussian Surfel Optimization

The optimization process is constrained to focus on completing missing areas rather than reconstructing the entire scene from scratch.

• Adaptive Density Control (ADC): The standard ADC module is modified. Instead of mean screen-space gradients, the maximum gradient is used for splitting and cloning to handle sparse RGB data better. A "was-densified" flag tracks newly created Gaussians.
• Training View Sampling: Views are sampled based on the number of projected high-ambiguity points, prioritizing images that contain the regions of interest.
• Loss Functions: The objective function combines standard photometric losses with specific regularizers:
  • Gradient Loss ($L_g$): Encourages edge reconstruction.
  • Scale Regularization ($R_s$): Prevents Gaussians from becoming too large, ensuring fine details are preserved.
  • Masked Depth/Normal Losses: Applied only to ambiguous regions to avoid smoothing out fine structures in well-scanned areas.
  • Uncertainty Masking: The uncertainty map $U$ is used to weight the loss, directing optimization toward uncertain regions.

2.4. Filtering and Sampling

After training, the Gaussian model is filtered to extract valid completion points:

• Outlier Removal: Gaussians with large scales (floaters) or low opacity are discarded.
• Ambiguity Filtering: Gaussians that were not densified or have low ambiguity (indicating they are in well-scanned areas) are removed.
• Distance Filtering: A critical step where Gaussians are compared to the original LiDAR scan. Points closer than a threshold ($t_{min} = 0.01$ m) are removed to prevent duplicating accurate LiDAR data. Points further than $t_{max}$ are removed as outliers.
• Sampling: Points are sampled from the filtered Gaussians, including sampling between neighboring Gaussians to bridge gaps in thin structures.

2.5. Large-Scale Extension

To handle building-sized scenes (tens of millions of points), the authors employ a divide-and-conquer strategy. The scene is subdivided into axis-aligned chunks, each processed independently with a subset of cameras and points. This allows parallel processing on multiple GPUs and circumvents GPU memory limitations.

3. Key Contributions

1. Analysis of LiDAR Artifacts: Identification of beam divergence as the root cause of systematic gaps and the proposal of a density-based ambiguity heuristic to detect transition regions for completion.
2. Focused Gaussian Surfel Splatting: Integration of the ambiguity heuristic into Gaussian surfel optimization, featuring constrained densification, scale regularization, and a filtering pipeline specifically designed for LiDAR completion.
3. Large-Scale Processing: Application of a chunking strategy to enable the completion of building-scale LiDAR datasets (e.g., 73 million points) within feasible timeframes.
4. Performance: Demonstration of a method that outperforms existing reconstruction and shape completion techniques in both synthetic and real-world scenarios.

4. Results

The method was evaluated on synthetic scenes (Attic, Kitchen, Museum) and real-world datasets (ScanNet++, Tanks&Temples, and custom captures).

• Quantitative Performance: SurfFill achieved a mean Chamfer Distance of 0.0055 m and an F1-score of 0.9176, significantly outperforming baselines like 2DGS (0.0618 m Chamfer), 3DGS, Neuralangelo, and COLMAP.
• Precision vs. Recall: While COLMAP achieved comparable recall (finding a similar number of missing points), SurfFill demonstrated superior precision (0.986 vs. 0.789), indicating that the reconstructed points are spatially more accurate and less noisy.
• Qualitative Results: The method successfully reconstructed fine details such as fence bars, chair legs, and hooks that were missing in the original LiDAR scans. It introduced minimal noise to well-scanned flat regions.
• Efficiency: The method processes building-sized scenes in approximately 3 hours on a single GPU (or ~70 minutes with 6 GPUs), compared to days for some NeRF-based methods.
• Ablation Studies: Removing the filtering step reduced the F1-score by ~2.5%, highlighting its importance in preventing noise in accurate regions. Scale regularization and uncertainty maps were crucial for stabilizing reconstruction in textured areas.

5. Significance and Claims

The paper claims that SurfFill effectively bridges the gap between the high precision of LiDAR and the high detail of photometric reconstruction. By leveraging the specific artifact patterns of LiDAR (transition zones) to guide a focused Gaussian surfel optimization, the method recovers missing geometric structures without compromising the integrity of the original scan.

The authors emphasize that their approach is distinct from direct shape completion methods (which fail on large scales) and standard Gaussian Splatting (which lacks the specific constraints for LiDAR gap filling). The method is particularly significant for applications requiring high-fidelity, large-scale 3D models where LiDAR data is incomplete due to physical sensor limitations.

Limitations Acknowledged:

• The method relies on the presence of transition regions; if a structure is completely removed without a trace, it cannot be recovered.
• Performance degrades with significant pose noise or in scenes with sparse view configurations (e.g., the Bridge scene).
• The method does not specifically target dark or reflective surfaces, as photometric methods also struggle there; it correctly leaves these regions unchanged rather than hallucinating geometry.
• Completed points lack the uniform spacing of the original LiDAR preprocessing, potentially appearing noisier visually, though geometrically accurate.