GSM-GS: Geometry-Constrained Single and Multi-view Gaussian Splatting for Surface Reconstruction

1. Problem Statement

While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction and novel view synthesis due to its rapid training speed and high-fidelity rendering, it faces significant challenges in geometric accuracy, particularly for surface reconstruction.

• Unstructured Nature: The unstructured and irregular nature of Gaussian point clouds often leads to inaccuracies in capturing complex surface microstructures.
• Texture Dependency:
  • In texture-rich regions, high-frequency texture components interfere with photometric consistency constraints, causing ambiguity between normal vector orientation and Gaussian anisotropy.
  • In texture-less (smooth) regions, the lack of gradient information leads to local minima in normal estimation, resulting in over-smoothed geometry and distorted surface curvature.
• Multi-view Inconsistency: Discrete Gaussian primitives often lack cross-view geometry-appearance constraints, leading to reconstruction artifacts and geometric distortion in novel views.

Existing methods (e.g., NeRF, standard 3DGS, PGSR) struggle to balance training efficiency with high-fidelity geometric reconstruction, especially in scenes with mixed texture qualities or occlusions.

2. Methodology: GSM-GS Framework

The authors propose GSM-GS, a synergistic optimization framework that integrates single-view adaptive sub-region weighting and multi-view spatial structure refinement. The core logic involves a dual-branch strategy for single views and a geometry-guided sampling strategy for multi-views.

A. Single-View Optimization: Adaptive Sub-Region Constraints

The method dynamically partitions the scene based on image gradient features to apply differentiated constraints:

1. Region Partitioning:
   • Texture-Rich Regions ($R$): Identified using Sobel image gradients.
   • Texture-Less Regions ($B$): Identified where gradients are below a threshold (75th percentile).
2. Confidence Weighting:
   • A weight map $W$ is generated based on the discrepancy between the rendered depth ($D$) and the unbiased depth ($\hat{D}$). Regions with low discrepancy (high confidence) are prioritized.
3. Dual-Branch Constraint Strategy:
   • For Texture-Rich Regions: A dual-branch constraint is applied. It combines a standard normal difference loss with an orthogonality constraint ($L_{cross}$). This constraint enforces the orthogonality between the depth gradient and the normal direction, preventing geometric ambiguity caused by high-frequency textures.
   • For Texture-Less Regions: A Total Variation (TV) regularization term weighted by color similarity is used. This smooths the geometry while preserving physical discontinuities, preventing over-smoothing and noise propagation in uniform areas.

B. Multi-View Optimization: Geometry-Guided Consistency

To enforce consistency across views, the method constructs a 3D continuous geometric correlation model:

1. Weight-Guided Dynamic Sampling:
   • Global confidence weights ($W_{avg}$) are computed by fusing view-specific weights from the current and neighboring views.
   • A binary geometric validity mask filters out near-field noise and occlusions.
   • Adaptive Sampling: High-confidence point clouds are sampled based on the global weights, ensuring constraints are only applied to stable geometric primitives.
2. Cross-View Normal Constraints:
   • Point clouds from the current view and neighboring views are aligned via rigid transformation ($SE(3)$).
   • PCA-based Normal Estimation: Local surface patches (3x3) are extracted, and Principal Component Analysis (PCA) is used to compute surface normals.
   • Curvature Weighting: An exponential decay weight is applied based on local curvature (eigenvalues of PCA). Flatter, more stable regions receive higher weights.
   • Loss Function: A weighted cosine similarity loss ($L_{mvgeo}$) minimizes the angle difference between normals of corresponding points in adjacent frames, enforcing geometric consistency.

C. Total Loss Function

The final training objective combines photometric loss with the proposed geometric constraints:
$$L = L_{rgb} + L_{svgeo} + \lambda_3 L_{mvgeo}$$
Where $L_{svgeo}$ is the single-view dual-branch loss and $L_{mvgeo}$ is the multi-view consistency loss.

3. Key Contributions

1. Single-View Adaptive Sub-Region Constraint: A novel dual-branch optimization strategy that decouples texture-rich and texture-less regions. It uses image gradients for partitioning and depth discrepancies for confidence weighting, applying orthogonality constraints for detailed areas and TV regularization for smooth areas.
2. Geometry-Guided Multi-View Consistency: A weight-guided dynamic sampling strategy combined with cross-view geometric normal correlation. This constructs a 3D continuous geometric model, effectively addressing multi-view consistency loss in complex scenes by filtering unstable primitives.
3. Comprehensive Evaluation: Extensive experiments on benchmark datasets (DTU, Mip-NeRF360, Tanks and Temples) demonstrating superior performance in both surface reconstruction and novel view synthesis compared to state-of-the-art methods (3DGS, PGSR, 2DGS, etc.).

4. Experimental Results

The method was evaluated on three major datasets:

• DTU Dataset (Geometric Accuracy):

  • Achieved a mean Chamfer Distance (CD) of 0.51 mm, outperforming PGSR (0.53 mm) and significantly beating other baselines like 2DGS (0.76 mm) and GOF (0.81 mm).
  • Achieved the best results in 13 out of 15 scenes.
  • Training time increased slightly (0.45h vs. 0.28h for PGSR) but yielded significantly higher geometric fidelity.

• Tanks and Temples Dataset (F1-Score):

  • Achieved a mean F1-Score of 0.36, representing a 16.4%–63.6% improvement over 2DGS, GOF, and RaDe-GS, and a 5.9% improvement over PGSR.
  • Demonstrated robustness in large-scale, unconstrained environments with dynamic illumination.

• Mip-NeRF360 (Novel View Synthesis):

  • Achieved competitive results in PSNR, SSIM, and LPIPS.
  • Notably achieved the best LPIPS (0.175) in the average of all scenes, indicating superior perceptual quality.
  • Qualitative results showed better preservation of fine details (e.g., floor lines, tree structures) and reduced artifacts (e.g., floor unevenness) compared to baselines.

• Ablation Studies:

  • Confirmed that both the single-view ($L_{svgeo}$) and multi-view ($L_{mvgeo}$) constraints independently improve performance.
  • The combination of both yielded the best results, proving the synergistic effect of the framework.

5. Significance

GSM-GS represents a significant advancement in 3D Gaussian Splatting by addressing the critical gap between rendering speed and geometric accuracy.

• Robustness: It specifically targets the weaknesses of current methods in texture-less regions and complex microstructures, which are common failure points in standard 3DGS.
• Efficiency vs. Accuracy: It achieves high-fidelity reconstruction with a minimal computational overhead increase, making it viable for practical applications in robotics, AR/VR, and autonomous driving where both speed and geometric precision are required.
• Generalization: The adaptive nature of the constraints allows the method to generalize well across diverse scenes, from indoor environments to large-scale outdoor terrains.

The paper concludes that while challenges remain with transparent materials and highly reflective surfaces, GSM-GS sets a new standard for geometry-constrained Gaussian Splatting, effectively suppressing geometric artifacts and enhancing surface reconstruction fidelity.