GS^2: Graph-based Spatial Distribution Optimization for Compact 3D Gaussian Splatting

1. Problem Statement

3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis and real-time rendering due to its high fidelity and speed. However, its practical deployment is hindered by excessive memory consumption, often requiring millions of Gaussian points to represent complex scenes.

Existing solutions primarily rely on pruning (removing points) to reduce memory. While effective in reducing point counts, current pruning methods suffer from two critical flaws:

1. Loss of Spatial Consistency: Removing points disrupts the continuity and coherence of the scene, leading to visual artifacts (e.g., holes, blurring, and noise).
2. Suboptimal Densification: Standard 3DGS densification strategies often over-parameterize simple scenes, creating redundant points that are later inefficiently pruned.
3. Lack of Global/Local Constraints: Most pruning methods ignore the global and local geometric relationships between remaining points, failing to optimize their spatial distribution after removal.

2. Methodology

The authors propose GSˆ2, a framework that combines adaptive point management with graph-based spatial optimization. The pipeline consists of three main phases (illustrated in Fig. 3 of the paper):

A. Adaptive Densification and Pruning (ADP)

This module replaces the standard 3DGS density control with a more intelligent approach:

• ELBO-based Adaptive Densification: Instead of using fixed iteration thresholds, the method uses an Evidence Lower Bound (ELBO) strategy. It monitors the trade-off between model complexity (number of points/covariance) and rendering quality (loss). Densification automatically terminates when the marginal gain in quality no longer justifies the increase in complexity. This prevents over-growth in simple scenes.
• Opacity-Aware Progressive Pruning: A dynamic pruning strategy removes Gaussian points with low opacity. To handle artifacts caused by anomalous high-opacity points, the authors introduce a regularization loss ($L_\alpha$) that includes both linear and higher-order terms to suppress blur and enforce faster convergence of low-opacity points.

B. Graph-based Spatial Distribution Optimization (GSDO)

This is the core innovation designed to restore spatial consistency after pruning. Since pruning leaves gaps, the remaining points must shift to cover the scene without breaking geometric continuity.

• Graph Construction: The pruned Gaussian points are treated as nodes in a graph. A k-nearest neighbor (KNN) graph is constructed based on initial embeddings.
• Feature Encoding: A lightweight graph encoder extracts spatial features:
  • Local Features: Computes residuals between a point and its neighbors to capture local geometric offsets.
  • Global Context: Uses max-pooling and averaging to derive a scene-level global feature.
  • Latent Representation: Concatenates local and global features to generate a latent vector $z_i$ for each point.
• Optimization Losses: The spatial distribution is refined using two unsupervised losses:
  1. Global Alignment Loss ($L_{cet}$): Aligns the centroids of the Gaussian points in Euclidean space with their counterparts in the feature space, ensuring global structural consistency.
  2. Local Smoothness Loss ($L_{smt}$): Penalizes discrepancies between neighboring points in both Euclidean and feature spaces, promoting local smoothness and preventing discontinuities.

C. Joint Optimization

During training, the Gaussian attributes (position, scale, rotation, color) are jointly updated alongside the graph-based spatial optimization to minimize the final composite loss.

3. Key Contributions

1. Adaptive Densification & Pruning: Introduced an ELBO-guided strategy that automatically stops densification and an opacity-aware pruning mechanism that removes redundant points while suppressing artifacts.
2. Graph-based Spatial Optimization: Proposed a novel GSDO module that uses a graph encoder and dual-constraint losses (global alignment + local smoothness) to redistribute pruned points, significantly enhancing spatial consistency without altering the original 3DGS representation format.
3. Superior Efficiency-Quality Trade-off: Demonstrated that GSˆ2 achieves rendering quality comparable to or better than full 3DGS using only a fraction of the points.

4. Experimental Results

The method was evaluated on Mip-NeRF 360 and Tanks & Temples datasets, comparing against SOTA baselines like LightGaussian, Mini-Splatting, and EAGLES.

• Point Reduction: GSˆ2 achieves high-quality rendering using only ~12.5% of the Gaussian points required by standard 3DGS (e.g., reducing from ~3M to ~0.3M points in Mip-NeRF 360).
• Rendering Quality:
  • Outperforms standard 3DGS in PSNR (e.g., 27.74 vs. 27.71 on Mip-NeRF 360) despite using significantly fewer points.
  • Surpasses all compared pruning-based baselines in PSNR, SSIM, and LPIPS metrics.
• Visual Quality: Qualitative results show that GSˆ2 maintains coherent spatial structures and avoids the visual artifacts (holes, blurring) common in other pruning methods.
• Efficiency:
  • Rendering Speed: Achieves 513 FPS, comparable to other compact methods and significantly faster than standard 3DGS (135 FPS).
  • Training Time: Slightly increases training time (16 mins vs. 10 mins for ADP-only) due to the graph optimization, but remains efficient.
• Generalization: The GSDO module can be integrated as a post-processing step into other pruning frameworks (e.g., LightGaussian, MaskGaussian), consistently improving their performance.

5. Significance

• Scalability: GSˆ2 addresses the memory bottleneck of 3DGS, making high-fidelity 3D reconstruction feasible for memory-constrained devices (e.g., mobile AR/VR, autonomous driving).
• Theoretical Insight: The paper highlights that simple pruning is insufficient; spatial redistribution guided by feature relationships is crucial for maintaining perceptual fidelity.
• Practical Impact: By achieving superior quality with drastically fewer parameters, GSˆ2 enables the deployment of real-time, high-quality 3D content in scenarios where bandwidth and storage are limited.

Conclusion: GSˆ2 represents a significant step forward in compact 3D Gaussian Splatting, successfully balancing extreme compression with high-fidelity rendering through intelligent spatial distribution optimization.