GoDe: Gaussians on Demand for Progressive Level of Detail and Scalable Compression

1. Problem Statement

3D Gaussian Splatting (3DGS) has revolutionized real-time novel view synthesis by offering high-fidelity rendering with competitive reconstruction quality. However, as scenes scale to high resolutions, the memory footprint of 3DGS models grows rapidly (often requiring millions of primitives), posing significant challenges for deployment on devices with limited GPU memory, bandwidth, or storage.

While recent compression techniques (pruning, quantization, learned representations) have reduced model sizes, they suffer from a fundamental limitation: they are inherently single-rate.

• Current State: To achieve different compression levels (e.g., 10MB vs. 50MB), existing methods require training or fine-tuning separate, independent models. This results in a collection of fixed operating points rather than a truly scalable representation.
• The Gap: In dynamic deployment scenarios (varying network conditions, heterogeneous devices), maintaining multiple specialized models is impractical. There is a need for a scalable compression approach where a single trained model can support multiple rate-distortion operating points without retraining.

2. Methodology: GoDe (Gaussians on Demand)

GoDe is a framework that reorganizes a single pre-trained 3DGS model into a progressive hierarchy of Gaussians, enabling scalable compression and adaptive Level-of-Detail (LoD) control. The process consists of three main stages:

A. Progressive Hierarchy Construction

Instead of training separate models, GoDe decomposes the full set of Gaussians $\mathcal{G}$ into a base set $\mathcal{G}_0$ and a sequence of disjoint enhancement sets $\mathcal{E}_1, \dots, \mathcal{E}_{L-1}$.

• Gradient-Informed Masking: The hierarchy is built top-down using an iterative procedure. The authors observe that the aggregated gradient sensitivity (the $L_2$ norm of gradients of the reconstruction loss with respect to all parameters of a Gaussian) serves as a stable, model-agnostic metric for importance.
  • Gaussians with high gradient norms contribute significantly to the global structure and are kept in lower levels (coarse).
  • Gaussians with low gradient norms represent fine-grained refinements and are assigned to higher levels.
• Iterative Process: Starting from the full model, the algorithm identifies the least important Gaussians (lowest gradient sensitivity) to form the next enhancement layer, removes them from the current set, and repeats. This avoids the bias of one-shot ranking.
• Geometric Progression: The size of each level is controlled via a geometric progression, allocating more capacity to coarse levels and progressively less to fine levels, mimicking natural rate-distortion behavior.

B. Quantization-Aware Joint Fine-Tuning

Once the hierarchy is constructed, a single joint fine-tuning stage is performed to ensure consistency across all levels under low-precision storage.

• Random Level Sampling: During optimization, a level $l$ is randomly sampled from the hierarchy. The model renders only the Gaussians up to that level ($\mathcal{G}_l$) and updates parameters. This forces shared parameters (especially in lower levels) to remain robust regardless of which higher levels are active.
• Quantization Simulation: The fine-tuning simulates quantization (e.g., 8-bit for appearance, 16-bit for geometry) using straight-through estimators to prepare the model for compressed storage.
• Sparsity: An $L_1$ penalty is applied to appearance coefficients to encourage sparsity and further reduce memory usage.

C. Scalable Compression and Adaptive Rendering

• Compression: Each level (base and enhancements) is compressed independently using Zstandard (ZSTD) with shared dictionaries for geometry and appearance. This creates a scalable bitstream where decoding the base layer yields a coarse image, and appending enhancement layers progressively refines quality.
• Adaptive Rendering: At inference time, the system can dynamically switch between levels based on available bandwidth or compute.
• Pop-in Mitigation: To prevent visual artifacts when switching levels, GoDe employs a runtime opacity interpolation mechanism. New Gaussians from an enhancement layer are initially rendered with zero opacity and smoothly faded in to their target opacity.

3. Key Contributions

1. Scalable Compression from a Single Pipeline: GoDe derives multiple discrete compression levels from a single pre-trained model after one joint fine-tuning stage, eliminating the need for per-rate retraining.
2. Gradient-Induced Hierarchy: Introduces an iterative masking strategy using aggregated gradient norms to organize primitives from coarse structure to fine refinements, providing a stable LoD hierarchy.
3. Model-Agnostic Design: The approach relies on generic optimization signals (gradients) and standard training techniques, making it applicable to various 3DGS variants (explicit, neural, and hierarchical) without architectural changes.
4. Practical Deployment: Combines quantization-aware training, opacity interpolation, and fast entropy coding to enable efficient storage, progressive decoding, and smooth adaptive rendering.

4. Experimental Results

The authors evaluated GoDe on standard benchmarks (Mip-NeRF360, Tanks&Temples, DeepBlending) across three 3DGS backbones: vanilla 3DGS, Scaffold-GS (neural), and Octree-GS (hierarchical).

• Rate-Distortion Performance: GoDe achieves performance comparable to, and often exceeding, state-of-the-art single-rate compression methods (e.g., HAC, Context-GS, RDO-GS).
  • On Mip-NeRF360, GoDe on Scaffold-GS and Octree-GS matched or slightly outperformed specialized methods.
  • On explicit 3DGS, GoDe achieved state-of-the-art curves.
• Efficiency:
  • Decoding: GoDe offers massive decoding speedups (up to 1762× faster at high compression rates) compared to prior methods, making progressive streaming practical.
  • Training: While non-scalable methods require training separate models for each operating point, GoDe produces all 8 operating points in a single training run.
• Visual Quality: Lower LoDs provide extremely compact models (e.g., ~4MB) with high FPS (>800), while higher LoDs recover fine details (reflections, transparency) with minimal PSNR loss compared to full models.
• Ablation Studies:
  • Gradient vs. Other Pruning: Gradient-based importance outperforms opacity-based or positional-gradient pruning.
  • Iterative vs. One-Shot: Iterative masking yields significantly better stability and quality than a single-pass ranking.
  • Hierarchical vs. Independent: While independent models optimized per-point yield slightly better PSNR (0.42–0.96 dB gain), the trade-off is negligible compared to the massive gains in deployability and storage efficiency.

5. Significance

GoDe addresses a critical bottleneck in the deployment of 3D Gaussian Splatting: scalability. By transforming the representation itself to be inherently scalable, it allows a single model to adapt dynamically to varying constraints (network bandwidth, device memory, rendering budget) without the computational overhead of training multiple models.

This work demonstrates that scalable compression and high reconstruction quality are not mutually exclusive. It shifts the paradigm from "compressing a model to a specific target" to "reorganizing a model to expose a continuum of operating points," paving the way for robust, adaptive 3D rendering in real-world, resource-constrained applications.