AdaGScale: Viewpoint-Adaptive Gaussian Scaling in 3D Gaussian Splatting to Reduce Gaussian-Tile Pairs

1. Problem Statement

3D Gaussian Splatting (3D-GS) has emerged as a state-of-the-art technique for novel view synthesis, offering faster rendering speeds than Neural Radiance Fields (NeRF) by using explicit 3D Gaussian primitives. However, achieving real-time performance (90–120 FPS) for high-resolution applications like AR/VR remains challenging.

• The Bottleneck: The rendering pipeline involves four stages: preprocessing, Gaussian-tile pair generation, sorting, and rasterization. Profiling reveals that the latter three stages consume ~88% of the runtime, and their computational cost is directly proportional to the number of Gaussian-tile pairs generated.
• Limitations of Existing Methods: Previous optimization techniques (e.g., GSCore, AdR-Gaussian, FlashGS) focus on geometrically accurate bounding boxes (AABB, OBB, Ellipse) to eliminate pairs with zero contribution. However, they fail to account for the varying importance of pairs that do contribute. Specifically, tiles intersecting the peripheral regions of a Gaussian (far from the center) contribute negligibly to the final pixel color but still incur the full computational cost of intersection testing and sorting.

2. Methodology: AdaGScale

The authors propose AdaGScale, a viewpoint-adaptive technique that reduces the number of Gaussian-tile pairs by dynamically scaling the effective size of Gaussians based on their peripheral color contribution.

Core Concept: Peripheral Score

The method relies on the observation that color contribution ($Contribution(P_k, G_i)$) drops sharply as the distance from the Gaussian center increases.

• Peripheral Score ($PS$): Defined as the sum of color contributions in the peripheral region of a Gaussian (where the alpha value $\alpha$ is between a minimum threshold $\tau$ and an intermediate threshold $x$).
• Goal: Identify a threshold $Th_i$ for each Gaussian such that ignoring contributions below $Th_i$ results in a controlled, acceptable loss of image quality (defined by a user parameter $K$).

Three-Stage Approximation Strategy

Calculating the exact peripheral score during the preprocessing stage is infeasible because it requires sorting-dependent transmittance values ($T_i$). AdaGScale solves this with an efficient approximation:

1. Transmittance Approximation: Instead of calculating pixel-specific transmittance $T_i$ (which requires depth sorting), the method approximates it using a single constant value per Gaussian: the maximum transmittance across all pixels ($max(T_i)$). Experiments show this introduces negligible PSNR degradation.
2. Depth-Based Upper Bound: Since $max(T_i)$ correlates with depth (farther Gaussians are more likely to be occluded), the authors derive a piecewise constant upper bound, $T^{Upper}_i(depth)$, using a calibration dataset. This allows estimation of transmittance using only the precomputed depth value.
3. Closed-Form Computation (Riemann Sum): The summation of discrete alpha values over the peripheral region is approximated using a Riemann sum integral. This yields a closed-form formula for the threshold $Th_i$:
   $$Th_i = \frac{K}{T^{Upper}_i(depth) \cdot 2\pi\sqrt{\det(G_{cov_i})}} + \tau$$
   This formula depends only on parameters available during preprocessing: depth, projected 2D covariance, and the target loss $K$.

The Dual-Size Strategy

A critical innovation of AdaGScale is the separation of the intersection test from the color accumulation:

1. Pair Generation (Reduced Size): The calculated $Th_i$ is used as the alpha threshold for the tile intersection test. If $Th_i > \tau$ (1/255), the Gaussian's effective coverage area shrinks, skipping peripheral tiles and generating fewer Gaussian-tile pairs.
2. Rasterization (Original Size): During the actual color accumulation (rasterization), the original alpha threshold ($\tau = 1/255$) is retained. This ensures that any pair that does pass the intersection test contributes its full, unaltered color value, preserving visual fidelity.

3. Key Contributions

1. Peripheral Score Definition & Approximation: The paper introduces a metric to quantify the importance of peripheral Gaussian regions and proposes an efficient, closed-form approximation method that can be computed during the preprocessing stage without retraining.
2. Viewpoint-Adaptive Scaling: AdaGScale dynamically adjusts the effective size of each Gaussian based on the current viewpoint, depth, and covariance, allowing for aggressive pair reduction where it matters least.
3. Dual-Size Pipeline: The technique decouples the intersection test size from the rendering size, enabling significant speedups while maintaining the original rendering accuracy for surviving pairs.
4. Seamless Integration: The method requires no retraining of the 3D Gaussian models and integrates directly into existing 3D-GS pipelines (including compressed and MLP-based variants).

4. Experimental Results

The authors evaluated AdaGScale on diverse datasets (Tanks&Temples, DeepBlending, Mip-NeRF360, Mill-19, UrbanScene3D) and various Gaussian models (Original 3D-GS, GaussianSpa, CityGaussian, Scaffold-GS) using an NVIDIA A6000 GPU.

• Speedup Performance:
  • Original 3D-GS: Achieved a geometric mean speedup of 5.46× with only a 0.5 dB PSNR drop.
  • City-Scale Scenes: On high-resolution city scenes (CityGaussian), AdaGScale achieved a massive 13.8× speedup over the original 3D-GS with a 0.5 dB PSNR drop.
  • Comparison: It significantly outperformed FlashGS (a state-of-the-art optimized renderer), achieving 1.84× to 1.98× speedups over FlashGS at similar quality levels.
  • MLP-Based Models: Even on Scaffold-GS (which uses MLPs for generation), it achieved a 1.65× speedup in the rendering stage.
• Pair Reduction: AdaGScale reduced the number of Gaussian-tile pairs by up to 43% compared to FlashGS.
• Quality: The actual PSNR degradation on test sets consistently stayed below the target threshold (e.g., targeting 0.5 dB drop resulted in ~0.26 dB actual drop for Original 3D-GS).

5. Significance

AdaGScale addresses a fundamental inefficiency in 3D Gaussian Splatting: the uniform treatment of all Gaussian-tile pairs regardless of their visual importance. By leveraging the observation that peripheral contributions are negligible, it enables real-time rendering of complex, high-resolution scenes on consumer and server-grade GPUs.

Its significance lies in:

• Enabling AR/VR: The achieved speedups (up to 13.8×) bring 3D-GS closer to the 90–120 FPS requirements for high-fidelity AR/VR headsets.
• Generalizability: It works across different scene scales (indoor to city-wide) and different model architectures (pruned, MLP-based, standard).
• Efficiency: It achieves these gains without the computational overhead of retraining or complex post-processing, making it a practical drop-in solution for existing 3D-GS pipelines.