Technical Summary: Spherical Voronoi

Problem Statement

Radiance field methods, particularly 3D Gaussian Splatting (3DGS), have achieved remarkable success in novel view synthesis. However, their modeling of view-dependent appearance often relies on Spherical Harmonics (SH). While SH are computationally efficient and easy to optimize due to their global support and orthogonality, they suffer from fundamental limitations:

1. High-Frequency Failure: SH struggle to represent high-frequency signals, such as sharp specular highlights (glints), without requiring a prohibitively large number of coefficients.
2. Artifacts: Attempting to reconstruct discontinuous or sharp functions with SH leads to Gibbs ringing artifacts.
3. Specular Reflections: SH fundamentally fail to capture specular reflections, which are critical for realistic glossy surfaces.

Alternative explicit representations, such as Spherical Gaussians (SG) and Spherical Betas (SB), offer better localization and can model discontinuities. However, they introduce significant optimization complexity. Their compact support leads to unstable gradients, sensitivity to initialization, and a tendency to converge to local minima, making them difficult to optimize in practice.

Methodology

The authors propose Spherical Voronoi (SV), a new explicit representation for spherical functions inspired by differentiable Voronoi diagrams. SV partitions the directional domain into learnable regions with smooth boundaries, providing a unified framework for appearance modeling.

1. Spherical Voronoi Representation

Unlike global bases (SH) or overlapping kernels (SG/SB), SV models a function $f(\omega)$ on the sphere as a weighted combination of values associated with $K$ directional sites $s_k$:
$$f_{SV}(\omega) = \sum_{k=1}^{K} w_k(\omega; \tau_k) c_k$$
The weights $w_k$ are computed via a softmax function:
$$w_k(\omega; \tau) = \frac{\exp(\tau_k s_k \cdot \omega)}{\sum_{k'} \exp(\tau_{k'} s_{k'} \cdot \omega)}$$

• Parameters: Each site has a direction $s_k$, a value $c_k$, and a temperature parameter $\tau_k$.
• Adaptivity: The temperature $\tau$ controls the sharpness of the partition. Low $\tau$ yields smooth transitions, while high $\tau$ approaches a hard Voronoi tessellation. This allows SV to represent both smooth diffuse lighting and sharp specular lobes.
• Optimization Stability: The softmax formulation ensures well-defined gradients for all sites, avoiding the "competition" and ill-conditioned updates seen in SG and SB when lobes are misaligned.

2. Application to Radiance Fields

The authors apply SV in two distinct settings:

A. View-Direction Parameterization (Diffuse Appearance)
In standard radiance field pipelines, SV replaces the view-direction representation (typically SH). Each Gaussian primitive is equipped with learnable SV sites and values. This provides a more expressive basis for view-dependent color than low-order SH, improving reconstruction quality for diffuse and moderately glossy surfaces without requiring neural decoders.

B. Reflection-Based Parameterization (Specular Appearance)
To address the failure of SH in modeling reflections, the authors extend SV to handle spatially varying reflections using Learnable Light Probes.

• Deferred Rendering Pipeline: Building on 2DGS, the method separates geometry and illumination. A geometry pass renders buffers for position, normals, roughness, and diffuse color.
• Light Probes: A set of spatially distributed probes is learned throughout the scene. Each probe encodes a local reflection field as an explicit SV function, queried in the reflected view direction $\omega_r$.
• Spatial Interpolation: For a surface point, the model queries the $k$-nearest probes and interpolates their SV contributions based on inverse distance.
• Roughness Control: The surface roughness $R$ modulates the SV temperature $\tau$ ($\tau = (1-R)\tau_{max} + R\tau_{min}$). Low roughness (shiny surfaces) results in high $\tau$ (sharp reflections), while high roughness (diffuse surfaces) results in low $\tau$ (broad lobes).
• Far-Field vs. Near-Field: The final specular color is a blend of a learnable far-field cubemap (distant illumination) and the near-field probe interpolation (local inter-reflections).

3. Acceleration

To mitigate the computational cost of evaluating thousands of SV sites per pixel, the authors introduce a cubemap-based partitioning scheme. The sphere is discretized into a low-resolution cubemap, where each texel is pre-assigned a small subset of candidate sites. At runtime, the softmax is computed only over this local candidate set, reducing complexity from $O(K)$ to $O(|S(t)|)$.

Key Contributions

1. Spherical Voronoi Representation: A new explicit representation for spherical functions based on a differentiable soft Voronoi partition. It overcomes the band-limitation of SH and the optimization instability of SG/SB.
2. Directional Radiance Modeling: The application of SV to view-dependent radiance in Gaussian Splatting, demonstrating consistent improvements in reconstruction quality over standard spherical bases across multiple benchmarks.
3. Reflection Modeling: A fully explicit and differentiable formulation for spatially varying reflections using Voronoi Light Probes. This achieves state-of-the-art results on reflective benchmarks without relying on implicit neural decoders.

Experimental Results

The paper evaluates SV on both synthetic and real-world datasets:

• Radiance Modeling (View-Direction): On benchmarks including Mip-NeRF 360, NeRF-Synthetic, DeepBlending, and Tanks&Temples, SV consistently outperforms SH, SG, SB, and even the strong neural baseline Zip-NeRF in terms of PSNR, SSIM, and LPIPS.
• Reflection Modeling: On reflective datasets (Ref-NeRF, GlossySynthetic, Ref-Real), the SV-based method achieves state-of-the-art performance. It surpasses Ref-GS (which uses an MLP decoder for near-field reflections) on Ref-NeRF and GlossySynthetic, and remains competitive on Ref-Real.
• Ablation Studies:
  • SV consistently yields higher reconstruction accuracy than SG, SB, and cubemap-based probes when parameter counts are fixed.
  • The method maintains real-time rendering speeds (0.45× the speed of standard 3DGS), which is faster than other reflection modeling methods like Ref-GS.
  • Training time is competitive (2.43× 3DGS), significantly lower than heavy formulations like GaussianShader.

Significance

The paper argues that carefully designed explicit representations can serve as powerful and efficient alternatives to neural appearance models. By leveraging the adaptive decomposition of the sphere and stable optimization behavior of Spherical Voronoi, the method enables high-quality view-dependent effects and specular reflections while maintaining interpretability and practical rendering performance. The work demonstrates that fundamental changes in how the directional component of radiance fields is modeled are necessary to overcome the plateau in photometric accuracy and the inability to reproduce complex glossy surfaces.