💻 computer science

Eulerian Gaussian Splatting using Hashed Probability Pyramids

This paper introduces Eulerian Gaussian Splatting, a probabilistic radiance field framework that replaces heuristic Gaussian manipulation with gradient-based optimization of a learnable volumetric probability density using hashed probability pyramids, thereby achieving state-of-the-art reconstruction quality on mip-NeRF 360 while maintaining 3DGS-level rendering speed.

Original authors: Mia Gaia Polansky, George Kopanas, Stephan Garbin, Todd Zickler, Dor Verbin

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mia Gaia Polansky, George Kopanas, Stephan Garbin, Todd Zickler, Dor Verbin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to recreate a 3D scene (like a room or a garden) using millions of tiny, glowing, fuzzy balls (Gaussians). This is what a popular technology called 3D Gaussian Splatting does. It's incredibly fast and looks great, but it has a flaw: it manages these balls using a set of rigid, hand-written rules. If a ball is in the wrong spot, the computer has to guess whether to move it, split it into two, or delete it. Sometimes these rules get confused, leading to a messy reconstruction.

This paper introduces a new method called Eulerian Gaussian Splatting (EGS). Instead of manually moving the balls around, the authors treat the entire 3D space like a weather map.

Here is the breakdown of their approach using simple analogies:

1. The Weather Map vs. The Moving Trucks

The Old Way (Lagrangian): Imagine you have a fleet of delivery trucks (the balls) driving around a city. If a package needs to go to a new location, you have to tell a specific truck to drive there, or tell a truck to split in half. This requires a complex traffic controller (heuristics) to decide which truck does what.
The New Way (Eulerian): Instead of moving trucks, imagine you have a weather map that shows where it is "likely to rain." You don't move the rain; you just change the map. If the map says "heavy rain" in a specific spot, you automatically generate a raindrop there. If it says "dry," no raindrop appears.
- In this paper, the "weather map" is a learnable probability density. The computer doesn't move the balls; it learns a map that tells it where to spawn new balls and where to stop spawning them, purely based on how well the picture looks.

2. The "Hashed Probability Pyramid" (The Smart Map)

Creating a weather map for a whole city at a microscopic level would require too much computer memory. To solve this, the authors built a Hashed Probability Pyramid.

The Analogy: Think of a map that starts as a low-resolution grid (like a pixelated image). As you zoom in, the map gets more detailed. However, instead of storing every single pixel's data, the map uses a clever "hash" trick (like a secret code) to reuse information for empty areas.
Why it helps: It allows the computer to have a super-detailed map of the scene without running out of memory. It can focus its "brain power" on the complex parts of the scene (like a tree with many leaves) while keeping the empty sky simple.

3. The "Control Variate" (The Noise Canceller)

When the computer tries to learn this map, it has to guess where to put the balls, look at the result, and then adjust the map. Because it's guessing randomly, the "feedback" (gradients) it gets is very noisy, like trying to hear a whisper in a loud concert. This usually makes learning slow and unstable.

The Innovation: The authors invented a special math trick called a Control Variate.
The Analogy: Imagine you are trying to judge how much a single singer contributes to a choir's sound. If you listen to the whole choir, it's hard to tell. But if you imagine the choir without that one singer, you can easily hear the difference.
The authors' math trick calculates exactly how much each individual ball contributed to the final image by comparing the full image to the image without that specific ball. This cancels out the background noise, giving the computer a very clear, stable signal on how to improve the map.

4. The Result: A Self-Organizing Garden

By combining these ideas, the system works like a garden that organizes itself:

It starts with a blank canvas and a "probability map."
It randomly sprouts "seeds" (Gaussian balls) based on the map.
It looks at the picture. If a part of the garden looks blurry or wrong, the map automatically increases the "probability" of seeds growing there. If a part looks too crowded or wrong, the map lowers the probability.
No human rules are needed to tell the computer to "split this ball" or "delete that ball." The math naturally moves the "mass" of the probability to where it is needed.

The Bottom Line

The paper claims that this method achieves state-of-the-art quality (it looks better than previous methods) on complex 3D scenes, while keeping the rendering speed just as fast as the original 3D Gaussian Splatting.

Crucially, it does this without needing a pre-scan of the scene (like a 3D laser scan) to get started. It can start from a completely random guess and figure out the shape of the world just by looking at the photos, thanks to its ability to learn the "probability map" directly through gradient descent. It bridges the gap between the stability of continuous math (like NeRFs) and the speed of discrete objects (like 3DGS).

Technical Summary: Eulerian Gaussian Splatting using Hashed Probability Pyramids

Problem Statement
Current 3D Gaussian Splatting (3DGS) methods achieve real-time rendering by representing scenes with discrete Gaussian primitives. However, optimizing these scenes relies on "Adaptive Density Control" (ADC), a set of hand-tuned heuristics (e.g., splitting, cloning, culling) to add or remove primitives during training. These heuristics can be brittle and may lead to suboptimal local minima. Conversely, Neural Radiance Fields (NeRFs) optimize continuous volumetric densities via gradient descent, offering greater stability but suffering from slow rendering speeds. The paper addresses the need for a framework that retains the rendering efficiency of 3DGS while adopting the principled, gradient-based optimization of continuous fields, eliminating the need for heuristic primitive manipulation.

Methodology
The authors propose Eulerian Gaussian Splatting (EGS), a probabilistic framework that shifts from a Lagrangian view (explicitly moving primitives) to an Eulerian view (optimizing a probability density function over the scene volume).

Probabilistic Rendering Model:
Instead of optimizing individual Gaussian positions, EGS defines a volumetric probability density function $p_\theta(\mu)$ over the 3D space. At each training iteration, a discrete set of $N$ Gaussian centers $\mu_i$ is sampled from this distribution. These sampled centers are then rendered using standard 3DGS rasterization. The loss is minimized by optimizing the parameters of the underlying probability distribution $\theta$ and the Gaussian attributes $\phi$ (color, scale, rotation, opacity).
Hashed Probability Pyramid:
To represent the probability density $p_\theta(\mu)$ efficiently at high resolutions without infeasible memory growth, the authors introduce a Hashed Probability Pyramid.
- Structure: It is a multi-scale hierarchical grid where the probability density is parameterized as a product of piecewise-constant functions across $L$ levels.
- Memory Efficiency: To avoid the cubic growth of parameters, the method uses hash encoding. At each level, the grid is tiled with a limited budget ( $B$ ) of $2\times2\times2$ blocks. A hash function maps grid indices to these blocks, allowing the representation of a high-resolution distribution (e.g., $4096^3$ ) using a compact set of parameters (approx. 86 million parameters for a $4096^3$ grid, requiring only 0.33 GB).
- Sampling: Sampling is performed efficiently by traversing the hierarchy from coarse to fine levels. The process is made end-to-end differentiable by tying random samples across levels via deterministic fractional part functions.
Gradient Estimation and Variance Reduction:
Optimizing a distribution via stochastic gradient descent introduces high variance. The standard pathwise estimator (using automatic differentiation through the sampling process) is found to be unstable.
- Control Variates: The authors derive an unbiased gradient estimator based on the score function, enhanced with control variates.
- Key Insight: The gradient contribution of a specific sample is weighted by its individual impact on the rendered image. Specifically, the difference between the full image $I$ and the image rendered without the $i$ -th Gaussian ( $I_{-i}$ ) is used.
- Efficiency: Crucially, the paper proves that $(I - I_{-i})$ can be computed directly from the opacity $o_i$ and the gradient of the loss with respect to opacity ( $\partial I / \partial o_i$ ) during the standard backward pass. This avoids the computational cost of rendering $M$ separate images while significantly reducing gradient variance.
Training Strategies:
- Defensive Sampling: Gaussian noise is added to a subset of samples early in training to prevent the probability density from collapsing to zero in regions that might later become important.
- Sample Rounding: Sample positions are rounded to the center of their finest grid bin to reduce variance caused by slight positional shifts.
- Duplicate Removal: Duplicate samples (arising from rounding) are removed before rendering, naturally pruning the number of Gaussians as the model optimizes.
- Refinement: After optimizing the probability and attribute grids, a final refinement step runs standard gradient-based optimization on the discrete set of sampled Gaussians for 5,000 iterations.

Key Contributions

Eulerian Formulation: A novel approach that bridges NeRF and 3DGS by optimizing a continuous probability field to govern discrete Gaussian placement, removing the need for heuristic density control.
Hashed Probability Pyramid: A memory-efficient, globally normalized, multi-scale representation for 3D probability densities that enables high-resolution optimization on a single GPU.
Low-Variance Gradient Estimator: A control variate-based estimator that isolates the contribution of each Gaussian to the rendered image, enabling stable training of the probability distribution without the high variance of standard pathwise estimators.
Heuristic-Free Training: The method achieves state-of-the-art results starting from a uniform random initialization, without relying on Structure-from-Motion (SfM) points or hand-tuned splitting/culling rules.

Results
The method was evaluated on the mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.

Performance: EGS achieves state-of-the-art reconstruction quality among models initialized randomly, outperforming baselines like 3DGS-MCMC and Taming-3DGS in overall PSNR.
Comparison to SfM-based Methods: While models initialized with COLMAP points generally perform slightly better, EGS starting from random initialization closes much of this gap, demonstrating robustness.
Ablation Studies: The custom gradient estimator is identified as the most critical component; without it, training becomes unstable and converges to poor results. Opacity regularization and defensive sampling also provide significant stability benefits.
Efficiency: While the training process is slower and more memory-intensive than standard 3DGS due to the large number of samples and hash-grid queries, the final rendering speed remains comparable to 3DGS.

Significance
The paper claims that EGS offers a "principled bridge" between continuous-field optimization and discrete Gaussian rendering. By replacing brittle heuristics with a learnable probability distribution, the framework allows the scene geometry to emerge purely from gradient cues. This enables a simple, end-to-end training procedure that generalizes across diverse scenes without requiring prior geometric initialization (SfM) or manual intervention, demonstrating that sampling-based formulations can achieve high-quality reconstructions while maintaining the efficiency of splat-based rendering.