💻 computer science

Mobile-GS: Real-time Gaussian Splatting for Mobile Devices

The paper proposes Mobile-GS, a mobile-tailored real-time Gaussian Splatting method that achieves efficient inference on edge devices by eliminating depth sorting through a depth-aware order-independent rendering scheme, enhancing visual quality with neural view-dependent effects, and reducing model size via distillation, quantization, and pruning.

Original authors: Xiaobiao Du, Yida Wang, Kun Zhan, Xin Yu

Published 2026-03-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiaobiao Du, Yida Wang, Kun Zhan, Xin Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a high-definition, 3D movie of a beautiful garden. In the world of computer graphics, this movie is made up of millions of tiny, glowing "paint splatters" (called Gaussians) floating in space. When you look at the scene from different angles, the computer has to figure out which splatters are in front and which are behind to paint the picture correctly.

This is the magic of 3D Gaussian Splatting (3DGS). It creates stunningly realistic images. But there's a catch: it's like trying to run a Hollywood blockbuster on a calculator. It requires so much memory and processing power that it usually crashes or runs too slowly on a smartphone.

The paper "Mobile-GS" introduces a new way to run these 3D movies on your phone, making them fast enough to watch in real-time (like 116 frames per second) without draining your battery or filling up your storage.

Here is how they did it, explained with simple analogies:

1. The Traffic Jam Problem (The Sorting Bottleneck)

The Old Way: Imagine a line of cars (the paint splatters) trying to enter a tunnel. To make sure the cars in the front aren't blocked by the ones in the back, the traffic cop has to stop every car, check its distance, and line them up perfectly from closest to furthest before letting them through. This "sorting" takes forever and causes a massive traffic jam on your phone's processor.

The Mobile-GS Solution: Instead of lining them up, Mobile-GS uses a "Depth-Aware Order-Independent" trick.

The Analogy: Imagine instead of a traffic cop, you have a smart sprinkler system. It doesn't care about the order of the cars. It just knows that cars far away should get a tiny drop of water (low weight), and cars close by should get a big splash (high weight).
The Result: The computer doesn't need to stop and sort the cars. It just sprays them all at once. This removes the biggest bottleneck, making the process incredibly fast.

2. The "Ghostly" Problem (Transparency Artifacts)

The Problem: Because Mobile-GS stops sorting the cars, sometimes things look a bit "ghostly" or see-through where they shouldn't be. If two cars overlap, the computer might get confused about which one is actually blocking the other.

The Mobile-GS Solution: They added a "Neural View-Dependent Enhancement."

The Analogy: Think of this as a smart artist standing next to the camera. When the computer gets confused about overlapping cars, this artist steps in. Based on the angle you are looking from, the artist says, "Hey, that car is actually solid, not transparent!" and fixes the color instantly.
The Result: Even though the computer isn't sorting the cars perfectly, this "smart artist" (a tiny neural network) fixes the visual glitches, making the image look sharp and real.

3. The Suitcase Problem (Storage & Compression)

The Problem: A full 3D scene is like a suitcase packed with 1,000 heavy winter coats. Your phone's memory is a tiny backpack; it can't fit the suitcase.

The Mobile-GS Solution: They use three tricks to shrink the suitcase:

Distillation (The Summary): Instead of keeping the full, complex "3rd-grade" description of every color, they teach the system to use a "1st-grade" summary. It's like summarizing a 500-page novel into a 5-page outline. It loses a tiny bit of detail but keeps the main story, saving massive space.
Vector Quantization (The Library): Imagine you have 10,000 unique shades of blue. Instead of writing down the code for every single one, you create a small library of 1,000 "standard blue codes." You just tell the computer, "Use code #42 for this part." This is like compressing a file into a zip folder.
Pruning (The Edit): They look at the scene and say, "These tiny, faint paint splatters in the background aren't doing much work. Let's throw them away." They remove the "dead weight" to make the scene lighter.

The Final Result

By combining these tricks, Mobile-GS turns a heavy, slow 3D movie into a lightweight, fast-running app.

Before: A 3D scene might take up 1.2 GB of space and run at 116 FPS on a supercomputer, but only 12 FPS (choppy) on a phone.
After (Mobile-GS): The same scene takes up only 4.8 MB (smaller than a few photos!) and runs at 116 FPS (smooth as butter) on a modern phone.

In short: Mobile-GS is like taking a luxury limousine, stripping off the heavy armor, giving it a turbo engine, and teaching it to drive itself, so it can zoom through your phone's screen just as smoothly as it does on a supercomputer.

1. Problem Statement

3D Gaussian Splatting (3DGS) has revolutionized high-quality 3D reconstruction and novel view synthesis. However, deploying it on mobile devices faces three critical bottlenecks:

Computational Overhead: The standard rendering pipeline relies on alpha blending, which requires sorting 3D Gaussians by depth (near-to-far) for every pixel. This sorting process is computationally expensive and creates a severe latency bottleneck on mobile GPUs.
Storage Costs: Original 3DGS models require hundreds of megabytes to gigabytes of storage, exceeding the memory constraints of mobile devices.
Visual Artifacts: Removing the sorting process (to gain speed) often leads to transparency artifacts and incorrect occlusion handling in overlapping geometries.

Existing lightweight methods (e.g., Scaffold-GS, Mini-Splatting) still rely on traditional sorting-based alpha blending, failing to achieve true real-time performance on edge devices.

2. Methodology

The authors propose Mobile-GS, a comprehensive framework tailored for mobile inference. It integrates four key innovations:

A. Depth-Aware Order-Independent Rendering

To eliminate the sorting bottleneck, Mobile-GS replaces traditional alpha blending with a depth-aware order-independent rendering scheme.

Mechanism: Instead of sorting, it computes a weighted accumulation of all Gaussians contributing to a pixel in parallel.
Weighting Strategy: A learnable weighting function $w(d_i)$ modulates contributions based on depth ( $d_i$ ) and scale ( $s_{max}$ ). It explicitly reduces the weight of distant Gaussians and increases the weight of larger/nearby ones.
Formula: The pixel color $C$ is computed as:
$C = (1 - T) \frac{\sum c_i \alpha_i w_i}{\sum \alpha_i w_i} + T c_{bg}$
where $T$ is global transmittance, and $w_i$ is the depth-aware weight. This allows for parallel processing without sorting.

B. Neural View-Dependent Enhancement

Order-independent rendering can cause transparency artifacts in occluded regions. To fix this without re-introducing sorting:

Approach: A lightweight Multi-Layer Perceptron (MLP) predicts a view-dependent opacity scalar and a depth attenuation factor ( $\phi$ ) for each Gaussian.
Inputs: The MLP takes the camera-to-Gaussian vector, scale, rotation, and spherical harmonic (SH) coefficients as input.
Effect: The network learns to dynamically suppress the transparency of occluded regions and enhance view-dependent effects, compensating for the lack of strict depth ordering.

C. Compression via Distillation and Quantization

To drastically reduce storage size:

First-Order SH Distillation: Instead of storing high-order (3rd order) SH coefficients, the model distills them into 1st-order SH (4 coefficients per channel) using a teacher-student framework. A scale-invariant depth distillation loss ensures geometric consistency.
Neural Vector Quantization (NVQ): Gaussian attributes are decomposed and clustered via K-Means. Each cluster is quantized using a specific codebook. The quantized features are then decoded by lightweight MLPs at inference time, eliminating the need to store raw high-dimensional vectors. Huffman coding is applied for final bitstream compression.

D. Contribution-Based Pruning

Strategy: Redundant Gaussians are identified and removed based on a joint analysis of opacity and spatial scale.
Mechanism: Gaussians with low opacity and small scale are considered insignificant. A voting mechanism accumulates "prune votes" over training iterations to ensure only consistently low-contributing primitives are permanently removed, preventing noisy fluctuations from affecting the model.

3. Key Contributions

First Mobile-Optimized 3DGS: Mobile-GS is the first method to achieve real-time rendering (>100 FPS) on mobile devices (Snapdragon 8 Gen 3) for high-resolution scenes.
Sorting-Free Pipeline: It successfully eliminates the depth-sorting bottleneck, enabling parallel rendering that is native to mobile GPU architectures.
Hybrid Quality-Speed Architecture: By combining order-independent rendering with neural view-dependent enhancement, it maintains high visual fidelity (comparable to original 3DGS) while avoiding the artifacts typically associated with sorting-free methods.
Extreme Compression: The method reduces model storage from ~1.2 GB (original 3DGS) to ~4.8 MB (a ~250x reduction) while maintaining high PSNR.

4. Experimental Results

The authors evaluated Mobile-GS on standard datasets (Mip-NeRF 360, Tanks&Temples, Deep Blending) and deployed it on a mobile device.

Rendering Speed:
- Mobile (Snapdragon 8 Gen 3): Achieves 116–127 FPS at 1600×1063 resolution.
- Desktop (RTX 3090 Ti): Achieves 1098 FPS on unbounded scenes.
Storage Efficiency:
- Reduced storage to 4.6 MB (Mip-NeRF 360) and 2.5 MB (Tanks&Temples), significantly outperforming other lightweight methods (e.g., LocoGS-S at 8.5 MB, Speedy-Splat at 79.4 MB).
Visual Quality:
- Achieves 27.12 PSNR on Mip-NeRF 360, comparable to the original 3DGS (27.21) and superior to other lightweight baselines.
- Qualitative results show sharper details and better handling of view-dependent effects compared to SortFreeGS and Speedy-Splat.
Ablation Studies:
- Removing the order-independent rendering drops FPS from ~1125 to ~684.
- Removing the neural view-dependent enhancement causes a significant drop in PSNR (from 27.12 to 26.68) due to transparency artifacts.
- Removing quantization increases storage from 4.6 MB to 121 MB.

5. Significance

Mobile-GS represents a pivotal step toward the practical deployment of high-fidelity 3D reconstruction on consumer hardware. By solving the sorting bottleneck and addressing storage constraints simultaneously, it enables applications such as:

Real-time AR/VR on smartphones.
On-device 3D scene exploration.
Efficient 3D content delivery for autonomous driving and robotics.

The paper demonstrates that with specialized architectural changes (order-independent rendering + neural enhancement), the trade-off between rendering quality and computational cost can be effectively managed for edge devices.