💻 computer science

VkSplat: High-Performance 3DGS Training in Vulkan Compute

The paper presents VkSplat, the first fully Vulkan-based 3D Gaussian Splatting training pipeline that achieves state-of-the-art performance with a 3.3× speedup and 33% VRAM reduction compared to CUDA+PyTorch baselines while ensuring cross-vendor GPU compatibility.

Original authors: Jingxiang Chen, Mohamed Ibrahim, Yang Liu

Published 2026-05-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jingxiang Chen, Mohamed Ibrahim, Yang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a stunning, photorealistic 3D world using millions of tiny, glowing, fuzzy balloons (called "Gaussians"). This is what 3D Gaussian Splatting (3DGS) does. It's a popular way to create 3D scenes from photos, but until now, building these worlds has been like trying to paint a masterpiece with a slow, heavy brush that only works on one specific brand of computer (NVIDIA).

The paper introduces VkSplat, a new tool that acts like a high-speed, universal paintbrush. It builds these 3D worlds much faster, uses less memory, and works on almost any graphics card, not just NVIDIA's.

Here is how they did it, explained with everyday analogies:

1. The Problem: The "Traffic Jam" and the "Exclusive Club"

The Traffic Jam: In the old way of building these 3D scenes, the computer had to check every single pixel against every single balloon to see if they touched. This created a massive traffic jam where thousands of workers (computer threads) were all trying to write on the same clipboard at once, causing them to wait for each other.
The Exclusive Club: The best tools for this job were written in a language (CUDA) that only works on NVIDIA graphics cards. If you had an AMD or Intel card, you were locked out or had to use a much slower, clunky version.

2. The Solution: VkSplat's "Universal Toolkit"

The authors built VkSplat using Vulkan, a universal language that speaks to almost all graphics cards (NVIDIA, AMD, Intel, etc.) equally well. They didn't just translate the old code; they rebuilt the engine from the ground up to be incredibly efficient.

3. The Secret Ingredients (The Optimizations)

A. The "Perfect Map" (Tile Culling)

Old Way: Imagine a delivery driver checking every house on a street to see if they have a package, even if the house is clearly on the other side of town. This wastes time.
VkSplat Way: They use a "scan-line" method. It's like drawing a perfect, tight box around the balloon and only checking the houses that the box actually touches. They calculate this mathematically so there are zero mistakes (no checking houses that aren't there, and no missing houses that are). This stops the traffic jam before it starts.

B. The "Smart Team Leader" (Adaptive Scheduling)

Old Way: Imagine a factory where every worker is assigned a fixed number of tasks, regardless of how many tasks are actually waiting. Some workers sit idle while others are overwhelmed.
VkSplat Way: They use a "smart team leader" who watches the line of work. If a specific area has a lot of balloons, the leader assigns more workers to that spot. If there are few, they assign fewer. They even have a "gambler's strategy" (Thompson sampling) that tries different work schedules during training to find the absolute fastest way to get the job done for that specific scene.

C. The "One-Stop Shop" (Fused Operations)

Old Way: Imagine a chef who chops vegetables, puts them in a bowl, walks to the fridge to get spices, walks back, mixes them, and then walks to the stove. This involves a lot of walking (moving data in and out of memory).
VkSplat Way: They built a kitchen where chopping, mixing, and cooking happen in one single motion. They combine the "math of the camera" (projection) and the "learning step" (optimizer) into one single step. This means the data never has to leave the "kitchen counter" (memory), saving huge amounts of time and space.

D. The "Smart Sorting" (32-bit Sorting)

Old Way: Sorting millions of items usually requires writing down very long, detailed addresses (64-bit numbers) for each item.
VkSplat Way: They realized they could use shorter, smarter addresses (32-bit numbers) that still keep everything in the right order. It's like using a zip code instead of a full street address to sort mail—it's faster and takes up less space, with no loss of accuracy.

4. The Results: Faster, Leaner, and Universal

The paper tested this new tool against the current gold standard (GSplat) on an NVIDIA RTX 3090 card:

Speed: It was 3.3 times faster. If the old way took 23 minutes, VkSplat took about 7 minutes.
Memory: It used 33% less video memory (VRAM). This is like fitting a massive moving truck into a compact car.
Quality: The final 3D images looked exactly the same as the high-quality standard.
Compatibility: They tested it on an AMD card (Radeon RX 7800 XT). It worked! While it was slower than on NVIDIA (due to hardware differences in how data is moved), it proved the tool works across different brands, breaking the "NVIDIA-only" barrier.

Summary

VkSplat is a new, highly optimized engine for creating 3D worlds. It replaces the slow, exclusive, and memory-hungry methods of the past with a fast, universal, and efficient system. It's like upgrading from a horse-drawn carriage that only runs on cobblestones to a high-speed train that runs on any track, getting you to your destination three times faster while carrying less luggage.

1. Problem Statement

Despite the rapid adoption of 3D Gaussian Splatting (3DGS) for high-quality novel view synthesis, practical deployment is hindered by two main limitations:

Performance Bottlenecks: Existing training pipelines are often sub-optimal in speed and memory usage, particularly for large scenes. Key bottlenecks include false-positive tile intersections during sorting, atomic contention during backward rasterization, and inefficient memory handling in optimizers (e.g., PyTorch's Adam).
Hardware Lock-in: Most state-of-the-art implementations rely heavily on the CUDA ecosystem and PyTorch, restricting training to NVIDIA GPUs. This limits accessibility for users with AMD, Intel, or mobile GPUs.
Fragmentation: Previous attempts to use cross-vendor APIs (like Vulkan) have either only addressed rasterization (not end-to-end training) or relied on PyTorch for the training loop, failing to eliminate CUDA dependencies entirely.

2. Methodology

The authors present VkSplat, a fully end-to-end 3DGS training pipeline implemented entirely in Vulkan Compute. The system is built upon the Slang shading language (targeting Vulkan) to ensure cross-vendor compatibility while maintaining high performance.

The core methodology involves a complete rewrite of the 3DGS pipeline with specific architectural changes:

Full Vulkan Compute: The entire pipeline, from initialization and densification to optimization, runs on the GPU without CPU-side intervention or PyTorch dependencies.
Slang Backend: Leveraging Slang allows the code to target multiple backends, though the current implementation focuses on Vulkan.

3. Key Technical Contributions

The paper details five major technical optimizations that allow VkSplat to outperform CUDA-based baselines:

A. Complete Tile Culling via Scan-Line Intersection

Problem: Existing methods (e.g., [KKLD23], [RSP∗24]) generate false-positive tile intersections (checking tiles that don't actually overlap the Gaussian), wasting sorting and rasterization time.
Solution: VkSplat uses a scan-line formulation to compute exact intersections. By treating the Gaussian as an ellipse and calculating closed-form intervals for intersecting rows/columns, the system generates a precise list of tile-depth pairs.
Impact: Eliminates false positives without requiring a separate culling pass, reducing VRAM usage and sorting overhead.

B. Adaptive Rasterization Backward

Problem: Standard per-pixel backward passes cause high atomic contention when updating Gaussian gradients.
Solution: The authors implement two strategies and use a Thompson sampling scheduler to dynamically select the fastest one based on the scene:
1. Per-Tile Parallelism: One thread block per tile with dynamically adjusted Gaussian batch sizes (optimized to $S = \sqrt{NP}$ ) to minimize latency.
2. Shared Memory Optimization: A two-pass approach (forward pass computes transmittance in shared memory; backward pass accumulates gradients per Gaussian) to minimize thread divergence.
Impact: Significantly reduces atomic contention and improves thread utilization.

C. Fused Projection Backward and Optimizer

Problem: Standard implementations store gradients separately and use PyTorch's Adam, leading to unnecessary memory footprints and kernel launches.
Solution: The projection backward pass and the Adam optimizer are fused into a single kernel.
- Values and gradients for scales and opacities are transformed directly between log/logit space within the kernel.
- Spherical Harmonics (SH) coefficients are stored in a column-based format aligned with subgroup sizes (128-bit values) to improve memory coalescing.
Impact: Eliminates intermediate memory storage for gradients and reduces kernel launch overhead.

D. 32-bit Tile-Depth Sorting

Problem: Vulkan's native sorting is often slower than CUDA's optimized built-ins.
Solution: Instead of using 64-bit keys (32-bit Tile ID + 32-bit Depth), the authors use 32-bit keys. They map the depth $z$ to a range where only the lower 23 mantissa bits of an FP32 float vary, packing the Tile ID into the higher bits.
Impact: Enables efficient 32-bit sorting on Vulkan hardware without sacrificing depth precision or image quality.

E. Fully-Fused Loss Gradient Evaluation

Solution: A single kernel computes the weighted sum of L1 and SSIM losses and their gradients directly.
Optimization: Reference images are stored as UINT8 RGBA (avoiding FP32 conversion), and alpha masking is handled with near-zero overhead.
Impact: Removes intermediate reduction steps and memory layout conversions (channel-last to channel-first).

4. Results

The authors evaluated VkSplat on 7 scenes from the Mip-NeRF 360 dataset against the GSplat baseline (a leading CUDA+PyTorch implementation) on an NVIDIA RTX 3090.

Performance:
- Speed: Achieved a 3.3× speedup over the GSplat baseline (e.g., 412s vs. 1384s for default densification).
- VRAM: Reduced memory usage by 33% (3.01 GiB vs. 4.56 GiB).
- Granularity: VkSplat was faster in every individual pipeline stage (Projection, Tiling, Rasterization, Loss, Optimizer).
Quality:
- Image quality metrics (PSNR, SSIM, LPIPS) were identical to the baseline, confirming that the optimizations did not compromise fidelity.
Cross-Vendor Compatibility:
- Successfully trained on an AMD Radeon RX 7800 XT.
- While AMD was slower overall (1201s vs. 575s on NVIDIA), it produced consistent quality metrics and VRAM usage. The performance gap was primarily attributed to slower host-to-device memory transfers on AMD, suggesting room for further hardware-specific tuning.

5. Significance

First End-to-End Vulkan 3DGS: This is the first fully-Vulkan-based 3DGS training pipeline that achieves state-of-the-art performance, breaking the CUDA monopoly.
Hardware Agnosticism: By removing PyTorch and CUDA dependencies, VkSplat enables 3DGS training on AMD, Intel, and mobile GPUs, democratizing access to high-quality 3D reconstruction.
Performance Leadership: Counter-intuitively, the Vulkan implementation outperforms the highly optimized CUDA baseline, proving that careful low-level GPU optimization (fused kernels, precise culling, adaptive scheduling) can yield gains even on non-NVIDIA hardware.
Extensibility: The use of Slang suggests the pipeline can be easily ported to other backends like Metal, DirectX, or WebGPU, further broadening the ecosystem.

Limitations: The current implementation lacks some "production-ready" features found in mature trainers, such as exposure correction, depth supervision, batching, and multi-GPU training. However, the core architecture is designed to accommodate these features.