Technical Summary: Rendering 3D Gaussians on a Graph Processor

Problem Statement

3D Gaussian Splatting (3DGS) has established itself as a leading method for real-time novel view synthesis, offering photorealistic rendering and full differentiability. However, current implementations are heavily optimized for GPU architectures, which rely on large off-chip DRAM, high memory bandwidth, and a global address space. On GPUs, 3DGS rendering is memory-bound; the dominant cost is the repeated movement of Gaussian data through the memory hierarchy (loading from DRAM to shared memory) rather than arithmetic computation.

This reliance on global memory access is inefficient for emerging resource-constrained platforms, such as AR glasses and mobile robots, where "on-sensor" computing with only local memory is desirable. The paper addresses the challenge of adapting 3DGS to a DRAM-free architecture that lacks a global address space and random-access memory, forcing data movement to be explicit and local.

Methodology

The authors implemented the first 3DGS renderer on Graphcore's Intelligence Processing Unit (IPU), a massively parallel processor comprising 1,472 independent tiles. Each tile possesses 624 KB of local SRAM, 6 hardware threads, and no external DRAM. The architecture follows a Bulk Synchronous Parallel (BSP) model with explicit, compile-time-defined communication channels.

Pipeline Overview

The rendering pipeline maps the four stages of 3DGS onto the IPU's BSP execution model, alternating between local computation and inter-tile communication:

1. Projection (Local Compute): Each tile projects its locally stored 3D Gaussians into 2D screen space, computing means, conics, and bounding boxes.
2. Routing (Compute + Exchange): Gaussians whose projected means fall outside a tile's local screen region are evicted. They are routed toward their "anchor tile" (the tile owning the screen region containing the Gaussian's mean) via Manhattan-distance hops on a North-East-West-South (NEWS) grid. This process repeats over multiple BSP supersteps until all Gaussians reach their anchor tiles.
3. Bloom (Compute + Exchange): Once a Gaussian reaches its anchor tile, it is propagated to neighboring tiles whose screen regions it overlaps. To prevent cyclic copying and channel saturation, the authors employ a tree-pattern propagation: the anchor tile sends copies horizontally, and those tiles forward copies vertically.
4. Compositing (Local Compute): Each tile independently sorts its local Gaussians by depth and performs front-to-back alpha compositing onto its framebuffer slice.

Key Architectural Constraints

• No Global Memory: Scene data never leaves on-chip SRAM. The entire 3D Gaussian map is distributed across the tiles; no single tile holds the whole scene.
• Explicit Data Movement: Unlike GPUs where threads can load arbitrary global addresses, IPU tiles can only send/receive data defined at compile time. The algorithm must explicitly manage the migration of primitives.
• Memory Budget: With ~1,472 tiles, the aggregate memory is sufficient for moderate scenes, but individual tiles have strict limits (approx. 192 KB available for Gaussian data after accounting for buffers).

Key Contributions

1. First SRAM-Only 3DGS Implementation: The paper presents the first rendering of 3D Gaussians on a MIMD (Multiple Instruction, Multiple Data) processor architecture that relies exclusively on on-chip SRAM, eliminating external DRAM access for scene data.
2. Compile-Time Routing Scheme: A novel mechanism for distributing Gaussian primitives across tiles based on screen-space ownership, operating within the constraints of fixed, compile-time communication channels.
3. Performance and Bottleneck Analysis: A detailed evaluation of the system's performance, identifying specific bottlenecks inherent to DRAM-free rendering:
   • Inter-tile Bandwidth Saturation: In dense regions, outgoing channels can fill before all Gaussian copies are propagated, leading to "tiling artifacts" (rectangular holes) in the rendered image.
   • Per-Tile SRAM Pressure: High Gaussian density in specific screen regions can exceed a single tile's buffer capacity, causing data loss.
   • Workload Imbalance: Non-uniform Gaussian density causes some tiles to be significantly slower than others, stalling the entire BSP barrier.
4. Insights for Future Architectures: The work suggests that explicit inter-SM (Streaming Multiprocessor) communication on GPUs could reduce DRAM access, and that 3D representations could be designed to better exploit spatial and temporal locality.

Results

• Visual Quality: For moderate-density scenes (e.g., 25k–90k Gaussians), the IPU rendering is visually near-identical to the GPU baseline, preserving fine details like text and thin structures.
• Throughput: The system achieves interactive framerates (approx. 20 FPS) on a single IPU for small-to-medium scenes. However, performance degrades with very dense scenes (e.g., 273k Gaussians) due to the aforementioned bottlenecks.
• Power Efficiency: While the IPU's raw throughput (FPS) is lower than high-end GPUs (GTX 1080, RTX 4090), its power consumption is significantly lower (~27W vs ~75–90W). The architecture demonstrates that 3DGS can run entirely on-chip, with host communication accounting for only 1.6% of execution time.
• Data Locality: The system exploits temporal locality effectively. For incremental camera movements (typical in SLAM/AR), the "churn rate" (fraction of Gaussians requiring re-routing) is very low (<12%), whereas random teleports require nearly 100% re-routing.

Significance and Claims

The paper does not claim the IPU is a superior general-purpose rendering engine compared to GPUs. Instead, it uses the IPU as an experimental platform to demonstrate that 3DGS does not fundamentally require DRAM.

The authors argue that by stripping away the global address space, the necessity of explicit data movement becomes apparent. This reveals that:

1. Locality is Key: Rendering is most efficient when data movement is minimized. The IPU's design forces the algorithm to exploit the fact that in incremental viewing scenarios (robotics, AR), most primitives remain near their previous screen-space locations.
2. Algorithmic Implications: The bottlenecks observed (bandwidth, capacity, imbalance) highlight trade-offs that could inform future GPU kernel designs. Specifically, the authors suggest that direct inter-SM communication on GPUs, rather than routing all cross-block data through DRAM, could reduce the dominant memory costs of 3DGS.
3. Future Directions: The work motivates the development of "on-sensor" and edge architectures for differentiable rendering, suggesting that future 3D representations and hardware should be co-designed to leverage spatial and temporal coherence, potentially reducing power and latency by avoiding global memory access.