💻 computer science

PD-4DGS:Progressive Decomposition of 4D Gaussian Splatting for Bandwidth-Adaptive Dynamic Scene Streaming

PD-4DGS introduces the first framework for progressive, bandwidth-adaptive 4D Gaussian Splatting streaming by decomposing dynamic scenes into hierarchical, independently transmittable layers, which drastically reduces initial latency and bitstream size while enabling on-demand rendering on mobile networks.

Original authors: Jiachen Li, Guangzhi Han, Jin Wan, Delong Han, Yuan Gao, Min Li, Mingle Zhou, Gang Li

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiachen Li, Guangzhi Han, Jin Wan, Delong Han, Yuan Gao, Min Li, Mingle Zhou, Gang Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to stream a high-quality, 3D video of a moving object (like a spinning fan or a person waving) to a friend's phone. The problem with current technology is that the "movie file" is so huge and complex that your friend's phone has to download the entire thing before it can show them even a single frame. On a slow mobile connection, this means staring at a black screen for over a minute (sometimes 15 minutes!) before the video starts.

PD-4DGS is a new method that solves this by changing how the file is built and delivered. Here is how it works, using simple analogies:

1. The Old Way: The "All-or-Nothing" Box

Think of current 4D video files like a giant, sealed shipping crate. Inside, there is everything needed to see the video: the basic shape, the movement, and the fine details.

The Problem: You cannot open the crate until the entire truck delivers it. If you are on a slow road (low bandwidth), you wait forever. You can't see the shape first and then the movement later; you have to wait for the whole crate.

2. The PD-4DGS Solution: The "Progressive Puzzle"

PD-4DGS breaks that giant crate into three smaller, stackable boxes that can be sent one by one. This is called Hierarchical Deformation Decomposition (HDD).

Box 1: The Static Scaffold (The "Snapshot")
- What it is: This is a tiny file (about the size of a high-res photo) containing just the basic shape of the object, frozen in time.
- The Magic: Because it's so small, your friend's phone can download it in less than 2 seconds even on a slow connection. They can immediately see a static picture of the object. No more black screen!
Box 2: The Global Motion (The "Big Moves")
- What it is: This is a slightly larger file that adds the "big" movements, like the whole object spinning or moving across the room.
- The Magic: Once this downloads, the static picture starts moving in a smooth, coarse way. It's like seeing a puppet move its arms.
Box 3: The Local Refinement (The "Fine Details")
- What it is: The final, largest chunk adds the tiny, high-speed details, like the flutter of a flag or the texture of hair moving.
- The Magic: This turns the coarse puppet into a photorealistic, high-definition video.

The Result: Instead of waiting 73 seconds to see anything, the user sees a picture in 1.7 seconds, then the motion starts, and finally, the high-definition details appear as the internet connection allows. It's like watching a video load progressively, just like a YouTube video buffering, but for 3D worlds.

3. How They Made It Small and Stable

To make this work, the researchers had to solve two tricky problems:

The "Flicker" Problem: When you compress a video, sometimes the image jumps or flickers between frames, like a strobe light.
- The Fix: They invented a special "stabilizer" (called Temporal Mask Consistency). Imagine a conductor keeping a choir in sync; this tool ensures that the parts of the image that are hidden or shown stay consistent from one second to the next, so the video looks smooth and doesn't jitter.
The "Training" Problem: Usually, when you teach a computer to build these three layers, it treats them all the same. But the "big motion" and "fine detail" layers are much harder to learn than the basic shape. If you treat them equally, the computer gets lazy on the hard parts.
- The Fix: They created a smart "coach" (called Capacity-Weighted Rollout). This coach watches the scene and says, "This scene is very complex; let's spend more time practicing the hard details," or "This scene is simple; let's focus on the basics." It automatically adjusts the training schedule so every layer gets the attention it needs without the user having to tweak settings.

4. The Bottom Line

Speed: On a standard mobile network, PD-4DGS gets the first image to the screen in 1.7 seconds, compared to over a minute for previous methods.
Size: It shrinks the total file size by about 60% compared to the best existing methods, without losing video quality.
Adaptability: It works perfectly with existing streaming technology (like DASH or HLS), meaning it can be used on current video players and networks without needing new infrastructure.

In short, PD-4DGS turns a "wait forever" 3D video experience into a "instant start, then get better" experience, making high-quality 3D streaming possible on regular mobile phones.

Technical Summary: PD-4DGS

Problem Statement

Current 4D Gaussian Splatting (4DGS) methods enable high-quality dynamic novel view synthesis but suffer from a critical "all-or-nothing" deployment bottleneck. Existing models are monolithic bitstreams (typically ~25 MB) that clients must download entirely before rendering a single frame. On mobile networks (e.g., 2 Mbps), this results in first-frame latencies ranging from 73 to 930 seconds, far exceeding the 3–5 second user tolerance window for interactive applications.

While progressive compression exists for static 3DGS, it operates only on spatial anchors and cannot partition the temporal deformation networks that dominate the size of dynamic models. Furthermore, existing 4DGS compressors are single-rate (requiring retraining for bitrate changes) or treat progressivity as an external layer, failing to address the internal structure of deformation networks. Consequently, 4DGS remains incompatible with modern adaptive-bitrate (ABR) streaming protocols like DASH or HLS.

Methodology

The authors propose PD-4DGS, the first framework for progressive compression and on-demand transmission of 4DGS. The approach relies on four core components:

1. Hierarchical Deformation Decomposition (HDD)

The framework externalizes the latent coarse-to-fine physical hierarchy of 4DGS into three independently transmittable bitstream layers, all sharing a single anchor set:

Layer 0 (Static Scaffold): Contains canonical anchors and the attribute-prediction MLP. It renders a static snapshot immediately upon arrival, requiring no temporal input.
Layer 1 (Global Deformation): Transmits a lightweight global deformation network ( $F_{global}$ , typically HexPlane + MLP) that predicts per-anchor displacements. This layer captures dominant motion with a small volume increase (<10% of total model size).
Layer 2 (Local Refinement): Transmits the local deformation network ( $F_{local}$ ) which predicts per-Gaussian residuals ( $\Delta\mu, \Delta\Sigma, \Delta\alpha, \Delta c$ ) to recover high-frequency details (e.g., hair, facial muscles).
This structure allows any prefix of the bitstream to be rendered, enabling native compatibility with ABR protocols.

2. Attribute-Level Rate–Distortion Optimization (R-DO)

To ensure the base layer (Layer 0) remains small enough for second-scale latency, the authors cast attribute pruning as a differentiable rate–distortion problem rather than using heuristic thresholds.

Learnable Soft Masks: Each anchor $i$ has a learnable mask $m_i \in [0, 1]$ that multiplicatively gates opacity and scale. This avoids the distribution mismatch between soft training and hard pruning.
Gaussian Entropy Model: A differentiable entropy model approximates the bit cost of continuous attributes (features, scales, offsets) based on a per-batch Gaussian prior. This directly couples the training loss to the final bitstream size.
Layer-wise Objective: Distortion and bit cost are combined, with aggressive compression weights ( $\lambda_{layer}$ ) applied to the base layer to minimize its size while preserving fidelity for deeper layers.

3. Temporal Mask Consistency (TMC)

To prevent flickering and ghost trails caused by high-frequency mask oscillations in dynamic rendering, two regularizers are introduced:

Binary Entropy Penalty: Drives mask values deterministically toward $\{0, 1\}$ to eliminate semi-transparent ghost trails.
Spatial Smoothness Constraint: Enforces consistency between spatially adjacent anchors, implicitly stabilizing the activation pattern across frames without explicitly modeling the temporal axis.

4. Capacity-Weighted Progressive Rollout Training

A uniform sampling strategy during training leads to the under-training of high-capacity deformation networks (Layers 1 and 2), causing progressive quality collapse.

Adaptive Sampling: The authors introduce a learned statistic $\rho$ (the inter-level mask activation rate) to dynamically adjust the sampling distribution $\pi(\rho)$ .
Mechanism: $\pi(\rho)$ interpolates between a uniform distribution (optimal for static scenes) and an aggressive distribution (optimal for dynamic scenes). This reallocates the training budget to deeper layers based on the scene's specific deformation complexity, without requiring per-scene hyperparameter tuning.

Key Results

Experiments were conducted on the Dycheck iPhone monocular benchmark, comparing PD-4DGS against state-of-the-art baselines (SC-GS, Deformable 3DGS, 4DGS, and MoDec-GS).

Bitstream Reduction: PD-4DGS reduces the streamed bitstream volume by 62.6% compared to the strongest dynamic baseline (MoDec-GS) at matched rendering fidelity. The average stream size is 6.88 MB versus 18.37 MB for MoDec-GS.
Rendering Quality: The method achieves an average mPSNR of 14.54 dB, within 0.06 dB of MoDec-GS, while matching its mSSIM (0.480). It surpasses baselines on several specific scenes (Apple, Paper-windmill, Teddy, Wheel).
Latency Breakthrough: On a 2 Mbps mobile link:
- Monolithic baselines require 73.5s to 930s for the first frame.
- PD-4DGS delivers a renderable static first frame (Layer 0 only, 0.44 MB) in **1.7 seconds**.
- The full model is downloaded in 27.5 seconds, a quarter of the time required by MoDec-GS.
Progressive Monotonicity: The framework restores strict layer-wise monotonicity (PSNR increases with each added layer) across all scenes, a property often lost in progressive training due to sampling mismatches.

Significance and Claims

The paper claims that PD-4DGS is the first framework to enable true on-demand progressive streaming for 4DGS. Its significance lies in:

Breaking the Deployment Barrier: By reducing first-frame latency from tens of minutes to under 2 seconds on mobile networks, it makes 4DGS practically viable for interactive applications.
Native ABR Compatibility: Unlike previous methods that require retraining for different bitrates or treat scalability as an external layer, PD-4DGS decomposes the rendering network itself, making it natively compatible with DASH/HLS protocols.
Intrinsic Scalability: The capacity-weighted rollout training resolves the "capacity-sampling mismatch" in dynamic scenes without manual hyperparameter tuning, ensuring that the representation is intrinsically scalable to varying scene complexities.

The authors note limitations, including a fixed three-layer topology that may be less effective for near-rigid scenes and evaluation limited to the Dycheck iPhone benchmark. However, the work establishes a new paradigm for bandwidth-adaptive dynamic scene streaming.