💻 computer science

SpeeDe3DGS: Speedy Deformable 3D Gaussian Splatting with Temporal Pruning and Motion Grouping

SpeeDe3DGS accelerates dynamic 3D Gaussian Splatting by introducing Temporal Sensitivity Pruning, Temporal Sensitivity Sampling, and GroupFlow to significantly reduce computational costs and training time while maintaining high reconstruction fidelity.

Original authors: Allen Tu, Haiyang Ying, Alex Hanson, Yonghan Lee, Tom Goldstein, Matthias Zwicker

Published 2026-06-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Allen Tu, Haiyang Ying, Alex Hanson, Yonghan Lee, Tom Goldstein, Matthias Zwicker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to create a high-speed, 3D movie of a dancing chicken. In the world of computer graphics, there's a popular technique called 3D Gaussian Splatting. Think of this as building your movie scene out of millions of tiny, fluffy, 3D "clouds" (Gaussians) that float in space. When you look at the scene, the computer blends these clouds together to create a sharp, realistic image.

The problem with making these scenes move (like a chicken flapping its wings) is that the old methods are incredibly slow and heavy. They treat every single fluffy cloud as a unique character that needs its own personal "brain" (a neural network) to figure out how to move at every single moment. It's like hiring a personal choreographer for every single feather on the chicken. The result? The computer gets overwhelmed, the movie renders slowly, and it takes forever to train.

The paper introduces SpeeDe3DGS, a new framework that acts like a super-efficient director for this 3D movie. It speeds things up by using three clever tricks to stop the computer from doing unnecessary work.

1. The "Cut the Fluff" Trick (Temporal Sensitivity Pruning)

The Problem: In the original method, the computer keeps millions of clouds, even though many of them are just sitting there doing nothing or moving in ways that don't actually change the picture much. It's like having a cast of 10,000 actors when only 1,000 are actually in the scene.

The Solution: The authors use a tool called Temporal Sensitivity Pruning (TSP). Imagine a film editor who watches the movie and asks, "If I remove this specific cloud, does the picture change?" If the answer is "No, not really," that cloud gets cut.

How it works: Instead of just looking at one frame, the editor looks at the whole timeline. If a cloud is boring or redundant over time, it gets deleted.
The Result: They cut the number of clouds by about 10 to 12 times. The movie is now much lighter and faster to render because the computer has fewer clouds to juggle.

2. The "Shake It Up" Trick (Temporal Sensitivity Sampling)

The Problem: Sometimes, when you cut out the "useless" clouds, you accidentally leave behind some weird, floating artifacts (called "floaters"). These are like ghostly, floating specks that appear out of nowhere when the camera moves, ruining the illusion.

The Solution: To prevent this, they use Temporal Sensitivity Sampling (TSS). Imagine the editor is nervous about cutting a cloud, so they give the timeline a little "shake" or "jitter" to see if the cloud behaves weirdly when the timing changes slightly.

How it works: They pretend the movie is playing at slightly different speeds or times. If a cloud looks stable in the real movie but turns into a floating ghost when the time is tweaked, the system knows it's unstable and cuts it out safely.
The Result: This stops those annoying floating ghosts from appearing, making the video smoother and cleaner without losing quality.

3. The "Group Dance" Trick (GroupFlow)

The Problem: Even after cutting the extra clouds, the computer still has to ask every single remaining cloud, "How do you move?" This is still a lot of work.

The Solution: They introduce GroupFlow. Instead of giving every cloud its own choreographer, they group clouds that move together (like a flock of birds or a rigid arm) and give the whole group one shared choreographer.

How it works: If a group of clouds is moving as a solid block (like a chicken's wing), the computer just calculates the movement for the whole wing once, rather than calculating it for every single feather individually.
The Result: This is the biggest speed boost. It turns a complex, individual calculation into a simple, shared one.

The Grand Finale: How Fast is It?

When they put all three tricks together, the results are staggering:

Rendering Speed: The movie plays back 13.7 times faster than the previous best method. In some scenes, it's nearly 30 times faster.
Training Time: It takes 2.5 times less time to teach the computer how to make the movie.
Quality: Despite cutting out 90% of the clouds and simplifying the math, the picture quality remains just as sharp and realistic as the slow, heavy original method.

In short: SpeeDe3DGS is like taking a bloated, slow-moving 3D animation and giving it a trim, a shake, and a group dance routine. The result is a lightning-fast, high-quality 3D movie that runs smoothly on standard computers, proving you don't need millions of individual "brains" to make a moving scene look real.

Technical Summary: SpeeDe3DGS

Problem Statement

Dynamic extensions of 3D Gaussian Splatting (3DGS), such as DeformableGS and 4DGS, achieve high-fidelity reconstructions of time-varying scenes by coupling Gaussian primitives with learned neural motion fields. However, these methods suffer from a significant efficiency–fidelity gap. Because they require per-Gaussian neural inference at every frame to compute deformations, they incur substantial computational costs, resulting in slow training and rendering speeds compared to analytic or non-neural representations. While neural motion fields offer superior reconstruction quality, their per-primitive inference overhead limits their applicability in real-time scenarios.

Methodology

The authors propose SpeeDe3DGS (Speedy Deformable 3D Gaussian Splatting), a framework designed to bridge this gap by reducing redundant neural inference while preserving the expressiveness of neural motion fields. The approach integrates three complementary modules:

1. Temporal Sensitivity Pruning (TSP)

TSP addresses the over-parameterization of dynamic 3DGS models by removing redundant or low-impact Gaussians. Unlike static pruning methods that evaluate gradients at fixed viewpoints, TSP computes a temporal sensitivity pruning score by aggregating second-order sensitivities of the reconstruction loss across all training views and timesteps.

Mechanism: It calculates the gradient magnitude of the projected contribution of each Gaussian ( $g_i$ ) over the set of training poses and timestamps ( $P_{gt}$ ).
Goal: By identifying Gaussians with low cumulative influence on the reconstruction across both space and time, TSP eliminates motion-dependent redundancies, directly reducing the number of primitives and the associated deformation inference cost.

2. Temporal Sensitivity Sampling (TSS)

TSS is a stabilization mechanism designed to prevent the removal of Gaussians that appear stable in observed frames but exhibit temporal instability (manifesting as "floaters") when extrapolated to unseen timestamps.

Mechanism: During sensitivity estimation, TSS injects a linearly annealed Gaussian perturbation into the timestamp input of the deformation function ( $t + X(i)$ ). This probes the model's sensitivity to small motion shifts.
Goal: Gaussians that appear well-behaved on training frames but drift under small temporal perturbations receive lower sensitivity scores and are pruned. This suppresses floating artifacts and improves temporal coherence without requiring ground-truth supervision.

3. GroupFlow

While pruning reduces the number of primitives, the remaining Gaussians still require individual neural inference for deformation. GroupFlow addresses this by distilling the learned neural motion field into efficient, shared rigid transformations.

Mechanism:
1. Motion Analysis: The trajectories of deformable Gaussians are analyzed to identify clusters with similar motion. Control points are selected via farthest point sampling, and Gaussians are assigned to the control point with the most similar trajectory.
2. Rigid Distillation: For each group, a time-varying SE(3) rigid transformation (rotation and translation) is estimated using Umeyama alignment, mapping the group from the canonical frame to the current frame.
3. Inference: Instead of predicting a unique deformation for every Gaussian, the system applies a single shared SE(3) transformation to all Gaussians within a group.
Goal: This reduces the number of motion trajectories from $N$ (per Gaussian) to $J$ (per group), significantly accelerating inference while maintaining temporal coherence through locally rigid motion modeling.

Key Contributions

Temporal Sensitivity Pruning (TSP): A gradient-based pruning strategy that aggregates sensitivity over time to remove redundant Gaussians in dynamic scenes, reducing deformation and rendering costs.
Temporal Sensitivity Sampling (TSS): A temporal perturbation technique that stabilizes the pruning process by exposing and removing temporally inconsistent primitives (floaters), thereby improving reconstruction stability.
GroupFlow: A method that distills learned neural deformation fields into grouped SE(3) transformations, replacing per-Gaussian inference with efficient group-wise rigid motion.

Experimental Results

The authors evaluate SpeeDe3DGS on the MonoDyGauBench (50 dynamic scenes) and specific datasets including NeRF-DS, D-NeRF, and HyperNeRF.

Performance on MonoDyGauBench:
- Integrating TSP and TSS into the DeformableGS baseline accelerates rendering by 6.78× and training by 2.18× on average, while using 10× fewer primitives and maintaining neural-field fidelity.
- Adding GroupFlow further increases rendering speed to 13.71× and reduces training time by 2.53×.
- The full SpeeDe3DGS pipeline surpasses all non-neural baselines in speed while outperforming them in image quality (PSNR, SSIM).
Specific Scene Results (HyperNeRF "chicken" scene):
- Achieves 9.88× faster rendering.
- Reduces Gaussians by 11.37×.
- Shortens training time by 2.87×.
Memory Efficiency:
- Pruning integration significantly reduces memory usage, enabling training on 24 GB VRAM (RTX A5000) for scenes that previously exceeded the memory limits of baseline methods (which often required 48 GB VRAM).

Significance and Claims

The paper claims that SpeeDe3DGS successfully bridges the efficiency–fidelity gap in dynamic 3D Gaussian Splatting. By combining temporally aware pruning with motion distillation, the framework achieves neural-field fidelity at real-time rendering speeds.

The authors emphasize that their approach does not merely trade quality for speed; rather, the regularization effects of TSS and GroupFlow often stabilize reconstruction in scenes with noisy or unstable motion, sometimes yielding higher image quality than the unpruned baseline. The work demonstrates that reducing redundant primitives and grouping similar motion trajectories allows dynamic 3DGS to approach the speed of analytic models without sacrificing the high-quality reconstruction capabilities of neural motion fields.