💻 computer science

Grassmannian Splatting I: Moving rank-2 Spacetime Surfels for Dynamic Scene Rendering

This paper introduces Grassmannian Splatting, a dynamic scene representation using rank-2 spacetime surfels defined by 3-planes in $\mathbb{R}^4$ that achieve state-of-the-art rendering quality with significantly faster training speeds by employing a closed-form motion model compatible with standard 3D Gaussian Splatting rasterizers.

Original authors: Aaron Maurice Berman, Shantanu Dave

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aaron Maurice Berman, Shantanu Dave

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to film a busy street scene where cars zoom by, people wave, and a balloon floats upward. To recreate this scene on a computer, most methods try to build a 3D model of every object and then teach it how to wiggle, stretch, or deform over time. It's like trying to animate a clay sculpture by manually squishing and pulling it frame by frame.

But the authors of this paper, Aaron Maurice Berman and Shantanu Dave, have a different idea. They suggest we stop trying to squish clay and start thinking in "spacetime sheets."

The Big Idea: Moving Sheets of Paper

Instead of building 3D blobs that change shape, the authors propose using Gaussians (which are like soft, fuzzy clouds of color) that live on flat, 3-dimensional sheets floating in a 4-dimensional world. Think of this 4D world as having three dimensions for space (up/down, left/right, forward/back) and one dimension for time.

In their system, every little "splat" (a fuzzy dot of color) is stuck to a specific flat sheet. Because the sheet is flat and exists in spacetime, it has a very special property: it moves at a constant speed in a straight line.

Here is the magic trick:

The Sheet: Imagine a piece of paper floating in the air. In this 4D world, that paper isn't just sitting still; it's sliding through time.
The Slice: When you want to see the scene at a specific moment (say, 2:00 PM), you "slice" through the 4D world at that time.
The Result: When you slice that moving 4D sheet, you get a perfect, flat 2D disk (a surfel) in your 3D view. As time moves forward, that disk slides across the screen at a steady speed, just like the sheet it came from.

The paper calls this Grassmannian splatting. It's a fancy name for a simple concept: if you define your objects as flat sheets moving through time, you don't need to teach them how to move. The movement is built into the geometry of the sheet itself.

What They Say "No" To

The authors are very clear about what they are not doing.

No Deformation Fields: They explicitly reject the idea of using complex "deformation fields" (mathematical maps that tell every point how to warp and twist). Their method doesn't learn how to bend or stretch; it only learns how to slide.
No Custom Code: They don't need to write special, complicated computer code (custom CUDA kernels) to make this work. They use the standard "rasterizer" (the engine that turns 3D data into 2D images) that everyone else uses for static 3D scenes. They just feed it a slightly different kind of data.
No "Rank-4" Blobs: Other methods use "full-rank" 4D Gaussians, which are like 4D eggs that get sliced into 3D eggs. The authors argue that for dynamic scenes, we actually want "rank-2" disks (flat pancakes), not 3D eggs. Their math proves that slicing their specific 4D sheets always gives you these flat pancakes.

How Sure Are They?

The authors are quite confident in their math and their results, but they are careful about their claims.

The Math is Proven: They have mathematically proved that if you slice their 4D sheets, you get a flat 2D disk that moves at a constant speed. They proved this using geometry and linear algebra (specifically involving something called the "Schur complement").
The Speed is Measured: They tested their method on 17 different video scenes (from a benchmark called MonoDyGauBench). They found that their method is 4.9 to 5.6 times faster to train than the best competing methods that focus on quality. They trained in about 1,814 seconds (roughly 30 minutes), while the next best took over 8,000 seconds.
The Quality is High, But Not Perfect: In terms of image quality (measured by PSNR, MS-SSIM, and LPIPS), their method came in second place among the methods they tested. It was very close to the winner (4DGS), but not quite there.
- They admit a specific limitation: Their method struggles with objects that rotate or spin in a coordinated way (like a spinning fan or a person doing a cartwheel). Because their sheets only slide in straight lines, they have to use hundreds of tiny, short-lived sheets to fake a rotation. This is why they didn't beat the "deformation" methods on those specific types of motion.

The "Secret Sauce" Details

The "Soft Clamp": There's a tricky moment in the math when a sheet is perfectly still (not moving through time). The math can break down there. The authors added a "soft clamp" (a tiny numerical safety net) that smoothly bridges the gap between a moving sheet and a static one, so the computer doesn't crash.
The "Lift": Since their disks are perfectly flat (mathematically "rank-2"), they are technically "singular" (hard for computers to handle). To fix this, they add a tiny, invisible amount of thickness (called a "rank-lift") just before the image is drawn. This makes the math work without actually changing how the image looks.
The Results: On the 17 test scenes, their method achieved a PSNR of 25.05 dB, which is very close to the top score of 25.70 dB. They trained in 1,814 seconds on an NVIDIA A10G, compared to 10,171 seconds for the top competitor (4DGS) on an RTX 3090.

The Bottom Line

The paper suggests that by changing how we think about 3D objects—viewing them as flat sheets sliding through time rather than squishy blobs—we can make dynamic scene rendering much faster and simpler. It's a clever geometric shortcut that avoids the heavy lifting of learning complex deformations. While it's not the absolute best at every single type of motion (especially spinning things), it's a massive win for speed and simplicity, proving that sometimes, the best way to move a scene is to just let the math slide.

Technical Summary: Grassmannian Splatting I

Problem Statement
Existing dynamic scene rendering methods face a trade-off between representational flexibility and computational efficiency. Deformation-based approaches (e.g., 4DGS, DeformableGS) learn complex deformation fields to warp canonical primitives, while native 4D Gaussian Splatting methods (e.g., RTGS) place full-rank 4D Gaussians in spacetime. However, native methods suffer from a fundamental geometric mismatch: conditioning a full-rank 4D Gaussian on time yields a rank-3 volumetric ellipsoid in the 3D slice. This volumetric nature is inefficient for representing thin surfaces and requires significant computational overhead. Conversely, methods using explicit trajectories or deformation fields often require custom CUDA kernels and learned motion models that lack the closed-form simplicity of static 3DGS.

Methodology
The authors introduce Grassmannian Splatting, a dynamic representation where primitives are Gaussians supported on 3-planes in spacetime $\mathbb{R}^4$ . These 3-planes correspond to spatial 2-planes (disks) undergoing uniform translation along their normals.

Geometric Primitive: Each primitive is defined by a unit normal $n \in S^3/\{\pm 1\} \cong Gr(3, 4)$ and an unconstrained factor $L \in \mathbb{R}^{4 \times 3}$ . The 4D covariance is constructed as:
$\Sigma_{4D} = (P_n L)(P_n L)^T$
where $P_n = I - nn^T$ is the orthogonal projector onto the 3-plane. This construction ensures $\Sigma_{4D}$ has rank at most 3, with the kernel direction aligned with $n$ .
Time Conditioning: When conditioned on a specific time $t_0$ , the 4D Gaussian slices into a rank-2 surfel (a disk) in $\mathbb{R}^3$ . The normal and normal velocity of this disk are derived directly from the spacetime normal $n$ , while the disk's shape and tangential drift are determined by $L$ . This provides a closed-form motion model without learned deformation fields.
Handling Static Orientations: A singularity arises when the plane normal approaches the static orientation ( $n = \pm e_0$ ), where the temporal variance vanishes. The authors employ a soft clamp in the Schur denominator ( $\sqrt{(\Sigma_{tt}^{pure})^2 + \epsilon^2}$ ) to regularize this singularity. This creates a continuous bridge between dynamic rank-2 behavior and static rank-3 behavior, allowing the method to represent both moving and static objects within a single family of primitives.
Rendering Pipeline: The conditioned rank-2 disk is lifted by a small isotropic variance ( $\sigma_{lift}^2 I_3$ ) to ensure invertibility for standard CUDA kernels. It is then fed into an unmodified 3DGS rasterizer via the precomputed covariance interface. Temporal opacity is modulated by a weight derived from the temporal variance, allowing primitives to fade in and out without explicit creation/destruction logic.

Key Contributions

Rank-Deficient Native Primitives: The paper presents the first native spacetime Gaussian representation whose time-conditioned slices are rank-2 surfels rather than volumetric ellipsoids. This aligns the primitive geometry with the physical nature of surfaces.
Closed-Form Motion Model: The motion of each primitive is induced purely by the geometry of the supporting 3-plane. There is no need to learn deformation fields or explicit trajectory curves; the constant-velocity motion is an intrinsic property of the plane.
Compatibility with Standard Rasterizers: The method requires no custom CUDA kernels. By utilizing a rank-lift and a precomputed covariance interface, it leverages existing, highly optimized 3DGS rasterization code.
Unified Static-Dynamic Representation: Through the numerical bridge (soft clamp), the method seamlessly interpolates between static (rank-3) and dynamic (rank-2) regimes, avoiding the need for separate static and dynamic components.

Results
Evaluated on the 17 HyperNeRF scenes of the MonoDyGauBench benchmark:

Training Speed: Grassmannian Splatting is the fastest method among all compared baselines, training 4.9 to 5.6 times faster than the strongest quality baselines (e.g., 4DGS) and 1.4 times faster than static 3DGS.
Quality: The method ranks second in PSNR, MS-SSIM, and LPIPS. It achieves a PSNR of 25.05 dB, outperforming DeformableGS by 0.47 dB but trailing 4DGS by 0.65 dB.
Efficiency: It renders at approximately 88 FPS (11.3 ms/frame) with a mean of 196k primitives.

Significance and Limitations
The paper claims significance in providing a computationally efficient, closed-form alternative to learned deformation fields for dynamic scene rendering. By restricting primitives to constant-velocity translations, the method achieves high speed and simplicity.

However, the authors explicitly acknowledge a limitation inherent to the primitive design: because each plane encodes only a single constant-velocity translation, coordinated rotation of extended objects cannot be represented by a single primitive. Instead, such motion must be approximated by assembling many short-lived primitives. The authors note that this specific class of motion accounts for the remaining quality gap between their method and the strongest deformation-based baselines. They suggest that sharing motion across primitives, rather than replacing them, is a natural direction for future extension.