🤖 AI

Liquid Neural Networks as a Drop-in Continuous-Time Deformation Field for Dynamic 3D Gaussian Splatting

This paper proposes a Liquid Neural Network-based deformation field using Closed-form Continuous-time cells to replace the standard MLP in Dynamic 3D Gaussian Splatting, thereby enforcing explicit temporal smoothness and improving reconstruction quality for high-frequency articulated motion without requiring numerical solvers.

Original authors: Mingzhao Li, Arghya Pal, Guan Yuan Tan

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mingzhao Li, Arghya Pal, Guan Yuan Tan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Fixing the "Stop-Motion" Glitch

Imagine you are trying to recreate a moving 3D scene (like a person dancing or a ball bouncing) using a computer. You have a video of it, and you want the computer to build a 3D model that looks perfect from any angle.

The current best method, called 3D Gaussian Splatting, works like a cloud of millions of tiny, fuzzy balloons (Gaussians) that float in space. To make them move, the computer uses a "deformation field"—a smart calculator (an MLP) that tells each balloon where to go at any specific moment in time.

The Problem:
Think of the current calculator like a stop-motion animator. It looks at Frame 1, calculates where the balloons go. Then it looks at Frame 2, calculates again. Then Frame 3. It treats every single moment as a completely separate puzzle.

The Glitch: Because it doesn't "know" that Frame 2 is just a tiny step after Frame 1, the movement can look jerky or jittery. It relies on the computer to magically figure out that the motion should be smooth, which doesn't always work perfectly, especially with fast or wiggly movements.

The Solution: The "Liquid" Calculator

The authors of this paper replaced that stop-motion calculator with something they call a Liquid Neural Network (LNN).

The Analogy: The River vs. The Staircase

The Old Way (MLP): Imagine a staircase. You can only stand on specific steps (Frame 1, Frame 2, Frame 3). If you want to be between steps, you have to guess. It's rigid and discrete.
The New Way (Liquid Network): Imagine a flowing river. The water moves continuously. There are no steps; it's a smooth, unbroken flow.

The new "Liquid" calculator is designed to understand time as a continuous flow, not a series of separate snapshots. It doesn't just guess the next step; it understands the physics of the movement between steps.

How It Works: The "Time Gate"

The secret sauce is a specific type of math cell called a Closed-form Continuous-time (CfC) cell.

Think of this cell as a smart traffic light for the data:

The Input: The calculator receives the position of a balloon and the current time.
The Gate: Inside the cell, there is a "time gate" (a mathematical switch). This gate opens and closes smoothly based on how much time has passed.
The Blend: Instead of jumping to a new position, the cell smoothly blends the balloon's current state with a new potential state. It's like a dancer gliding from one pose to another rather than snapping between them.

Why is this special?
Usually, to make a computer understand continuous time, you need a heavy, slow engine (called a "solver") that does complex math at every single step. This is like driving a tank to go to the grocery store.

The Paper's Trick: They found a "closed-form" solution. This means they figured out a shortcut formula that gives the same smooth, continuous result as the heavy engine, but it runs just as fast as a simple calculator. It's like finding a secret tunnel that lets you drive a sports car instead of a tank, with no traffic jams.

What Did They Find?

The team tested this new "Liquid" system on two types of scenes:

Synthetic Scenes (D-NeRF): Computer-generated animations of things like a "Hell Warrior" swinging a sword or a "Bouncing Ball."
Real-World Scenes (NeRF-DS): Videos of real objects like cups, sieves, and bells moving.

The Results:

Smoother Motion: On scenes with fast, wiggly, or high-frequency movement (like the swinging sword), the new system was significantly better. It stopped the "smearing" or "ghosting" where objects look blurry because the movement was jerky.
No Extra Cost: The new system didn't require more computer power or take longer to train than the old method. In fact, they could make it smaller and faster while still getting better results.
Robustness: It handled "noisy" real-world videos better, keeping the shapes crisp even when the camera was shaky or the lighting was tricky.

The Bottom Line

The authors didn't change the whole 3D reconstruction process; they just swapped out the "engine" that controls the movement.

By switching from a discrete, step-by-step calculator to a smooth, liquid-time calculator, they made dynamic 3D scenes look more natural and less jerky, without making the computer work any harder. It's a "near-zero friction" upgrade that turns a choppy stop-motion animation into a fluid, continuous movie.

Technical Summary: Liquid Neural Networks as a Drop-in Continuous-Time Deformation Field for Dynamic 3D Gaussian Splatting

Problem Statement
Dynamic 3D Gaussian Splatting (D-3DGS) reconstructs moving scenes from monocular video by deforming a canonical set of 3D Gaussians using a Multi-Layer Perceptron (MLP) conditioned on frame time $t$ . While $t$ is a continuous physical variable, the standard D-3DGS deformation MLP treats it as a discrete input. The architecture lacks temporal coupling between adjacent timesteps; it is fitted independently for each frame in the training set, and temporal smoothness emerges only as a byproduct of optimization rather than being structurally enforced. Existing continuous-time alternatives, such as Neural ODEs or Latent-ODE variants (e.g., ODE-GS), introduce explicit continuous-time priors but incur significant computational costs due to the requirement of numerical solvers in the training loop. The authors identify a gap for an architectural design that provides continuous-time semantics without the overhead of numerical integration.

Methodology
The paper proposes redesigning the deformation field $F_\theta$ of D-3DGS as a stack of Closed-form Continuous-time (CfC) cells, a type of Liquid Neural Network (LNN). This design preserves the remainder of the D-3DGS pipeline (canonical Gaussians, rasterizer, loss functions, and training schedules) while replacing only the deformation network.

CfC Cell Structure: Each cell is the closed-form solution of a Liquid Time-constant (LTC) Ordinary Differential Equation (ODE). Unlike standard MLPs, a CfC cell computes an updated hidden state $h'$ via a sigmoidal time gate $\sigma_\tau = \sigma(W_{a}z \cdot \tau + W_{b}z)$ , where $\tau$ is the elapsed time signal. This gate interpolates between two candidate hidden states ( $g$ and $h_{cand}$ ), effectively baking a learned, smooth response to time variation directly into the loss landscape without invoking a numerical solver.
Depth-as-Time: The authors instantiate the deformation field as a stack of $D$ CfC cells. Crucially, they do not use the cells in a recurrent manner across video frames. Instead, the network depth acts as the temporal recurrence mechanism. For every forward pass, the hidden state is reset to zero, and the frame time $t$ is fed as the elapsed-time signal to every cell in the stack. This "depth-as-time" approach allows the network to function as a feed-forward block with continuous-time semantics.
Implementation: The implementation is a drop-in replacement for the D-3DGS deformation module. The authors use a backbone width of 64, hidden size of 128, and 6 CfC cells (default configuration), matching the positional encoding schemes of the original D-3DGS.

Key Contributions

Liquid Deformation Field: The introduction of a deformation field for D-3DGS built from a stack of CfC cells that exposes an explicit, time-gated update over frame time $t$ . This design maintains the exact same inference cost profile as a standard feed-forward MLP.
Empirical Evaluation: Comprehensive testing on eight synthetic D-NeRF scenes and seven real-world NeRF-DS scenes. The study reports PSNR, SSIM, LPIPS, parameter counts, and MACs (Multiply-Accumulate operations), demonstrating that the liquid field matches or exceeds the MLP baseline.
Architectural Positioning: A clear delineation of where CfC fits in the spectrum of continuous-time methods. The paper positions CfC as the "cheap end" of the spectrum: it offers continuous-time semantics and robustness to jitter without the computational penalty of ODE/SDE solvers required by methods like ODE-GS.

Results

Synthetic Scenes (D-NeRF): The liquid deformation field matches or exceeds the retrained D-3DGS MLP baseline on 6 out of 8 scenes. The aggregate mean PSNR is nearly identical (38.25 dB for CfC vs. 38.26 dB for MLP). The most significant gains were observed in scenes with high-frequency articulated motion (e.g., Hook +1.49 dB, Hell Warrior +0.82 dB), where the time-gated nonlinearity acts as an implicit smoothness prior.
Real-World Scenes (NeRF-DS): On seven real-world scenes dominated by pose noise, the CfC approach outperformed the MLP baseline across all mean metrics (PSNR: 23.86 dB vs. 23.39 dB; SSIM: 0.8491 vs. 0.8403; LPIPS: 0.1891 vs. 0.2011). The largest improvement occurred on the As scene, which features heavy specular motion, where the CfC method achieved a +2.74 dB PSNR gain.
Compute Efficiency: When configured with matched depth and width to the baseline MLP ( $D=8, W=256$ ), the CfC stack incurred a ~50% overhead in parameters and MACs. However, the authors' default configuration ( $D=6, W=128$ ) actually reduced the computational footprint to 36% below the MLP baseline while maintaining superior or comparable quality.
Ablation: Experiments on the Hell Warrior scene confirmed that the CfC cell architecture outperforms the standard MLP backbone. The optimal depth was found to be $D=6$ , and GELU activation yielded the best results, suggesting that unbounded smooth backbones compose better with the cell-level nonlinearity.

Significance and Claims
The paper claims to present a "near-zero-friction architectural design" that transforms the discrete MLP deformation field of D-3DGS into an explicit continuous-time function of $t$ . The significance lies in achieving continuous-time semantics and robustness to noisy supervision (such as pose drift and motion blur) without the computational burden of numerical solvers.

The authors modestly frame their contribution as an architectural improvement rather than a fundamental shift in the reconstruction paradigm. They acknowledge that the "depth-as-time" approach does not utilize the long-horizon temporal memory typical of recurrent LNNs, as the per-Gaussian set is non-stationary. Consequently, the continuity guarantee is derived solely from the time-gated nonlinearity. The paper concludes that while ODE/SDE-based methods may still be superior for extrapolation or heavily corrupted data, the CfC approach offers the most economical choice for the standard interpolation regime in dynamic 3D scene reconstruction, allowing practitioners to adopt continuous-time priors incrementally.