🤖 AI

Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videos

The paper presents GCA (Gaussian Constitutive Alignment), a framework that learns implicit constitutive laws for dynamic 3D Gaussian Splatting from monocular videos by combining LoRA-based adaptation with rank-based depth-geometric anchors and a constitutive prior regularizer to overcome the limitations of existing methods in physical interpretability and monocular stability.

Original authors: Xiaoyang Liu, Kai Han

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiaoyang Liu, Kai Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

To understand how objects move and change shape, scientists have long tried to teach computers the hidden rules of physics. When a ball bounces, a rubber band stretches, or clay squishes under pressure, they are all following specific laws of material behavior. These laws, known as constitutive laws, describe how a substance reacts to forces. For a computer to create a realistic digital twin of a real-world object, it must learn these rules. The challenge has always been that while humans can easily guess these properties just by watching a video, computers struggle. They often get confused by the lack of depth in a flat image or fail to distinguish between different types of materials when the data is noisy. Without a clear way to learn these rules from a single camera view, digital models often look stiff or behave in ways that defy reality.

A team of researchers at the University of Hong Kong has developed a new method to solve this problem, allowing computers to learn these hidden physical laws simply by watching a video of a moving object. Their approach, called GCA, focuses on objects represented by thousands of tiny, glowing dots that form a 3D shape. The researchers start with a static 3D model of an object, created from multiple camera angles, and then feed the computer a single video of that same object moving and deforming. The goal is to teach the computer the specific rules that govern how that object bends, stretches, or squishes, without needing any other sensors or measurements.

The researchers found that previous methods often failed in this setting because they relied too heavily on matching colors or exact pixel positions, which can be misleading when lighting changes or the object rotates quickly. Instead, their new system uses two clever strategies to guide the learning process. The first strategy focuses on geometry. Rather than trying to match every single color in the video, the system looks at the relative depth of the object's surface. It checks if the order of points from near to far remains consistent, even if the exact distance is hard to tell. This allows the computer to build a stable understanding of the object's shape as it moves, ignoring the confusing noise that often plagues single-camera recordings.

The second strategy addresses the mystery of what the object is made of. Since the computer does not know if it is watching rubber, plastic, or fluid, the system tests several common physical models at once. It treats these known models not as rigid rules, but as gentle suggestions. The computer tries to fit the video data to these suggestions and sees which ones stay stable. If the true material is something unusual that doesn't fit any standard model, the system still works because it only uses these models as a guide to keep the learning process from going off track. This allows the computer to learn the specific behavior of the object without being forced into a box that doesn't fit.

In their tests, the researchers showed that this method works significantly better than existing techniques. On synthetic benchmarks where the true physics were known, their system reduced the error in predicting the object's shape by nearly half compared to the strongest previous method. The system remained robust even when the video data was sparse or noisy, a situation where other methods often failed completely. By combining a flexible learning approach with these two guiding strategies, the researchers have created a way for computers to infer the invisible rules of material science just by watching a video, bringing us closer to digital worlds that move and react with the same natural logic as our own.

=== SUMMARY ===

Technical Summary: Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videos

Problem Statement

The paper addresses the challenge of learning intrinsic physical dynamics (specifically, implicit constitutive laws) for deformable objects from a single fixed-viewpoint monocular video. While humans can intuitively infer physical properties from videos, extracting precise physical models from visual signals remains difficult.

The core difficulty lies in the ill-posed nature of the inverse problem under monocular supervision, characterized by two main ambiguities:

Geometric Ambiguity: Single-viewpoint observations suffer from scale-shift ambiguities and lack depth information, making 3D reconstruction unstable.
Material Ambiguity: The physical parameters and constitutive models governing the material behavior are unknown and underconstrained.

Existing approaches face a trade-off:

Explicit methods rely on predefined constitutive equations (e.g., hyperelastic models). While interpretable, they require manual model selection and fine-tuning for specific materials, struggle with complex real-world properties, and often fail under monocular supervision due to optimization divergence.
Implicit methods (e.g., NeuMA) use neural networks to learn constitutive relations without predefined laws. However, they are prone to converging to local minima when supervision is noisy or sparse, a problem exacerbated in monocular settings where pixel-level color supervision is unreliable due to lighting changes and self-occlusions.

Methodology: GCA Framework

The authors propose GCA (Gaussian Constitutive Alignment), a framework that unifies LoRA-based adaptation with two key alignment modules to learn implicit constitutive laws from monocular dynamic videos. The system assumes access to a static multi-view scan for initial 3D geometry reconstruction (represented by 3D Gaussians) and learns dynamics from a subsequent single-view video.

1. Neural Constitutive Laws with LoRA

The dynamical system $M_\theta$ follows elastoplastic dynamics, discretized into a state transition function. Instead of training all parameters from scratch, GCA employs Low-Rank Adaptation (LoRA) to fine-tune a pre-trained neural material model (based on NCLaw).

Base Model: A frozen base model provides fundamental physical priors (elasticity and plasticity laws).
Adaptation: Only low-rank updates ( $\Delta E_\theta$ and $\Delta P_\theta$ ) are learned. This acts as a regularizer, preventing overfitting to visual noise while maintaining parameter efficiency.

2. Rank-based Depth-Geometric Anchors (RDGA)

To address geometric ambiguity and the unreliability of pixel-level color supervision in monocular videos, RDGA establishes robust geometric constraints using scale-invariant rank-based depth alignment.

Mechanism: Instead of minimizing absolute depth errors (which are unreliable due to scale/shift ambiguity in monocular depth estimators), RDGA enforces the preservation of relative depth ordering (rank consistency) between rendered depth maps and estimated depth maps.
Process:
- Feature matching networks identify correspondence pairs between the rendered view and the ground-truth video frame.
- Local neighborhoods around these points are analyzed using Spearman rank correlation.
- Only pairs with high rank consensus in stable interior regions (excluding edges) are used as "anchors" for supervision.
Benefit: This reduces reliance on unreliable pixel-level color supervision and mitigates the scale-shift ambiguity inherent in monocular depth estimation.

3. Constitutive Prior Regularizer (CPR)

To resolve material ambiguity without sacrificing the generalization benefits of implicit modeling, CPR integrates classical constitutive models as soft differentiable priors.

Hypothesis Set: The system considers a set of candidate elastic (e.g., Corotated, StVK, Sigma) and plastic (e.g., Identity, von Mises, Drucker-Prager) models.
Parameter Stability Analysis: For each candidate model, the system attempts to solve an inverse problem to fit explicit parameters to the current state of the neural network.
Weighting: Models that yield stable parameter estimates (low variance) are assigned higher credibility weights.
Regularization: The final loss includes a weighted combination of residuals between the neural network's predictions and the candidate classical models. Crucially, this acts as a soft prior; even if the true material does not perfectly match any hypothesis, the regularization guides the optimization toward physically plausible solutions.

Key Contributions

GCA Framework: A novel approach for learning implicit constitutive laws from monocular dynamic videos, bridging the gap between the generalization of implicit methods and the stability of explicit physics.
RDGA Module: A scale-invariant geometric alignment mechanism that replaces unreliable pixel-level supervision with robust rank-based depth consensus, specifically designed to handle the ambiguities of monocular dynamic observations.
CPR Module: A soft regularization strategy that leverages classical constitutive models as differentiable priors. This resolves material ambiguity and prevents optimization divergence without forcing the model into a specific predefined law.
LoRA Integration: Demonstrating that fine-tuning via LoRA, rather than full parameter training, acts as an effective regularizer against visual noise in sparse supervision settings.

Experimental Results

The authors validate GCA across synthetic, real-to-sim, and real-world datasets:

Synthetic Benchmarks: GCA achieves a 48% lower Chamfer Distance compared to the strongest baseline on synthetic benchmarks involving six diverse material types (elastomers, gels, rubber, etc.).
Robustness: The method demonstrates superior performance on challenging real-to-sim datasets and maintains robustness under monocular supervision, whereas adapting explicit methods without the proposed framework leads to optimization divergence.
Ablation Studies: Experiments confirm that replacing the rank-based formulation with standard L1 loss degrades performance below the baseline, validating the necessity of RDGA. Similarly, full fine-tuning (without LoRA) leads to worse physical accuracy due to overfitting.

Significance and Claims

The paper claims that GCA successfully addresses the conflicting trade-offs between explicit and implicit modeling paradigms. By unifying visual-physical bidirectional alignment, the framework enables the reliable learning of intrinsic dynamics from monocular video.

Generalization: The method generalizes across diverse material types without requiring manual model selection.
Stability: It overcomes the local minima issues common in implicit methods when supervision is sparse and noisy.
Practicality: It provides a pathway for spatial intelligence applications (digital modeling, interaction, manipulation) by extracting precise physical models from standard monocular video inputs, a setting where previous methods often fail.

The authors emphasize that the core contribution is not the design of new neural architectures, but the solution to the visual-physical alignment problem, providing reliable supervisory signals to train such architectures from noisy, single-viewpoint data.