💻 computer science

Grow with the Flow: 4D Reconstruction of Growing Plants with Gaussian Flow Fields

The paper introduces GrowFlow, a novel 4D reconstruction method that models plant growth as a continuous flow field using neural ordinary differential equations coupled with 3D Gaussian primitives to achieve temporally consistent and physically plausible rendering of expanding vegetation.

Original authors: Weihan Luo, Lily Goli, Sherwin Bahmani, Felix Taubner, Andrea Tagliasacchi, David B. Lindell

Published 2026-03-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Weihan Luo, Lily Goli, Sherwin Bahmani, Felix Taubner, Andrea Tagliasacchi, David B. Lindell

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to film a magic trick where a tiny seed instantly transforms into a giant, blooming rose. Now, imagine you want to create a computer model that doesn't just show the final flower, but can rewind the movie to show exactly how every single petal and leaf grew, moved, and changed shape, frame by frame.

That is the challenge the paper "Grow with the Flow" tackles. The authors, a team from the University of Toronto and Vector Institute, have built a new system called GrowFlow.

Here is a simple breakdown of how it works, using some everyday analogies.

The Problem: Why Existing Methods Fail

Current computer vision tools are great at handling things that move, like a person walking or a car driving. They treat the scene like a rubber sheet. If a person raises their hand, the rubber sheet stretches to match.

But plants don't just stretch; they grow. New leaves appear, branches split, and the plant gets bigger.

The "Rubber Sheet" Problem: If you try to stretch a rubber sheet to make a new leaf appear out of thin air, the sheet tears or looks weird.
The "Stop-Motion" Problem: Other methods treat every second of the video as a separate, static picture. They don't realize that the leaf at 1:00 PM is the same leaf as the one at 1:05 PM, just bigger. This breaks the "story" of the growth.

The Solution: GrowFlow

GrowFlow changes the rules. Instead of stretching a rubber sheet or taking separate photos, it treats the plant like a river.

1. The River Analogy (Continuous Flow)

Imagine the plant is a river flowing downstream.

The Water: The water represents the "Gaussian particles" (tiny 3D dots that make up the image).
The Current: The river's current represents the growth.
In a river, the water flows smoothly. It doesn't teleport. A drop of water at the top of the waterfall is the same drop that hits the bottom, just moving along a smooth path.

GrowFlow uses a Neural ODE (a fancy math tool) to map this "current." It learns the rules of the flow so that if you ask, "Where was this specific leaf 10 minutes ago?" the system can trace it back smoothly along the river's path.

2. The "Reverse Movie" Trick

Here is the clever part. Plants grow forward (small to big), which is hard to model because new stuff keeps appearing.

The Trick: GrowFlow starts with the fully grown plant (the big, mature flower) and learns to shrink it backward in time.
Why? It's much easier to teach a computer to make a big object disappear smoothly than to teach it to make a new object appear out of nowhere.
The Result: Once the computer learns how to shrink the plant perfectly, the team simply hits "Play" in reverse. Now, they have a perfect, smooth video of the plant growing from a seed to a flower, with every single particle tracking its own history.

3. The "Ghost" Problem Solved

In older methods, when a leaf grows, the computer often gets confused. It might think the leaf at 1:00 PM is a totally different object than the leaf at 1:05 PM. This causes "ghosting" or floating artifacts in the video.

GrowFlow's Fix: Because every single 3D dot (Gaussian) is part of the same continuous flow, the system knows that the dot at the tip of the leaf is the same dot that was near the stem earlier. It keeps the identity of every part of the plant intact, like a loyal GPS tracker that never loses the signal.

Why This Matters

This isn't just about making pretty videos.

For Farmers: It allows them to measure exactly how fast a plant is growing, how big its leaves are, and how it reacts to sunlight, without touching the plant.
For Science: It gives researchers a way to study the "life story" of a plant in 3D, seeing exactly how it developed over time.

Summary

Think of GrowFlow as a time-traveling camera that understands the physics of growth. Instead of just taking snapshots, it learns the "flow" of the plant's life, allowing us to watch a plant grow, shrink, and evolve with perfect smoothness and accuracy, solving the puzzle of how new things appear in a 3D world.

1. Problem Statement

The paper addresses the challenge of reconstructing the time-varying 3D appearance of growing plants from multi-view timelapse imagery. Unlike standard dynamic scenes (e.g., moving objects), plant growth involves:

Continuous Topological Changes: New geometry (leaves, branches) emerges gradually, altering the scene's structure and topology over time.
Limitations of Existing Methods:
- Deformation-based methods: Map a canonical representation to each timestep but lack constraints on physical plausibility, often leading to geometrically impossible mappings.
- 4D Gaussian Splatting (4DGS): Typically treats different timesteps as independent sets of Gaussians or uses temporal masking. This breaks temporal correspondence, meaning the same physical leaf cannot be tracked consistently across time, and new geometry is introduced abruptly rather than continuously.

The core goal is to create a representation that offers temporally coherent geometry, allowing for the tracking of individual plant parts and physically plausible growth dynamics.

2. Methodology: GrowFlow

The authors propose GrowFlow, a dynamic scene representation that couples 3D Gaussian Splatting (3DGS) with Neural Ordinary Differential Equations (Neural ODEs).

Core Concept: Reverse Growth & Continuous Flow

Instead of modeling growth forward (which requires non-differentiable particle addition), GrowFlow models the process in reverse:

Reverse Growth Formulation: The system starts with the mature plant (at time $T$ ) and learns a continuous flow field that "shrinks" the plant backward to its initial state ( $t=0$ ).
Differentiability: By modeling the disappearance/shrinking of existing Gaussians (scaling down to zero or occlusion) rather than adding new ones, the process remains fully differentiable, enabling end-to-end gradient-based optimization.
Forward Synthesis: Once the reverse flow is learned, it is integrated forward in time to reconstruct realistic growth trajectories.

Technical Components

3D Gaussian Primitives: The scene is represented by a set of Gaussians $G_i$ $G_{i}$ , parameterized by center ( $\mu$ $μ$ ), rotation ( $q$ $q$ ), scale ( $s$ $s$ ), opacity ( $o$ $o$ ), and color ( $c$ $c$ ).
- Time-Varying Parameters: Position, rotation, and scale evolve over time.
- Time-Invariant Parameters: Opacity and color are kept fixed (assuming controlled lighting), ensuring that the identity of a Gaussian persists throughout the trajectory.
Neural ODE Flow Field:
- The evolution of geometric parameters is governed by a velocity field $F_\phi$ defined as a Neural ODE.
- $\dot{\theta}_i(t) = F_\phi(\theta_i(t), t)$ , where $\theta$ represents geometric parameters.
- Integration of this velocity field ensures $C^1$ -continuous trajectories, guaranteeing smooth, physically plausible motion without sudden jumps.
Architecture (HexPlane + MLP):
- The velocity field is parameterized using a HexPlane encoder (interpolating features from 6 spatio-temporal planes) followed by Multi-Layer Perceptron (MLP) decoders. This provides a strong inductive bias for capturing spatial and temporal variations.
Training Strategy:
1. Static Reconstruction: Optimize a standard 3DGS model on the fully grown plant (time $T$ ).
2. Boundary Reconstruction: To avoid numerical instability in long-range ODE integration, the model trains progressively. It integrates backward from $T$ to $T-1$ , caches the state, and uses it as the initial condition for the next step.
3. Global Optimization: After caching boundary states, the model performs global optimization by randomly sampling timesteps and integrating the flow field to enforce consistency across the entire sequence.

3. Key Contributions

GrowFlow Representation: The first dynamic representation that couples 3D Gaussians with Neural ODEs to model continuous, non-rigid plant growth, ensuring temporal correspondence for every primitive.
Reverse-Growth Formulation: A novel optimization strategy that sidesteps the non-differentiable problem of topology changes (adding new matter) by learning a continuous shrinkage process and reversing it.
New Dataset: The authors introduce the first multi-view timelapse dataset of real growing plants, featuring three species (blooming flower, corn, paperwhite) captured with a calibrated turntable system.
State-of-the-Art Performance: The method achieves superior geometric coherence and image quality compared to existing dynamic scene methods.

4. Experimental Results

The method was evaluated on both synthetic data (Blender) and the captured real-world dataset.

Metrics:
- Geometric Accuracy: Measured via Chamfer Distance (CD) against ground-truth meshes (synthetic only).
- Photometric Quality: PSNR, SSIM, and LPIPS.
Performance:
- Synthetic Data: GrowFlow significantly outperformed baselines (Dynamic 3DGS, 4D-GS, 4DGS) in both training and interpolated (unseen) timesteps. It achieved a CD of 0.10 vs. >0.77 for baselines, demonstrating superior geometric fidelity.
- Real Data: While baselines struggled with interpolation (oscillating between growing/shrinking or turning Gaussians black), GrowFlow produced smooth, physically plausible growth trajectories.
- Temporal Consistency: Visualizations of temporal slices showed that GrowFlow maintained alignment with ground truth, whereas baselines exhibited geometric drift and structural distortions.
Ablation Studies:
- Removing the HexPlane encoder (replacing with MLP) degraded image fidelity and temporal stability.
- Removing the Boundary Reconstruction step caused severe geometric drift and failure to converge due to accumulated numerical errors in long-range integration.

5. Significance and Future Work

Scientific Impact: GrowFlow provides the first temporally coherent representation for modeling growing 3D structures, enabling accurate tracking of individual leaves and branches over time. This is crucial for plant phenotyping, agriculture, and biological research.
Limitations: The current formulation assumes monotonic growth (structure only accumulates). It does not yet handle structural loss (e.g., leaf senescence, petal drop).
Future Directions: Extending the framework to non-monotonic growth, coupling with trait extraction modules for automated measurement (stem length, branching angles), and applying the continuous flow concept to other emerging geometries (e.g., crystals, embryos).

In summary, GrowFlow bridges the gap between high-fidelity rendering and physically plausible dynamic modeling by treating plant growth as a continuous flow field, solving the critical issue of temporal correspondence in 4D reconstruction.