💻 computer science

DisFlow: Scene Flow from Distance Field for Object Pose, Velocity Tracking, and Dynamic Object Reconstruction

DisFlow is a novel real-time framework that leverages Gaussian Process Implicit Surfaces to estimate scene flow from distance fields, enabling simultaneous 6DoF dynamic object pose estimation, motion tracking, and high-quality surface reconstruction through probabilistic fusion in the object frame.

Original authors: Lan Wu, Sheila Sutjipto, Jennifer Wakulicz, Teresa Vidal-Calleja

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lan Wu, Sheila Sutjipto, Jennifer Wakulicz, Teresa Vidal-Calleja

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to catch a flying ball. To do this safely, the robot needs to know three things at once: where the ball is, how fast it's moving, and what the ball actually looks like in 3D space.

Most current robot "eyes" are good at one or two of these, but struggle to do all three together in real-time. Some are great at guessing the position but forget what the object looks like. Others build a perfect 3D model of the object but get confused when it starts moving fast.

DisFlow is a new "brain" for robots that solves this by doing everything at once. Here is how it works, using simple analogies:

1. The "Object-Centric" Backpack

Imagine you are walking through a crowded room while holding a backpack. If you try to map the room from your own perspective, the people and furniture seem to be zooming past you in a chaotic blur. It's hard to keep track of who is who.

DisFlow changes the perspective. Instead of mapping the world from the camera's point of view, it puts the object inside a "virtual backpack" (the object frame).

The Analogy: Imagine the robot is wearing a backpack that is glued to the moving object. Even if the object spins, flies, or rolls, the backpack moves with it. Inside the backpack, the object never moves; it stays perfectly still.
The Benefit: Because the object is "still" inside this virtual backpack, the robot can build a perfect, stable 3D map of it without getting confused by the motion. It doesn't have to constantly delete or fix parts of the map because the object isn't "moving" relative to the map.

2. The "Invisible Rubber Sheet" (The Distance Field)

To understand the shape of the object, DisFlow doesn't just take a photo; it creates an invisible "rubber sheet" or a distance field around the object.

The Analogy: Think of the object as a statue. DisFlow wraps it in an invisible, stretchy skin. If you poke a finger into this skin, the system knows exactly how deep you are and which way is "up" (the surface normal).
The Magic: This isn't just a static skin. Because the robot knows how this "skin" changes from one millisecond to the next, it can calculate flow. It's like watching a river: by seeing how the water ripples and moves, you can tell exactly how fast the current is flowing and in what direction. DisFlow uses these ripples to figure out the object's speed and rotation instantly.

3. The "Smart Puzzle" (Probabilistic Fusion)

As the robot watches the object, it gets new pieces of the puzzle every fraction of a second.

The Analogy: Imagine you are assembling a 3D puzzle, but some pieces are blurry or missing. DisFlow doesn't just force the pieces together; it keeps a "confidence score" for every piece.
The Result: If a part of the object is hidden or blurry, the system knows, "I'm not 100% sure about this spot." If the view is clear, it says, "I'm very confident here." This allows the robot to know not just where the object is, but how sure it is about that location. This is crucial for safety—if the robot isn't sure, it can be more careful.

What Did They Prove?

The authors tested DisFlow on moving objects (like a cracker box and a mustard bottle) and compared it to other top methods.

Tracking: It tracked the position and speed of the objects much more accurately than previous methods, even when the objects were spinning or had smooth, boring shapes (like a bottle) that usually confuse robots.
Reconstruction: It built a 3D model of the object that was smoother and more accurate than standard methods.
Speed: It did all of this in real-time, meaning it can keep up with fast-moving objects without lagging.

The Bottom Line

DisFlow is like giving a robot a pair of glasses that not only see an object but also "feel" its movement and "know" its shape simultaneously. By keeping the object in a stable "backpack" and watching how the invisible "skin" around it ripples, the robot can predict where the object is going and what it looks like, all while knowing how confident it is in its own vision.

Note: The paper focuses strictly on the technical performance of tracking, velocity estimation, and 3D reconstruction. It does not claim to solve specific medical problems or clinical applications, but rather provides a foundational tool for general robotic perception.

Technical Summary: DisFlow – Scene Flow from Distance Field

Problem Statement

Estimating the pose and motion of dynamic objects is a fundamental challenge in robotic perception, essential for tasks such as manipulation, grasping, and navigation. While classical Simultaneous Localization and Mapping (SLAM) and reconstruction methods excel in static environments, they struggle with dynamic objects due to the difficulty of jointly reasoning about object motion and surface geometry from partial, noisy observations.

Existing approaches face specific limitations:

Tracking-only methods (e.g., optical flow-based or learning-based pipelines) often provide pose and velocity estimates but fail to maintain dense, temporally consistent surface reconstructions, limiting their utility for contact reasoning.
Reconstruction-only methods (e.g., TSDF fusion) build explicit surfaces but are typically designed for static scenes. They often lack probabilistic representations and uncertainty estimates, which are critical for safety and planning.
Learning-based methods rely on large-scale training data and specific object categories, limiting generalization to unseen objects.

There is a gap for a framework that simultaneously delivers real-time 6DoF pose tracking, velocity estimation, and dense surface reconstruction with uncertainty quantification, specifically for dynamic objects.

Methodology: The DisFlow Framework

DisFlow addresses this gap by estimating scene flow directly from a distance field, enabling online 6DoF pose and velocity estimation alongside probabilistic surface reconstruction. The framework operates in an object-centric reference frame, ensuring geometric consistency over time.

1. Implicit Surface Representation (GPIS)

The object geometry is represented using Gaussian Process Implicit Surfaces (GPIS). Unlike discrete volumetric grids (e.g., TSDFs), GPIS provides a continuous, differentiable, and probabilistic model.

Constraints: The model is trained using both surface points (zero level set) and surface normals (gradient constraints).
Prediction: For any query point, GPIS predicts the signed distance and its gradient, along with a variance (uncertainty) estimate.
Scalability: To enable real-time operation, the 3D domain is partitioned using an octree. Each node maintains a local GP trained on nearby samples, allowing for efficient incremental updates as new frames arrive.

2. Scene Flow Formulation (DisFlow)

The core innovation is the computation of DisFlow, which describes how surface points are transported over time as the object moves.

Transport Equation: The method utilizes the transport equation $\frac{\partial d(x, t)}{\partial t} + v(x, t)\nabla d(x, t) = 0$ , where $d$ is the signed distance function and $v$ is the scene flow.
Mechanism: By querying the GPIS representation for distance and gradients at high frame rates, the system derives a continuous flow field. This flow establishes implicit point-to-point correspondences between consecutive observations without requiring explicit feature matching.

3. Pose and Velocity Estimation

Pose estimation is formulated as an optimization problem that aligns the observed point cloud (transformed into the object frame) with the implicit surface.

Objective: Minimize the sum of squared signed distances ( $E = \sum d(x_j)^2$ ).
Optimization: The problem is linearized around the current pose estimate to solve for a twist increment $\xi$ (translation and rotation) in closed form. The gradient $\nabla d(x)$ acts as the correspondence direction.
Regularization: To ensure robustness against noise and partial observations, a temporal regularization term enforces smoothness between consecutive frames, penalizing deviations from the predicted velocity.

4. Object-Centric Fusion

Unlike traditional methods that fuse data in the camera or world frame (requiring complex dynamic point removal), DisFlow fuses observations directly into the object frame.

Process: Once the pose $T^k_{oc}$ is estimated, new points are transformed into the object frame and added to the corresponding octree clusters.
Benefit: In the object frame, the surface is inherently static. This simplifies the pipeline, eliminates the need for dynamic point filtering, and ensures the geometry remains temporally consistent even during rapid motion.

Key Contributions

DisFlow Framework: A novel system that leverages instantaneous distance fields to recover scene flow, enabling real-time 6DoF pose and linear/angular velocity estimation coupled with probabilistic surface reconstruction.
Object-Centric Strategy: A formulation that naturally handles dynamic objects by maintaining the map in the object frame, ensuring geometric consistency and simplifying the fusion process.
Unified Output: The system simultaneously delivers real-time pose trajectories, object velocities, dense surface reconstructions, and uncertainty estimates, outperforming baselines that focus solely on tracking or reconstruction.

Experimental Results

The framework was evaluated on the Fast-YCB dataset and real-world sequences involving rotating humans and flying objects.

Quantitative Evaluation

Pose Tracking: DisFlow significantly outperformed baselines (DOPE, PoseRBPF, TrackNet, ROFT) on the Fast-YCB dataset.
- For the 003 cracker box, it achieved an ADD-AUC of 92.44% (vs. 78.50% for ROFT) and a translation RMSE of 0.790 cm.
- For the challenging 006 mustard bottle (smooth, symmetric), it achieved an ADD-AUC of 94.83% and translation RMSE of 0.498 cm, demonstrating robustness on objects with weak geometric constraints.
Velocity Tracking: Compared directly with ROFT (the only baseline providing velocity), DisFlow reduced linear velocity RMSE by more than half (e.g., from 5.237 cm/s to 2.311 cm/s for the cracker box) and significantly improved angular velocity accuracy.
Surface Reconstruction: Using Chamfer distance and Hausdorff distance metrics, DisFlow produced more accurate meshes than TSDF-based fusion.
- Chamfer distance improved from 0.0059 (TSDF) to 0.0045.
- F-scores at fine thresholds (1mm and 2mm) were substantially higher, and normal consistency was improved.

Qualitative Evaluation

Consistency: In sequences involving 360° human rotation and rapid "flying doughnut" motions, the reconstructed object remained a seamless, fixed point cloud in the object frame, with no visible drift or misalignment upon returning to the start.
Uncertainty Visualization: The probabilistic nature of GPIS allowed for the visualization of uncertainty on the mesh, highlighting regions with limited observations, which is valuable for active sensing.

Significance and Limitations

The paper claims that DisFlow bridges the gap between tracking-oriented and reconstruction-oriented methods. Its significance lies in providing a unified, probabilistic framework that delivers dense geometry, motion estimates, and uncertainty simultaneously in real-time. This is particularly valuable for downstream robotic tasks requiring safe interaction, such as grasp planning and collision checking.

Limitations:
The authors note that the method relies on geometric constraints. For objects that are nearly flat, possess minimal structure, or exhibit high symmetry (e.g., cans or bottles), the constraints may be weak, potentially leading to ambiguous optimization solutions. The current work focuses on a geometry-only formulation, leaving the integration of RGB image features for future work.