🤖 AI

FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching

FlowMaps is a latent flow matching model that learns spatio-temporal patterns of human-object interactions to predict multimodal distributions of future object locations, significantly improving robotic navigation and search performance in dynamic household environments.

Original authors: Francesco Argenziano, Miguel Saavedra-Ruiz, Sacha Morin, Charlie Gauthier, Daniele Nardi, Liam Paull

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francesco Argenziano, Miguel Saavedra-Ruiz, Sacha Morin, Charlie Gauthier, Daniele Nardi, Liam Paull

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking for your lost keys in a house where you live with a roommate. If you ask a standard robot, "Where are my keys?", it might check the table where you left them yesterday. But in a real home, things move. Your roommate might have moved the keys to the kitchen counter, or maybe the bathroom sink, depending on their daily routine.

This is the problem FlowMaps solves. It is a new type of AI designed to help robots understand that objects in a house aren't static statues; they are like actors in a play, moving around based on human habits.

Here is a simple breakdown of how it works, using some everyday analogies:

1. The Problem: The "Moving Target"

In a house, humans constantly move things. A coffee mug might be on the counter in the morning, in the dishwasher at noon, and on the dining table in the evening.

Old Robots: Act like they have amnesia. They assume if an object was in a spot yesterday, it's there today. If you ask them to find your glasses, they look in the last place they saw them and fail.
The Challenge: Robots need to guess where an object might be, not just where it was. But since humans have different habits, there isn't just one answer. The glasses could be on the nightstand, the bathroom sink, or the desk. The robot needs to consider all these possibilities at once.

2. The Solution: "FlowMaps" (The Crystal Ball)

The researchers created a system called FlowMaps. Think of it as a weather forecast for objects.

Just as a weather forecast doesn't say "It will definitely rain at 2:00 PM," but rather shows a map with a 70% chance of rain in one area and a 30% chance in another, FlowMaps doesn't guess one single spot for an object.
Instead, it creates a cloud of possibilities. It says, "Based on how humans usually behave, there is a high chance the phone is on the desk, a medium chance it's on the nightstand, and a small chance it's in the living room."

3. How It Learns: The "Pattern Detective"

How does the robot learn these habits without being told explicitly?

The Training: The team didn't teach the robot by saying, "Humans put glasses on nightstands." Instead, they used a video game simulator (ProcTHOR) to generate thousands of fake homes. They programmed "virtual humans" to live in these homes for weeks, moving objects around in realistic ways (e.g., moving a book from the shelf to the table).
The Learning: The robot watched these virtual humans move things around. It learned the rhythm of the house. It realized that even though it didn't know why the human moved the cup, the pattern of movement was consistent.
The Magic: It learned to predict the future location of an object by looking at the current scene and the object's label (e.g., "Phone"), without needing to know the specific human activity causing the move.

4. The Secret Sauce: "Flow Matching"

The technical engine behind this is called Flow Matching.

The Analogy: Imagine you have a drop of ink in a glass of water. You want to know where that ink will be after 10 seconds.
- Traditional methods might try to guess the exact path the ink takes.
- Flow Matching is like learning the current of the water. It learns the "flow" that pushes the ink from its starting point to its ending point.
In this paper, the "ink" is the object's location. The "current" is the human habit. The model learns the invisible current that moves objects through the house over time. Because it learns the flow, it can predict many different possible destinations (multimodal) rather than just one straight line.

5. The Results: Finding the Lost Phone

The researchers tested this in two ways:

In Simulation: They ran over 600 scenarios where a robot had to find an object in a virtual house. FlowMaps was much better at finding the object than previous methods. It didn't just guess one spot; it gave the robot a ranked list of the most likely spots, allowing the robot to check the best guesses first.
In the Real World: They put the system on a real robot (a TIAGo robot) in a lab. The robot had to find a cell phone that a person had moved between desks. The robot used FlowMaps to predict where the phone would be hours later. It successfully found the phone by checking the predicted spots in order.

Summary

FlowMaps is like giving a robot a "sixth sense" for human habits. Instead of just remembering where things were, it understands how things move based on routines. By predicting a cloud of likely future locations rather than a single point, it helps robots navigate messy, changing homes much more effectively, finding lost items faster and with fewer mistakes.

Key Takeaway: The paper claims this method works better than current state-of-the-art approaches for finding objects in dynamic homes, both in simulations and on a real robot, by modeling object movement as a continuous flow of possibilities rather than a fixed guess.

Technical Summary: FlowMaps

Problem Formulation

Robotic agents operating in human-inhabited environments face significant challenges due to the intrinsic dynamism of these spaces. Unlike static scenes, everyday objects are frequently moved by human interactions, causing their locations to change over time. This creates a critical difficulty for tasks such as Object Navigation (ObjNav), where a robot must locate a target object (e.g., "find my glasses") without the guarantee that it remains in its previously observed position.

The core problem addressed is the inference of the distribution over future bounding boxes of a dynamic object, $p(b_{q,\tau_f} | M_\tau, l_q, \tau_f)$ , given the current scene map $M_\tau$ , a queried object label $l_q$ , and a prediction horizon $\tau_f$ . Existing approaches often rely on discrete receptacle-level relations, assume static scenes, or depend on costly Vision-Language Models (VLMs) and hand-designed priors. Furthermore, object displacements are inherently multimodal (an object may plausibly move to multiple distinct locations) and occur in continuous 3D space, requiring a modeling approach that captures both spatial continuity and multiple plausible modes without explicit labeling of human activities.

Methodology: FlowMaps

The authors propose FlowMaps, a latent flow matching (FM) framework designed to estimate multimodal distributions over future object locations in continuous 3D space. The architecture consists of two primary modules:

Variational Autoencoder (VAE):
- Encodes object tokens (comprising a normalized 3D bounding box and a semantic label) into a compact latent space $z \in \mathbb{R}^{d_z}$ .
- The VAE is trained to reconstruct both geometry (bounding box parameters) and semantics (object class labels).
- Once trained, the encoder is frozen to provide target latents for the flow model, while the decoder maps generated latents back to object predictions.
Latent Flow Matching Network:
- Built upon a Conditional Diffusion Transformer (CDiT) architecture, this network learns a vector field $u_\theta^t$ that transports samples from a simple Gaussian base distribution $p_0$ to the target latent distribution $p_1$ (representing future object states).
- Map Encoder: Aggregates the scene context (dynamic objects and static background furniture) into a sequence of contextualized tokens using a Transformer encoder. This encoder is permutation-invariant and relies on bounding box embeddings for spatial structure.
- CDiT Blocks: A stack of CDiT blocks refines a noisy query latent by cross-attending to the scene context. The conditioning vector includes the flow time $t$ , the final timestamp $\tau_f$ , and the queried label $l_q$ .
- Training: The model is trained using Conditional Flow Matching (CFM) on a conditional optimal transport path. Target latents are derived from ground-truth future bounding boxes encoded by the frozen VAE. Source-target pairs are matched via an exact mini-batch optimal transport plan to ensure straighter trajectories.
- Inference: Given a scene and query, the map encoder computes context tokens once. The flow is then integrated from a sampled noise vector $z_0$ to a final latent $z_1$ , which is decoded into a predicted bounding box. Repeated sampling generates a multimodal distribution of plausible future locations.

Data Generation:
To train the model at scale, the authors utilize ProcTHOR to generate dynamic indoor environments. They simulate three distinct human habits (routines) to create semantically consistent object movement patterns:

Habit #1: Preference-driven (objects return to a small set of favored locations).
Habit #2: Balanced (time distributed uniformly across relevant locations).
Habit #3: Highly dynamic (frequent transitions between locations).
This process yields over 1.8 million training samples per habit, ensuring the model learns latent regularities from data rather than explicit activity labels.

Key Contributions

FlowMaps Architecture: A novel FM-based framework for representing and predicting continuous, multimodal spatio-temporal object distributions in dynamic environments, operating directly in 3D space rather than discrete receptacle categories.
Downstream Application (ObjNav): Demonstration of FlowMaps as a prior for Object Navigation in dynamic scenes. The method enables robots to reason about likely object placements over time, outperforming zero-shot baselines and trained expert policies.
Comprehensive Evaluation: An extensive quantitative and qualitative evaluation covering over 600 simulated ObjNav episodes across disjoint environments and real-world deployments. The study validates the method's soundness, effectiveness, cross-environment generalization, and practical viability.

Experimental Results

The evaluation was conducted in both simulated (ProcTHOR/AI2-THOR) and real-world (TIAGo robot) settings.

Distributional Performance: FlowMaps achieved a minFDE of 0.342m, outperforming the strongest baseline (FreqPrior at 0.370m) by 7.6%. It demonstrated superior density (0.886) and coverage (0.999) metrics compared to baselines, indicating that its samples concentrate tightly around ground-truth modes while covering the plausible space effectively. Single-point predictors (e.g., EmpiricalMean) performed significantly worse (minFDE > 2m), highlighting the necessity of multimodal modeling.
Object Navigation (ObjNav): Across three distinct habits, FlowMaps consistently achieved the highest Mean Success Rate (mSR) and Mean Success weighted by Path Length (mSPL).
- In Habit #1, FlowMaps achieved an mSR of 69.26%, surpassing the next best method (SGM at 67.26%).
- In Habit #3 (the most dynamic and challenging), FlowMaps achieved an mSR of 64.39%, significantly outperforming graph-based methods (HOMER: 58.93%, SGM: 56.84%) and zero-shot LLM baselines.
- Notably, FlowMaps' SR@1 (success on the first prediction) often exceeded the SR@5 of competing baselines, indicating high precision in its top-ranked predictions.
Real-World Deployment: The method was successfully deployed on a TIAGo robot to locate a "CellPhone" in a real-world lab setting, demonstrating the ability to transfer learned priors to physical environments.

Significance and Claims

The paper claims that modeling object dynamics through continuous, multimodal spatio-temporal distributions significantly improves robotic search and navigation in changing household environments. Key assertions include:

Implicit Learning: Predicting object locations does not require explicitly identifying the human activity causing the displacement; the structure emerges directly from data.
Generalization: The learned prior generalizes to unseen household layouts and object arrangements, provided they share similar object routines (habits), without requiring per-household fine-tuning.
Efficiency: By avoiding explicit scene graph construction or heavy VLM reasoning during inference, FlowMaps remains efficient while outperforming prior dynamic ObjNav methods.
Modality: The approach effectively handles the inherent multimodality of object displacements, providing a richer prior for robotic reasoning than static scene assumptions or discrete state predictions.

The authors acknowledge limitations, including the current reliance on a closed set of object classes and receptacles, the need for substantial training data (mitigated by simulation), and the evaluation being limited to three specific habit types. Future work aims to address open-vocabulary settings and improve sample efficiency for online fine-tuning.