🤖 AI

CoFL: Continuous Flow Fields for Language-Conditioned Navigation

CoFL is an end-to-end navigation policy that generates continuous flow fields from bird's-eye view observations and language instructions, enabling smooth, reactive, and high-performing zero-shot navigation in unseen environments by replacing brittle modular pipelines with a scalable, trajectory-integrated approach trained on a large-scale dataset.

Original authors: Haokun Liu, Zhaoqi Ma, Yicheng Chen, Masaki Kitagawa, Wentao Zhang, Jinjie Li, Moju Zhao

Published 2026-03-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haokun Liu, Zhaoqi Ma, Yicheng Chen, Masaki Kitagawa, Wentao Zhang, Jinjie Li, Moju Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to walk through a crowded room to find a specific chair, but you can only give it instructions in English.

Most current robots are like over-caffeinated tourists with a broken GPS. They try to figure out the room, then the map, then the path, then the steps, all separately. If they misread a sign (the "perception" step), they get lost forever. Others are like blindfolded dancers who memorize a specific sequence of 100 steps to get from point A to point B. If they stumble on step 10, they can't recover; they just keep dancing into the wall.

CoFL (Continuous Flow Fields) is the robot's new superpower: It stops thinking in steps and starts thinking in currents.

Here is how it works, using simple analogies:

1. The "River" Analogy (The Core Idea)

Instead of telling the robot, "Take 5 steps forward, turn left, take 2 steps," CoFL paints the entire room with an invisible river of wind.

The Map: Imagine the floor is a giant canvas.
The Instruction: You say, "Go to the sofa."
The Magic: CoFL instantly draws a river of arrows flowing everywhere in the room, all pointing toward the sofa.
- Near the sofa, the arrows swirl gently to help you park.
- Near a wall, the arrows curve away to push you around the obstacle.
- In the middle of the room, they flow straight and smooth.

The robot doesn't need to plan a path. It just needs to drop a leaf (the robot) into this river. The river naturally carries the leaf to the sofa, smoothly avoiding rocks (obstacles) along the way. If the robot gets pushed off course, it just drifts back into the current and keeps going.

2. The "Weather Forecast" vs. "The Itinerary"

Old Robots (The Itinerary): They try to write a strict itinerary: "Step 1: Walk 2 meters. Step 2: Turn 45 degrees." If the floor is slippery or a chair moves, the itinerary breaks.
CoFL (The Weather Forecast): It predicts the "wind" for the whole room right now. It doesn't care about the specific steps; it just knows the direction and speed needed at every single spot on the floor. This makes it incredibly flexible. If you ask it to start from a different spot, the "river" is already there waiting for it.

3. How They Taught the Robot (The "Video Game" Training)

To teach this robot, the researchers didn't just show it a few photos. They built a massive video game simulator with over 500,000 levels (using real 3D scans of houses and offices).

The Training: They didn't just show the robot "Go to the chair." They generated a "perfect river" for every single room in the game.
The Lesson: The robot learned to look at a room (from a bird's-eye view, like a drone looking down) and instantly predict: "If I am here, the wind should blow this way. If I am there, it should blow that way."

4. The "Zero-Shot" Superpower

The coolest part? The robot was trained only in the video game. It never saw a real robot before.

When they put it in a real room with real furniture and real obstacles, it worked perfectly immediately. It didn't need to "re-learn" or "fine-tune." It was like a video game character stepping out of the screen and walking into your living room, knowing exactly how to dodge your coffee table without ever having seen it before.

Why is this a big deal?

Safety: Because it sees the "currents" around obstacles, it rarely crashes. It flows around things like water.
Smoothness: It doesn't jerk or stop-and-go. It moves like a fluid.
Speed: It calculates the path instantly. It doesn't need to think hard about every single step; it just follows the flow.

In summary: CoFL turns navigation from a rigid set of instructions into a natural, flowing movement, allowing robots to navigate complex rooms as easily as a leaf floating down a stream.

1. Problem Statement

Language-conditioned navigation (LCN) aims to translate high-level semantic instructions (e.g., "go to the sofa") into low-level, continuous motion in complex environments. Existing approaches suffer from two primary limitations:

Modular Pipelines: Methods that stitch together separate perception, grounding, and planning modules (e.g., VLMs + geometric planners) are brittle. Small errors in upstream components (like object detection or semantic grounding) cascade into catastrophic navigation failures.
Discrete or Generative End-to-End Policies: Current end-to-end models often predict discrete action tokens (limiting motion smoothness) or generate action chunks via iterative denoising (e.g., Diffusion Policies). The latter introduces high latency due to multi-step sampling and lacks an explicit geometric representation of the workspace, leading to trajectories that may be plausible in endpoint but unsafe (colliding with obstacles) during execution.

The paper proposes a paradigm shift: instead of predicting a sequence of actions or waypoints, the policy should predict a continuous flow field over the entire workspace. This allows for smooth, reactive, and geometrically consistent navigation from any starting point.

2. Methodology: CoFL

CoFL (Continuous Flow Fields) is a transformer-based, end-to-end policy that maps a Bird's-Eye View (BEV) observation and a language instruction to a continuous vector field.

A. Core Formulation

Instead of outputting a trajectory $\tau$ , CoFL learns a navigation field $v(x | I, \ell)$ that maps any 2D location $x$ in the workspace to an instantaneous velocity vector.

Input: A BEV RGB image $I$ and a natural language instruction $\ell$ .
Output: A continuous flow field where querying any point $x$ yields a velocity vector.
Trajectory Generation: Trajectories are generated by numerically integrating the predicted field (Euler integration) starting from the robot's current position $x_0$ . This ensures the motion is inherently smooth and reactive to the current state.

B. Network Architecture

The model consists of two main modules (Fig. 2):

Vision-Language Encoder:
- Uses a frozen SigLIP 2 backbone to extract visual tokens from the BEV image and text tokens from the instruction.
- Fuses these tokens using a transformer decoder to create a task-relevant context representation $C$ that understands the environment relative to the instruction.
CoFL Decoder:
- A query-based transformer decoder. It takes 2D normalized spatial coordinates $X$ as queries.
- These queries attend to the context tokens $C$ .
- Output Heads: Instead of regressing raw $(v_x, v_y)$ , the decoder predicts a non-negative magnitude ( $M$ ) and a unit direction ( $D$ ) for each query. The final velocity is $V = M \odot D$ .
- This separation allows the model to learn the "strength" of the flow (speed) and the "direction" independently, improving training stability.

C. Training and Dataset

Dataset Construction: The authors created a large-scale dataset of 500k+ BEV image–instruction pairs from Matterport3D and ScanNet.
Procedural Annotation: Since real-world flow fields are hard to annotate, they are generated procedurally.
- A semantic map is used to define free space and obstacles.
- A cost-weighted geodesic distance field is computed towards the goal.
- Obstacle repulsion is added to create a potential field.
- The flow field is derived as the negative gradient of this potential, scaled by the distance-to-go to ensure quasi-constant velocity.
Loss Function: The model is trained using stratified sampling of query points. The loss combines Direction Loss (cosine similarity) and Magnitude Loss (L2 norm difference) between the predicted and ground-truth flow vectors.

D. Inference Strategy

Integration: Trajectories are generated via forward Euler integration.
Time Rescaling: To ensure the robot reaches the goal within a fixed horizon without exploding velocities near the end, a deterministic inverse-time rescaling is applied to the velocity magnitude.
Efficiency: Inference is a single forward pass to query the grid, followed by integration, avoiding the iterative sampling steps required by diffusion models.

3. Key Contributions

CoFL Policy: A novel transformer-based policy that reformulates navigation as flow field estimation, enabling smooth, continuous, and geometrically aware motion generation in a single forward pass.
Large-Scale BEV Dataset: The creation of a 500k+ sample dataset with procedurally annotated flow fields and trajectories, bridging the gap between 3D indoor scenes and language-conditioned navigation.
Superior Generalization: Demonstrated state-of-the-art performance on unseen scenes (Matterport3D and ScanNet), outperforming both modular VLM planners and generative diffusion policies in terms of safety and precision.
Zero-Shot Real-World Transfer: Successfully deployed on a physical robot in indoor environments without any real-world fine-tuning, validating the robustness of the learned flow-field representation.

4. Experimental Results

The paper evaluates CoFL against Pure VLMs, VLM + Planners, and various Diffusion Policy (DP) baselines.

Navigation Performance (Simulation):
- Safety: CoFL significantly reduces the Collision Rate (CR). On Matterport3D, CR dropped from ~0.86 (Diffusion baselines) to 0.22. On ScanNet, it dropped from ~0.88 to 0.39.
- Precision: CoFL maintains low Final Goal Error (FGE) comparable to or better than baselines (e.g., 0.14 vs 0.17 on Matterport3D).
- Smoothness: CoFL produces smoother paths (lower curvature) compared to discrete token-based methods.
- Efficiency: CoFL achieves real-time inference (~15ms) with controllable latency by adjusting grid resolution, whereas Diffusion policies require multiple denoising steps (increasing latency) to improve quality.
Real-World Deployment:
- Deployed on a differential-drive robot with an overhead camera.
- Achieved 100% success rate in reaching targets in obstacle-rich environments.
- Maintained a mean minimum clearance of 0.20m from obstacles.
- Operated with low latency (~28ms), enabling stable closed-loop replanning.

5. Significance and Impact

Geometric Awareness: By explicitly modeling the workspace as a flow field, CoFL inherently encodes obstacle avoidance and global geometry, solving the "semantic-geometric gap" found in modular VLM approaches.
Efficiency vs. Quality Trade-off: CoFL breaks the trade-off often seen in generative models where higher quality requires more sampling steps. It achieves high-quality, safe trajectories with a single forward pass.
Scalability: The use of procedural annotation allows for the generation of massive training datasets, which is crucial for training robust navigation policies that generalize to unseen environments.
Real-World Viability: The zero-shot transfer to physical robots demonstrates that flow-field representations learned in simulation are robust enough to handle real-world noise and dynamics, paving the way for safer autonomous navigation systems.

In conclusion, CoFL presents a compelling alternative to discrete action prediction and iterative generative modeling, offering a unified, efficient, and safe framework for language-conditioned navigation.