🤖 AI

From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation

This paper introduces FlowPilot, a mapless navigation policy for long-horizon sidewalk tasks that leverages anchored flow matching for pre-training on large-scale fleet data and human-in-the-loop preference learning to enhance social compliance and counterfactual reasoning, achieving robust performance with only a monocular RGB camera.

Original authors: Honglin He, Zhizheng Liu, Yukai Ma, Bolei Zhou

Published 2026-06-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Honglin He, Zhizheng Liu, Yukai Ma, Bolei Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a new delivery robot how to navigate a busy city sidewalk. You want it to walk miles, dodge pedestrians, avoid parked scooters, and follow social rules (like not cutting people off) using only a single camera on its head.

This paper introduces FlowPilot, a new "brain" for these robots that solves three major problems with current technology. Here is how it works, explained through simple analogies.

The Problem: The "Overconfident Student"

Current robots are trained using Imitation Learning. Think of this like a student watching hours of videos of a master driver.

The Issue: If the student just copies the video, they might get confused when the real world gets messy. They might make small mistakes that pile up (like a snowball rolling downhill), get stuck because they don't know how to handle a situation they haven't seen before, or act rude to pedestrians because they didn't learn the "social rules" of the sidewalk.
The Gap: Just watching videos (imitation) isn't enough to make a robot that is both safe and socially polite.

The Solution: FlowPilot's Two-Step Training

The authors created a two-step training process to fix this: Step 1 is "Learning the Basics" and Step 2 is "Learning the Nuance."

Step 1: The "Anchored Flow" (Learning the Basics)

Instead of just guessing one single path, the robot needs to understand that there are many ways to get around an obstacle (e.g., go left, go right, or slow down).

The Analogy: Imagine a river flowing around rocks. Sometimes the water splits into multiple streams. Old methods tried to predict just one stream, or they were too chaotic.
The Innovation: FlowPilot uses something called "Anchored Flow Matching."
- Anchors: Think of these as "mental bookmarks" or distinct driving styles (e.g., "The Aggressive Passer," "The Polite Yielder"). The robot learns these distinct styles first.
- Flow: Once it has these bookmarks, it learns how to smoothly flow between them. This allows the robot to generate smooth, realistic paths that cover all the different ways a human might navigate a tricky spot, rather than getting stuck in a single, rigid pattern.

Step 2: The "Human-in-the-Loop" (Learning the Nuance)

Even with great basics, the robot might still be a bit clumsy or socially awkward when deployed on a specific robot with specific cameras and wheels.

The Analogy: Imagine the robot is a new employee. It knows the job description (from Step 1), but it doesn't know your specific office culture yet.
The Innovation: The team introduced a Preference Learning system.
- A human supervisor watches the robot in real life.
- If the robot starts to do something unsafe or rude, the human gently steers it back on track.
- The robot learns: "Ah, the human preferred this action over that action."
- Instead of just memorizing the human's correction, the robot learns the preference. It understands, "In this situation, being polite is better than being fast." This step bridges the gap between "copying a video" and "aligning with human values."

The Results: A Smarter, Safer Robot

The team tested this system in both computer simulations and real-world city streets.

In the Simulation: The base version of FlowPilot was successful 42% of the time (a huge jump from other methods).
In the Real World: When they added the "Human Preference" training (FlowPilot-HP), the robot became much better at following rules.
- It needed human intervention 40% less often.
- It made 52% fewer mistakes that required a human to step in.
- It was much better at "social compliance," meaning it didn't cut people off or get too close to obstacles.

Summary

Think of FlowPilot as a robot driver that first learns the complex physics of driving from a massive library of videos (Step 1), and then gets a "mentorship" from a human supervisor who corrects its bad habits and teaches it the unwritten rules of the road (Step 2). The result is a robot that can navigate long distances on busy sidewalks safely, smoothly, and politely, using nothing but a single camera.

Technical Summary: From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation

Problem Statement

Autonomous long-horizon sidewalk navigation is critical for micro-mobility applications like robotic food delivery and assistive wheelchairs. Unlike road driving, sidewalk navigation involves unpredictable terrains, dense pedestrian interactions, and strict social norms, all while operating under severe constraints on onboard computing and battery life. The authors identify three primary challenges in existing approaches:

Compounding Errors and Distribution Shift: Standard Imitation Learning (IL) policies trained on large-scale offline data often fail to recover from errors in closed-loop settings, leading to compounding mistakes.
Lack of Social Compliance: Policies trained purely on demonstrations may not adhere to implicit social norms (e.g., yielding to pedestrians) or handle safety-critical states effectively.
Action Representation Limitations: Common action parameterizations struggle with the multimodal nature of sidewalk navigation (e.g., passing an obstacle on either side). Unimodal Gaussians are too restrictive, while diffusion or flow-matching policies can introduce excessive randomness or temporal inconsistency.
Generalization-Precision Dilemma: While training on diverse data improves generalization across environments, it often compromises the precision required for specific robot instances with unique camera parameters and mechanical attributes.

Methodology: FlowPilot

The paper proposes FlowPilot, a mapless navigation policy that utilizes only a single monocular RGB camera and GPS. The framework bridges the gap between large-scale imitation and robot-specific alignment through a two-stage process.

1. Anchored Flow Matching for Pre-training

To address the multimodal nature of sidewalk navigation, the authors introduce anchored flow matching as the action representation.

Anchor Generation: A constrained K-means clustering is applied to normalized action spaces from demonstration data to generate a set of $K$ anchor actions. These anchors serve as prototypical multimodal behaviors.
Flow Matching: The policy learns a velocity field to generate trajectories. Instead of regressing directly to a single trajectory, the model interpolates between noise and an expert trajectory conditioned on a specific anchor. This allows the model to capture diverse, complex, multimodal distributions while ensuring smooth, temporally consistent future trajectories.
Gated Cross-Attention: To prevent the model from relying too heavily on goal coordinates (which can lead to "shortcut" behaviors ignoring obstacles), the architecture employs a gated cross-attention mechanism. This mitigates "attention sink" on the goal, forcing the policy to ground decisions in scene context (obstacles, sidewalk boundaries) rather than just the destination.
Pre-training: The model is pre-trained on a large-scale dataset (SideWalks-300, 300 hours of teleoperation data from 281 robot instances) combined with other visual-action datasets. This stage uses supervised losses for visual odometry, mode classification (anchor assignment), and trajectory regression.

2. Human-in-the-Loop Preference Alignment

To bridge the gap between general imitation and specific robot precision, FlowPilot employs a reward-free preference learning scheme.

Data Collection: During deployment, a human operator monitors the robot via a teleoperation platform. When the policy predicts an unsafe or socially non-compliant action, the operator intervenes with a corrective command.
Preference Pairs: These interventions create preference pairs $(x^+, x^-)$ , where $x^+$ is the human-corrected action and $x^-$ is the policy's original (undesirable) prediction.
Alignment Objective: The policy is fine-tuned using a preference loss that increases the likelihood of the preferred action ( $x^+$ $x^{+}$ ) while suppressing the rejected action ( $x^-$ $x^{-}$ ).
- Score-Level Alignment: A scoring head is trained to assign higher probability to the preferred anchor mode than the rejected one, regularized by a KL-divergence term to preserve the pre-trained prior.
- Velocity-Level Alignment: An asymmetric trajectory-level objective is used. The preferred action serves as a positive regression target. Crucially, the rejected action is not directly pushed away (which could destabilize gradients in continuous flow matching); instead, it is regularized toward its anchor prior. This preserves the continuity of the action space while discouraging specific undesirable behaviors.

Key Contributions

Anchored Flow-Matching Policy: A novel action representation that combines discrete behavior priors (anchors) with continuous flow refinement to model complex, multimodal action distributions for long-horizon navigation.
Gated Conditioning Mechanism: An architectural design that mitigates goal-driven shortcuts by encouraging the policy to utilize scene context over direct goal coordinates.
Reward-Free Preference Learning: A human-in-the-loop fine-tuning algorithm that aligns the policy with social norms and robot-specific safety requirements using intervention data, without requiring explicit reward functions.
Comprehensive Evaluation: Extensive validation in both simulation and real-world environments across diverse sidewalk scenarios and robot embodiments.

Experimental Results

Open-Loop Evaluation

On a held-out evaluation set, FlowPilot-Base achieved state-of-the-art performance in trajectory prediction accuracy:

minMOE: 6.63 (lowest among baselines)
mAP: 0.87 (highest confidence in correct modes)
L2 Error: 1.04 (lowest endpoint error)

Closed-Loop Simulation

In a closed-loop simulation benchmark with 300 trials per scenario:

Success Rate (SR): FlowPilot-Base achieved 42%, significantly outperforming baselines like S2E (35%) and ViNT (27%).
Route Completion (RC): Achieved 66%.
Collision Rate (CR): Reduced to 0.43, compared to 0.61 for the ablation without Gated Cross-Attention.
Cross-Embodiment: FlowPilot-HP (with preference tuning) maintained high performance (SR 0.44–0.55) across novel robot configurations (R1, R2) with shifted dynamics and camera parameters, outperforming standard adaptation methods like DAgger.

Real-World Deployment

Evaluated on wheeled robots and robot dogs over 2.8 km of diverse routes (including night and complex lighting):

Intervention Rate (IR): FlowPilot-HP reduced IR by 40.0% and Normalized Intervention Rate (NIR) by 52.1% compared to the base model (FlowPilot-Base).
Social Compliance: The model demonstrated significantly higher Distance Compliance Rate (DCR) and Time Compliance Rate (TCR) than baselines, maintaining safer clearance from pedestrians and obstacles.
Robustness: The system successfully navigated long-horizon routes (up to 100m+ segments) and challenging conditions, including low-light environments.

Significance and Claims

The paper claims that FlowPilot represents a significant step forward in autonomous sidewalk navigation by addressing the limitations of pure imitation learning. The authors argue that:

Anchored Flow Matching provides a necessary representation for capturing the multimodal and control-sensitive nature of sidewalk navigation, which standard Gaussian or diffusion policies struggle with.
Alignment via Preference Learning is essential for transitioning from a broadly capable, general-purpose policy to a precise, socially compliant, and safe robot-specific controller. The results demonstrate that even a small amount of human intervention data, when processed through their preference learning framework, can drastically reduce unsafe behaviors and improve social compliance.
Scalability and Robustness: The framework successfully scales from large-scale pre-training to specific deployment, maintaining robustness across different robot embodiments and environmental conditions without requiring heavy sensors or explicit mapping.

The authors conclude that while their monocular approach has limitations regarding scale ambiguity and fine-grained 3D cues, the proposed framework effectively balances the trade-off between generalization and precision, offering a practical solution for real-world micro-mobility applications.