Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance

1. Problem Definition

The paper addresses the challenge of outdoor social navigation for robots in open-world environments. Unlike traditional navigation systems that rely on predefined coordinates or pre-built High-Definition (HD) maps, this work focuses on a scenario where a robot must accompany a human user based on abstract, high-level natural language instructions (e.g., "I want to go for a walk" or "Take this package to Building B").

Key Challenges:

• Intent Grounding: Translating vague human intent into a concrete, real-world destination without a pre-existing semantic map.
• Long-Horizon Planning: Navigating kilometer-scale routes in dynamic outdoor environments (cities, parks) rather than short indoor paths.
• Social Compliance: Adhering to social norms (personal space, traffic rules) and ensuring safety in complex scenarios like crowded crossings, traffic lights, and pedestrian interactions.
• Map-Free Operation: Avoiding the high cost and maintenance burden of constructing and updating city-scale HD maps.

2. Methodology: The "Walk With Me" Framework

The authors propose a hierarchical, map-free framework that decouples where to go (global planning) from how to walk (local control). The system operates in a closed-loop perception-reasoning-action cycle.

A. High-Level Semantic Planning (The "Where")

Instead of using HD maps, the system leverages public map services (e.g., Google Maps, Baidu Maps) as lightweight geographic priors.

1. POI-Aware Destination Grounding:
   • The system takes the user's instruction and current GPS location.
   • A High-Level Vision-Language Model (VLM) queries the map API for Points of Interest (POIs) matching the intent (e.g., searching for "parks" near the user).
   • The VLM selects a specific destination from the candidates and provides a natural language rationale.
2. Route Retrieval & Waypoint Construction:
   • The system queries the map's walking-route API to get a coarse path.
   • This path is re-sampled into a sequence of geo-referenced waypoints, preserving semantic cues (e.g., "crossing," "traffic light") for downstream safety reasoning.

B. Adaptive Dual-Policy Navigation (The "How")

During execution, the robot continuously observes the environment and decides how to act using an observation-aware routing mechanism.

1. Observation Construction: At each step, the robot aggregates the current RGB image, pose, trajectory history, and the next waypoint.
2. Joint Routing & Safety Decision (High-Level VLM):
   • The High-Level VLM analyzes the visual scene and route context to determine if the current situation is Routine or Complex/Safety-Critical.
   • It simultaneously decides whether to Proceed or Stop-and-Wait.
   • Complex situations include crowded crossings, ambiguous traffic, or dense crowds.
3. Low-Level VLA Execution:
   • If "Proceed": The step is passed to a Low-Level Vision-Language-Action (VLA) policy. This policy predicts a short-horizon, socially compliant trajectory based on the visual observation and the next waypoint.
   • If "Stop-and-Wait": The robot halts immediately. It waits, captures a new image, and re-queries the High-Level VLM until a safe "proceed" decision is made.

C. Closed-Loop Execution

The system fuses the VLA's predicted trajectory with onboard SLAM (Simultaneous Localization and Mapping) and low-level motion controllers. The state is updated iteratively until the destination is reached or a safety budget is exceeded.

3. Key Contributions

1. Problem Formulation: Defined the task of map-free, long-horizon outdoor social navigation driven by high-level human instructions, bridging the gap between abstract intent and physical execution.
2. Hierarchical Framework: Proposed Walk With Me, which couples a High-Level VLM (for intent grounding, route planning, and safety reasoning) with a Low-Level VLA (for socially compliant local trajectory generation).
3. Observation-Aware Routing: Introduced a novel mechanism where a High-Level VLM dynamically routes tasks: routine navigation is handled by the efficient Low-Level VLA, while complex/safety-critical scenes trigger explicit high-level reasoning and stop-and-wait behaviors.
4. Real-World Validation: Successfully instantiated the framework on a commodity wheeled robot (Athena 2.0 Pro) using public map APIs, demonstrating reliable navigation in real-world outdoor environments without pre-built HD maps.

4. Experimental Results

The system was evaluated in 20 real-world trials across two application categories: Last-Mile Delivery and Blind Guidance.

• Overall Performance: Achieved a 60% Success Rate (SR) across all trials.
  • Last-Mile Delivery: 70% SR (Tasks: Delivering milk tea to Building B, Package to Building A).
  • Blind Guidance: 50% SR (Tasks: "Go for a walk," "Go shopping").
• Ablation Studies:
  • High-Level VLM: Different VLM backbones significantly impacted performance. MiMo-Embodied (Open-source) achieved the highest SR (60%), outperforming general-purpose models like Qwen3-VL-8B (30%) in this specific navigation context.
  • Low-Level VLA: Socially aware policies performed best. SocialNav achieved 60% SR, significantly outperforming classical visual navigation models like GNM (20%) and ViNT (20%), highlighting the necessity of social awareness in local control.
• Qualitative Analysis: The system successfully handled intent grounding (e.g., selecting a specific park for a "walk"), managed traffic lights (stopping for red lights), and navigated around pedestrians. However, ambiguity in complex visual scenes sometimes led to divergent reasoning among different VLMs.

5. Significance and Future Work

Significance:

• Scalability: By relying on public map APIs rather than custom HD maps, the system is scalable to any city with digital map coverage, removing a major barrier to deploying service robots in the real world.
• Human-Centricity: The framework prioritizes safety and social norms, making it suitable for sensitive applications like guiding visually impaired users or delivering goods in crowded areas.
• Architectural Insight: It demonstrates that a hybrid approach—combining the semantic reasoning of foundation models (VLMs) with the control precision of learned policies (VLA)—is superior to end-to-end approaches for long-horizon tasks.

Limitations & Future Directions:

• Current performance is limited by the reliability of public map data, GPS noise, and the robustness of VLMs in rare or highly dynamic social scenarios.
• Future work aims to improve uncertainty handling, integrate richer multi-modal perception, and expand deployment to more diverse outdoor environments.