💻 computer science

IntentNav: Learning Spatial-Visual Object Navigation from Human Demonstrations

IntentNav is a spatial-visual imitation framework that learns human-like object navigation policies by inferring high-level search intent from demonstrations via frontier-based labeling and training a VLM to select grounded candidates, achieving state-of-the-art performance and zero-shot transfer across diverse robot platforms.

Original authors: Yuxin Cai, Zongtai Li, Maonan Wang, Muyi Bao, Haokun Zhu, Ruofei Bai, Ding Zhao, Zirui Li, Wenshan Wang, Wei-Yun Yau, Ji Zhang, Chen Lv

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuxin Cai, Zongtai Li, Maonan Wang, Muyi Bao, Haokun Zhu, Ruofei Bai, Ding Zhao, Zirui Li, Wenshan Wang, Wei-Yun Yau, Ji Zhang, Chen Lv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you walk into a friend's house for the first time, and they ask you to find their coffee mug. You don't just wander aimlessly, checking every single drawer in every room. Instead, you use your brain to make smart guesses: "Mugs are usually in the kitchen," so you head there first. If you see a hallway, you peek in to see if it looks like a dining area. If you've already checked the living room and found nothing, you remember that and don't go back there. You are balancing what you see (visual clues) with where you've been (spatial memory) to find the object efficiently.

This paper, IntentNav, teaches robots to do exactly that. It creates a system that learns how to search for objects by watching how humans do it, rather than just following rigid, pre-programmed rules.

Here is a breakdown of how it works, using simple analogies:

1. The Problem: Robots Get "Lost" in Their Own Thoughts

Current robots often struggle with this task. They might:

Spin in circles: Like a dog chasing its tail, they make small, repetitive moves without covering new ground.
Forget where they've been: They might walk back into a room they just checked, thinking it's new.
Get stuck on details: They might focus too much on the immediate view (like a person staring at a single shoe) and miss the bigger picture of the house layout.

2. The Solution: "IntentNav" (The Robot's Inner Compass)

The researchers built a system that learns from human demonstrations. Think of it as a robot apprentice watching a master explorer.

The "Human-Intent" Translator: Humans move smoothly, but a robot sees a long list of tiny steps (move forward, turn left, move forward). The system uses a clever trick called Frontier-based Human-Intent Labeling.
- Analogy: Imagine watching a movie of someone searching a house. The system doesn't just watch the foot movements; it looks ahead to see where the person is heading next. It asks, "Why did they turn left there? Oh, they saw a kitchen door!" It then labels that turn as a "smart decision" to teach the robot.
The "Bird's-Eye View" Map (BEV): Instead of looking at the world only through the robot's eyes (like a first-person video game), the system builds a Top-Down Map (like a Google Maps view).
- On this map, the robot marks three things:
  1. Explored areas: "I've been here."
  2. Unknown frontiers: "I haven't been there yet."
  3. Potential targets: "That looks like a fridge!"
- This map acts as a shared "decision board" where the robot can see the whole picture at once.

3. How the Robot Decides: The "Menu" System

Instead of guessing the next tiny movement (like "turn 5 degrees left"), the robot looks at a menu of specific destinations (waypoints) on its map.

The Menu: The system presents the robot with a list of interesting spots to go to next (e.g., "The kitchen doorway," "The hallway end," "The living room corner").
The Brain (VLM): A powerful AI brain (a Vision-Language Model) looks at the map, the list of destinations, and what the robot "saw" when it first looked at those spots. It then picks the best destination from the menu.
The "Intent-Aligned" Training: The robot is trained not just to pick the exact spot the human picked, but to pick a spot that is in the same direction.
- Analogy: If a human walks toward the kitchen, and the robot picks a spot in the kitchen, that's a win. It doesn't matter if the robot picks the exact same tile the human stepped on; what matters is that they are heading in the same intent.

4. The Results: A Robot That Actually "Gets It"

The paper shows that this method works incredibly well:

Better Search: The robot finds objects faster and takes more efficient paths than other learning-based robots. It stops spinning in circles and stops revisiting rooms it already checked.
Zero-Shot Transfer: This is the most impressive part. The robot was trained using data from a computer simulation of houses. When the researchers put the exact same brain onto real physical robots—a wheeled robot, a four-legged dog robot, and a humanoid robot—it worked immediately.
- Analogy: It's like teaching a pilot to fly in a flight simulator, and then having them hop into a helicopter, a boat, or a car and knowing exactly how to navigate without needing a new lesson.

Summary

IntentNav is a new way to teach robots to search. Instead of forcing them to memorize every step, it teaches them to:

Look at the big picture (using a top-down map).
Learn from human intuition (understanding why a human went somewhere).
Choose smart destinations from a list, rather than guessing tiny movements.

The result is a robot that explores new environments like a human would: purposeful, efficient, and rarely getting stuck in loops.

Technical Summary: IntentNav

Problem Statement

Object Navigation (ObjectNav) requires an embodied agent to locate an unobserved target object within an unknown environment under partial observability. The core challenge lies in performing long-horizon, target-conditioned search: the agent must infer promising search directions from partial observations and adapt as new evidence emerges without exhaustive traversal.

While recent Vision-Language Models (VLMs) offer strong semantic reasoning, existing approaches struggle with stable long-horizon search. Methods relying on first-person RGB observations often lack explicit spatial structure, leading to locally plausible but globally incoherent strategies (e.g., repetitive backtracking or turning in place). Conversely, map-mediated reasoning methods often abstract away fine-grained visual details by converting spatial maps into text, losing the geometric context necessary for structured memory. Furthermore, many learning-based policies are trained on oracle shortest-path trajectories, optimizing for goal-reaching under full observability rather than modeling the high-level search intent required under partial observability.

Methodology: IntentNav

IntentNav is a spatial-visual imitation framework that learns candidate-level ObjectNav decisions from human demonstrations. It operates within a unified Bird's-Eye-View (BEV) decision space, grounding both exploration frontiers and target hypotheses.

1. Frontier-based Human-Intent Labeling

To bridge the gap between low-level human action sequences and high-level waypoint decisions, the authors introduce a labeling strategy that infers search intent:

Mechanism: The system replays human demonstrations in a simulator. If a target is visible, the label is the target candidate. If not, the system looks ahead to future frontier evolution.
Logic: It identifies the earliest future step where the current frontier set significantly changes (diverges). The candidate in the current set that is most inconsistent with this future frontier set (i.e., the path the human did not take) is identified, and the candidate that best aligns with the human's continued exploration direction is selected as the ground truth label.
Outcome: This converts low-level trajectories into candidate-level proxy supervision that captures the demonstrator's high-level search intent.

2. BEV-Grounded Spatial-Visual Candidate Policy

The policy selects the next waypoint from a unified set of candidates defined in a persistent BEV map.

Candidate Representation: Each candidate $x_i$ $x_{i}$ is grounded by two complementary features:
- Egocentric Visual Memory: The RGB image captured when the candidate was first instantiated, providing local semantic context (e.g., room type, nearby objects).
- Agent-Centric Geometry: A feature encoding the normalized distance and relative bearing of the candidate from the agent's current pose.
Architecture: A VLM (fine-tuned from InternVL3-2B) receives the BEV map, serialized candidate blocks (containing visual memory and geometry), and the target category.
Output: Instead of generating free-form coordinates, the model predicts a reserved "Candidate-ID" token, converting waypoint selection into a controlled classification task.

3. Intent-Aligned Objective

To encourage human-like exploration, the training objective moves beyond standard cross-entropy:

Angular Soft Target: For frontier candidates, the loss function assigns partial credit to candidates near the demonstrated direction based on angular proximity. This prevents the model from being overly penalized for selecting a nearby valid alternative.
Hard Commitment: For verified target candidates, standard hard cross-entropy is used to encourage decisive commitment.
Combined Loss: A weighted sum of the hard candidate loss and the angular soft target loss guides the policy toward directionally consistent exploration.

Key Contributions

Frontier-based Human-Intent Labeling: A novel method to derive candidate-level supervision from low-level human trajectories by reasoning about future frontier evolution, enabling the learning of human search intent.
BEV-Grounded Spatial-Visual Policy: A unified decision interface that combines a persistent BEV spatial frame with egocentric visual memory and agent-centric geometry, allowing the VLM to reason over semantic evidence, spatial relations, and search history simultaneously.
State-of-the-Art Performance & Zero-Shot Transfer: The framework achieves top performance on standard benchmarks and demonstrates the ability to transfer zero-shot across different robot embodiments (wheeled, quadruped, humanoid) without further VLM fine-tuning.

Experimental Results

The authors evaluated IntentNav on three Habitat ObjectNav benchmarks: MP3D, HM3D-v1, and HM3D-v2.

Benchmark Performance: IntentNav achieved state-of-the-art results among learning-based methods across all datasets.
- MP3D: 53.8% Success Rate (SR), 23.1% Success weighted by Path Length (SPL).
- HM3D-v1: 70.5% SR, 34.6% SPL.
- HM3D-v2: 82.2% SR, 38.5% SPL.
- These results outperform both training-free VLM systems and other learning-based policies, with significant gains in SPL indicating more efficient search behaviors.
Search Behavior Analysis: Trajectory-level metrics show that IntentNav significantly increases Exploration Coverage Ratio (ECR) and reduces Redundant Revisit Ratio (RRR) compared to egocentric action baselines. The policy exhibits more purposeful probing motions and less local scanning than random or standard baselines.
Real-World Deployment: The same MP3D-trained checkpoint was deployed on a wheeled robot, a Unitree Go2 quadruped, and a Unitree G1 humanoid. The system operated in real-time (approx. 0.8s inference per step) across 10 real-world scenes and 8 target categories without additional fine-tuning.

Significance and Claims

The paper claims that IntentNav addresses the critical limitation of current VLM-based navigation: the inability to maintain coherent, long-horizon search strategies under partial observability. By grounding decisions in a unified BEV space and learning directly from human search intent rather than shortest paths, the method produces behavior that is both efficient and robust.

The authors emphasize that the candidate-level interface is key to this success. It decouples high-level decision-making from low-level control, allowing the same policy to transfer across diverse physical embodiments (wheeled, quadruped, humanoid) without retraining. The work demonstrates that learning from human demonstrations, when properly structured to capture high-level intent, can yield navigation policies that outperform both purely learning-based and training-free VLM approaches in complex, unseen environments.

Limitations noted by the authors:

The system assumes a mostly static environment; dynamic changes may render stored map regions and visual memories stale.
High-level waypoint selection is decoupled from low-level execution; execution failures or local planner constraints are not explicitly fed back into the VLM policy.