NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions

This paper introduces the NavSpace benchmark to systematically evaluate the spatial intelligence of navigation agents through six task categories and 1,228 trajectory-instruction pairs, revealing limitations in current models and proposing SNav, a new spatially intelligent navigation model that outperforms existing agents on both the benchmark and real robot tests.

The Gist

Imagine you are teaching a robot to navigate your house. You might say, "Go get my bag from the dining room." A smart robot understands "dining room" and "bag."

But what if you say, "Imagine you are the lamp on the table. Walk to the left of where you are standing, then go down to the basement and check if the lights are on?"

This is where most robots (and even advanced AI) get confused. They can understand the words, but they fail at the spatial math required to actually do it. They don't know what "left of the lamp" means from the lamp's perspective, nor do they have a good sense of "basement" vs. "upstairs" or "3 meters away."

This paper, NavSpace, is like a new, very difficult driver's license test for robots. It's designed to see if they truly understand space, or if they are just guessing based on keywords.

Here is the breakdown of the paper in simple terms:

1. The Problem: Robots are "Wordy" but "Space-Challenged"

Current robots are great at following simple commands like "Go to the kitchen." But in the real world, humans give complex instructions involving:

• Verticality: "Go to the 3rd floor."
• Precision: "Turn exactly 30 degrees and walk 2 meters."
• Perspective Shifting: "Imagine you are the cat; walk to your right."
• Conditionals: "If the light is off, go to the living room; if it's on, stay here."

The researchers realized that while we have tests for how well robots understand language, we didn't have a test for how well they understand space.

2. The Solution: The "NavSpace" Test

The team created a new benchmark called NavSpace. Think of it as a gym for robot brains, specifically designed to build "spatial muscles."

• The Dataset: They collected over 1,200 real-world navigation scenarios.
• The 6 Categories: They tested robots on six specific "spatial skills":
  1. Vertical Perception: Knowing which floor you are on (e.g., "Go to the top floor").
  2. Precise Movement: Following exact distances and angles (e.g., "Walk 3 meters, turn right").
  3. Viewpoint Shifting: Changing your perspective (e.g., "If you were the TV, where would you go?").
  4. Spatial Relationships: Understanding order and position (e.g., "Stop between the sofa and the table").
  5. Environment State: Reacting to conditions (e.g., "If you see a dog, stop").
  6. Space Structure: Understanding shapes and loops (e.g., "Walk around the table in a circle").

3. The Results: The "Smart" Robots Failed

The researchers tested 22 different AI models, including the most famous ones from OpenAI (GPT-5), Google (Gemini), and specialized robot models.

• The Shocking Result: Even the "smartest" AI models (like GPT-5) scored terribly. Their success rate was often below 20%.
• The Analogy: It's like a student who can write a perfect essay about driving but fails the actual driving test because they can't judge the distance to a stop sign. The AI can talk about space, but it can't navigate it.
• Why? The AI models get confused when they have to translate a mental image into physical movement. They lose track of where they are after a few steps.

4. The Hero: Introducing "SNav"

Since the existing models failed, the authors built their own robot brain called SNav (Spatially Intelligent Navigation).

• How they built it: Instead of just feeding the robot random instructions, they created a special training method. They taught the robot to:
  • Count floors.
  • Measure distances precisely.
  • Imagine different viewpoints.
  • React to "If/Then" scenarios.
• The Result: SNav became the new champion. It significantly outperformed the big commercial models and the specialized robot models. It proved that if you specifically train a robot on spatial reasoning, it becomes much better at navigating the real world.

5. The Real-World Test

They didn't just test this in a computer simulation. They put their robot (a four-legged dog-like robot) in a real office and campus.

• The Challenge: They asked the robot to do things like "Walk around a table and come back" or "Go to the second door on the left."
• The Outcome: SNav succeeded about 32% of the time, while the other top models failed almost 100% of the time.

The Big Takeaway

This paper tells us that understanding language is not the same as understanding space.

To build a truly helpful robot that can navigate our messy, multi-story, complex homes, we can't just rely on "smart" language models. We need to teach them the geometry of the world. NavSpace is the ruler we use to measure that skill, and SNav is the first robot to show us that with the right training, robots can finally learn to "see" space the way we do.

Technical Summary

Here is a detailed technical summary of the paper "NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions."

1. Problem Statement

While instruction-following navigation has advanced significantly through benchmarks like R2R and RxR, existing evaluations primarily focus on semantic understanding (language and visual object recognition) rather than spatial intelligence. Current benchmarks fail to systematically assess an agent's ability to perceive spatial scales, relative orientations, environmental states, and complex spatial structures. Consequently, the spatial reasoning capabilities of both Multimodal Large Language Models (MLLMs) and specialized navigation models remain unclear, and there is no standardized benchmark to evaluate or improve these specific skills in embodied agents.

2. Methodology

A. The NavSpace Benchmark

The authors introduce NavSpace, the first benchmark designed specifically to evaluate spatial intelligence in instruction-following navigation.

• Data Collection Pipeline:
  1. Questionnaire Survey: A survey identified six critical categories of spatial intelligence essential for navigation.
  2. Trajectory Collection: Annotators teleoperated agents in the Habitat 3.0 simulator (using HM3D scenes) to record navigation trajectories.
  3. Instruction Annotation: Annotators used GPT-5 to analyze trajectories and generate candidate instructions, which were then refined by humans.
  4. Human Cross-Validation: A separate annotator executed the generated instructions to ensure they were accurate and executable.
• Dataset Statistics: The benchmark contains 1,228 high-quality trajectory-instruction pairs.
• Six Task Categories:
  1. Vertical Perception: Understanding floor levels and relative heights (e.g., "go to the top floor").
  2. Precise Movement: Executing specific distances and angles (e.g., "turn right 30°, go 3m").
  3. Viewpoint Shifting: Reasoning from a non-agent perspective (e.g., "Imagine you are the TV...").
  4. Spatial Relationship: Handling sequential ordering and relative positions of multiple objects (e.g., "between the sofa and table").
  5. Environment State: Conditional logic based on environmental observations (e.g., "if the light is off, go to...").
  6. Space Structure: Complex path planning like circling objects or navigating to extreme locations.

B. The SNav Model

To address the limitations of existing models, the authors propose SNav, a spatially intelligent navigation large model.

• Architecture: Built upon LLaVA-Video 7B, it integrates a Vision Encoder (SigLIP), a Projector (MLP), and an LLM (Qwen2).
• Training Strategy:
  1. Vanilla SNav: Initialized via co-training on three tasks: Navigation Action Prediction, Trajectory-based Instruction Generation, and General Multimodal Data Recall.
  2. Spatial Intelligence Enhancement: The model is further fine-tuned using a data generation pipeline that creates synthetic spatial instructions from open-source datasets (R2R, MP3D). This pipeline generates data for:
     • Cross-floor navigation (using floor segmentation).
     • Precise movement (converting path steps into natural language).
     • Environment state inference (using "if/otherwise" templates).
     • Spatial relationships (using regex to extract ordinal and relational phrases).

3. Key Contributions

1. NavSpace Benchmark: The first benchmark dedicated to spatial intelligence in embodied navigation, featuring 1,228 manually curated, high-quality instruction-trajectory pairs across six distinct spatial reasoning categories.
2. Comprehensive Evaluation: A rigorous evaluation of 22 agents, including state-of-the-art MLLMs (GPT-5, Gemini 2.5 Pro, Qwen2.5-VL), lightweight navigation models, and navigation large models.
3. SNav Model: A new baseline model that outperforms existing navigation models and MLLMs on NavSpace and real-world robot tests, demonstrating that targeted spatial data injection significantly improves embodied navigation.

4. Experimental Results

A. Simulation Performance (NavSpace)

• MLLMs: Even the most advanced proprietary MLLMs (e.g., GPT-5, Gemini 2.5 Pro) struggle significantly, with average Success Rates (SR) below 20%. Open-source MLLMs perform even worse (SR < 10%), often comparable to random chance.
• Lightweight Models: Traditional lightweight navigation models (e.g., BEVBert, ETPNav) fail almost entirely on spatial tasks, indicating they rely on shallow semantic-to-action mapping rather than true spatial reasoning.
• Navigation Large Models: Models like StreamVLN and NaVid perform better than lightweight models but still lag behind SNav.
• SNav Performance: SNav achieves the highest performance across all categories, with an average Success Rate of 26.0% and Oracle Success of 36.7%, significantly outperforming GPT-5 (SR 14.2%) and StreamVLN (SR 19.2%).
  • Key Insight: SNav's ablation study confirms that the spatial instruction generation pipeline is crucial for its success.

B. Real-World Robot Tests

• Setup: Tested on an AgiBot Lingxi D1 quadruped in office, campus, and outdoor environments.
• Results: SNav achieved a 32% Success Rate, significantly outperforming NaVid (14%) and NaVILA (6%). This demonstrates the model's ability to transfer spatial reasoning capabilities from simulation to real-world hardware.

5. Significance and Discussion

• Gap in Current AI: The paper reveals a critical disconnect: MLLMs that excel at static spatial benchmarks (e.g., VSI-Bench) fail in dynamic embodied navigation. This suggests that translating spatial perception into precise, sequential actions is a distinct and harder challenge than static reasoning.
• Limitations of Current Models: Current MLLMs suffer from "perception-action inconsistency," where their internal reasoning about space contradicts the actions they generate. Lightweight models lack the capacity for deep spatial reasoning.
• Future Directions: The work establishes that improving embodied intelligence requires parallel advancements in:
  1. Spatial Perception: Better understanding of scale, structure, and state.
  2. Inferential Mechanisms: Robust methods to translate spatial understanding into reliable action sequences.
• Impact: NavSpace provides a necessary standard for evaluating and improving the "spatial intelligence" of robots, moving the field beyond simple semantic navigation toward complex, human-like spatial reasoning.