PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildings

The paper introduces PM-Nav, a novel framework that leverages priori-semantic maps and hierarchical chain-of-thought prompting to overcome the challenges of language-driven navigation in functional buildings with highly similar features, achieving substantial performance improvements over existing methods in both simulation and real-world environments.

The Gist

Imagine you are trying to find the "One-Stop Service Centre" in a massive, brand-new government office building. The building is huge, the hallways are all exactly the same, the doors look identical, and there are no unique decorations. If you asked a standard robot to "go find the service centre," it would likely get lost immediately, spinning in circles because every corner looks like every other corner.

This paper introduces PM-Nav, a new way to help robots navigate these confusing "Functional Buildings" (like schools, hospitals, and offices). Think of PM-Nav as giving the robot a super-powered GPS and a human-like thinking process combined.

Here is how it works, broken down into simple analogies:

1. The Problem: The Robot is "Blind" to the Big Picture

Current robots are like tourists who only look at their feet. They see a wall, then another wall, and try to guess where they are. In a house with unique furniture (a red sofa, a blue rug), this works fine. But in a functional building where every room is a white box with a white door, the robot gets confused. It lacks the "big picture" knowledge that humans use.

2. The Solution: The "Annotated Map" (The Priori-Map)

Instead of just showing the robot a raw photo of the building, the researchers convert the building's blueprint into a special, easy-to-read map.

• The Analogy: Imagine you have a complex subway map. A normal map just shows lines. The PM-Nav map is like a subway map where every station is labeled with a number, and the tracks between them are named "Segment 13" or "Turn 2."
• What it does: It turns a confusing visual mess into a clear, step-by-step list of instructions (e.g., "Go from Room 14 to Room 7 via Segment 13"). This helps the robot understand the logic of the building, not just the pictures.

3. The Brain: The "Chain of Thought" (H-CoT)

Once the robot has this special map, it doesn't just guess. It uses a step-by-step thinking process (called a Hierarchical Chain-of-Thought).

• The Analogy: Think of a human giving directions to a friend. You don't just say "Go there." You say: "First, walk straight until you see the big red door. Then, turn left at the fork. Then, walk past the elevator."
• What it does: The robot's AI (a Large Language Model) looks at the special map and breaks the journey down into tiny, logical steps. It plans the whole route before it even starts moving, ensuring it knows exactly which "fork" in the road to take.

4. The Hands and Eyes: The "Teamwork" System

This is where the magic happens. The robot doesn't rely on just one brain or one camera. It uses a three-person team to make decisions:

1. The Strategist (VLM): This is the big brain. It looks at the panoramic view (a 360-degree photo) and says, "I think the target is roughly 30 degrees to the left." It's good at the big picture but a bit fuzzy on details.
2. The Spotter (GroundingDINO & SAM): These are the sharp-eyed assistants. They look at the Strategist's guess and say, "Wait, I see a specific sign or door frame right there." They pinpoint the exact location.
3. The Driver (PixelNav): This is the muscle. It takes the "rough guess" from the Strategist and the "exact pin" from the Spotter and calculates the perfect steering angle to move the robot forward without hitting a wall.

The Analogy: It's like a treasure hunt where one person holds the map (Strategist), another person spots the X on the ground (Spotter), and a third person actually digs the hole (Driver). They talk to each other constantly to make sure they don't dig in the wrong spot.

5. The Results: From "Lost" to "Master Navigator"

The researchers tested this in both computer simulations and a real university building.

• The Old Way: Other robots (like SG-Nav) were like lost puppies. In difficult tasks, they succeeded 0% of the time. They just couldn't tell one room from another.
• The PM-Nav Way: With the special map and the teamwork system, the robot's success rate skyrocketed. In some tests, it was 6 to 12 times better than the old methods. It could navigate complex, confusing buildings where other robots gave up.

Summary

PM-Nav is like giving a robot a human's ability to read a map and think ahead, combined with a team of specialists to spot landmarks and steer precisely. It solves the problem of "looking the same" in big buildings by turning the building's layout into a clear, logical story that the robot can follow step-by-step.

Technical Summary

Here is a detailed technical summary of the paper "PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildings."

1. Problem Statement

The paper addresses the significant challenges faced by language-driven embodied navigation agents when operating in Functional Buildings (FBs) (e.g., hospitals, schools, government offices). Unlike standard indoor environments (homes/offices) which have diverse layouts, FBs are characterized by:

• Highly similar features: Rooms and corridors often look identical, making visual differentiation difficult.
• Complex structures: Large open spaces with intricate topologies.
• Limitations of Current SOTA: Existing methods (like SG-Nav and InstructNav) rely heavily on visual perception or simplified maps. They fail in FBs because:
  1. They lack global planning capabilities based on priori spatial knowledge.
  2. Current Vision-Language Models (VLMs) struggle with spatial logical reasoning required to interpret complex maps.
  3. There is significant perceptual ambiguity, making it hard for agents to decide the next action without precise localization.

2. Methodology: PM-Nav Framework

The authors propose PM-Nav, a framework that mimics human navigation by utilizing priori maps and landmarks. The system operates in three main stages:

A. Map Parsing (Semantic Priori-Map Construction)

To bridge the gap between raw maps and VLM reasoning:

• Transformation: Raw environmental maps are parsed into structured semantic priori-maps using a Tkinter interface.
• Segmentation: The map is divided into "segments" (e.g., seg13) connecting specific rooms (e.g., room14 to room7).
• Annotation: Key waypoints (start, turn, branch, end) are defined. This reduces cognitive load on the VLM by converting complex geometry into a logical, text-friendly topological representation.

B. VLM Planning (Hierarchical Chain-of-Thought)

The system uses a Hierarchical Chain-of-Thought (H-CoT) prompting strategy combined with the annotated priori-map:

• Input: The VLM (GPT-4o) receives the semantic map, the starting segment, and the target room.
• Reasoning: The H-CoT prompt guides the model to:
  1. Analyze spatial relationships between the current segment and the target.
  2. Identify the sequence of waypoints required.
  3. Determine the first-person action (e.g., "Go straight," "Turn left").
• Output: A global path plan decomposed into consecutive road segments, simplifying the task to "reach the next landmark" rather than processing the entire global path at once.

C. Multi-Model Collaborative Action Generation

To execute the plan, the system employs a closed-loop process: Target Determination → Search → Verification → Target Update.

• Coarse-Grained Action: The VLM analyzes panoramic images (captured at 30° intervals) to predict a general direction and locate landmarks.
• Fine-Grained Action:
  • Landmark Detection: GroundingDINO and SAM (Segment Anything Model) collaborate to precisely mark the target landmark in the image.
  • Precision Control: The mask center of the landmark and the current view are fed into a PixelNav neural network (fine-tuned) to output precise steering angles.
• Localization: Before planning, the agent identifies at least two surrounding landmarks to determine its current position. If landmarks are scarce, the agent actively explores until positioning is achieved.

3. Key Contributions

1. PM-Nav Framework: The first work to deeply explore embodied navigation specifically for Functional Buildings, mimicking human reliance on priori maps.
2. Semantic Priori-Map & H-CoT: A novel method to parse maps into segmented semantic structures and a prompt template that integrates these cues with symbolic annotations, significantly enhancing VLM spatial reasoning.
3. Multi-Model Collaboration: A hybrid action generation mechanism combining VLMs (for logic), base vision models (for detection), and PixelNav (for precise control) to handle coarse-to-fine navigation.
4. New Dataset & Environments: Construction of a novel navigation dataset and 6 distinct FB simulation environments (based on Gazebo) to address the lack of suitable benchmarks for this domain.

4. Experimental Results

The authors conducted extensive tests in both simulation and a real-world school environment (Foshan Graduate School).

• Simulation Performance:
  • Compared to SOTA methods (SG-Nav, InstructNav), PM-Nav achieved average improvements of 511% and 1175% in Success Rate (SR) and Success weighted by Path Length (SPL) for easy/medium tasks.
  • For "Hard" tasks (complex paths), existing methods failed completely (0% SR), while PM-Nav achieved 46% SR.
• Real-World Performance:
  • In real-world FB tests, PM-Nav showed 650% and 400% improvements over baselines for easy tasks.
  • It successfully navigated medium and hard tasks where baselines failed entirely, achieving 55% and 15% SR respectively.
• Ablation Studies:
  • Input Type: Using H-CoT with annotated priori-maps (H-PM) improved SR by over 7-fold compared to standard prompts.
  • Action Refinement: The inclusion of the fine-grained action module (PixelNav) was critical; without it, success rates for hard tasks dropped to 0%.
  • Localization: The system maintained >90% localization success when redundant landmarks were available, dropping to 63.3% in scarce landmark scenarios (requiring active exploration).

5. Significance

This paper demonstrates that integrating priori spatial knowledge with VLM reasoning is essential for navigating complex, feature-similar environments.

• Practical Impact: It provides a viable solution for service robots in critical infrastructure (hospitals, schools) where traditional visual-only navigation fails due to visual homogeneity.
• Methodological Advance: It proves that "structured semantic maps" + "hierarchical reasoning" + "multi-model collaboration" can overcome the logical reasoning limitations of current VLMs in spatial tasks.
• Future Direction: The work highlights the need for better end-to-end map parsing and more efficient exploration strategies to further reduce unnecessary traversal.