Technical Summary: SAIN - Structure-Aware Interactive Navigation with Active Dialogue Grounding for Mobile Robot

1. Problem Formulation

The paper addresses Interactive Instance Goal Navigation (IIGN), a task where an embodied agent must locate a specific target instance within an unknown indoor environment based on an ambiguous, category-level instruction (e.g., "Find the chair") provided by a human or oracle. Unlike standard ObjectNav, where reaching any instance of a category suffices, IIGN requires distinguishing the intended target from same-category distractors.

The core challenge lies in resolving two coupled uncertainties:

1. Instance Uncertainty: The instruction does not uniquely identify the target object.
2. Exploration Uncertainty: The target may lie in an unobserved branch, room, or corridor.

Existing methods often treat dialogue answers as transient textual context for immediate action decisions. This approach fails in long-horizon tasks because short-term conditioning is lost once the agent moves to new observations or topological branches. The paper argues that for IIGN to succeed, interactive answers must be converted into persistent, structured navigation states that support map construction, object reasoning, and long-term planning.

2. Methodology: SAIN Framework

The proposed SAIN (Structure-Aware Interactive Navigation) is a zero-shot framework built on the dialogue-to-state principle. Instead of converting oracle answers directly into actions, SAIN compiles them into persistent internal memories that guide a unified navigation policy.

A. Structured Memory Representation

SAIN maintains four structured maps and a set of state variables:

• Value Map ($V_t$): A target-conditioned exploration prior over frontiers, derived from vision-language matching.
• Graph Map ($G_t$): A topological skeleton of the explored space, exposing spatial uncertainty and defining path proposals.
• Room Map ($R_t$): Partitions the space into room-like regions with semantic labels to guide room-level scoring.
• Object Map ($O_t$): Stores point clouds and candidate labels for object instances, maintaining consistency over time.
• Persistent State Variables:
  • Target Facts ($F_{tar}$): Accumulated attributes from information questions.
  • Route/History Masks ($M^{route}_t, M^{hist}_t$): Grounded topological corridors and decaying historical paths derived from route questions.
  • Candidate Labels: States (Confirmed, Pending, Rejected) for detected objects.

B. Active Questioning and State Updates

SAIN triggers three specific types of questions based on the current state of uncertainty:

1. Information Questions: Triggered when target facts are insufficient. Answers are parsed into Target Facts ($F_{tar}$) to refine the target description for instance verification and room relevance scoring.
2. Route Questions: Triggered at unqueried branch nodes when no target is confirmed. Answers are grounded to the topological graph via a bounded Depth-First Search (DFS) and Branch-Balanced Selection. The selected path is converted into a Route Session, generating short-term ($M^{route}_t$) and long-term decaying ($M^{hist}_t$) corridor masks.
3. Disambiguation Questions: Triggered for "Pending" candidates that are close and stable. The oracle's Yes/No answer updates the object label to Confirmed or Rejected.

C. Target Perception and Instance Reasoning

To handle open-vocabulary detection noise, SAIN employs an Entropy-Aware VQA filter. It queries a Vision-Language Model (VLM) with a mask and the target prompt, analyzing the next-token logits to estimate uncertainty. Only detections with low entropy and high confidence are accepted.
Accepted candidates undergo Similarity-Based Instance Verification:

1. The VLM generates a descriptive context for the candidate.
2. An LLM refines this evidence against the accumulated Target Facts ($F_{tar}$), filtering out ambiguous attributes.
3. A similarity score is computed between the refined candidate evidence and the target facts.
4. Based on thresholds, the candidate is labeled as Rejected, Pending, or Confirmed.

D. Unified Navigation Policy

The policy consumes the full structured state to rank frontiers and select actions. The frontier score $s(\xi)$ is a multiplicative combination of:

• Value Score ($s_{val}$): Semantic relevance of the frontier.
• Route/History Priors: Masks derived from grounded route answers.
• Room Prior: Relevance of the frontier's room to the target.

If a candidate is Pending, the agent navigates to it for verification. If Confirmed, the agent switches to PointGoal navigation to reach the target.

3. Key Contributions

1. SAIN Framework: A zero-shot IIGN framework that converts active dialogue into persistent navigation states (target evidence, route guidance, candidate reasoning) rather than transient context.
2. Structured Memory Design: A novel architecture coupling value, room, graph, and object maps with three complementary question types (information, route, disambiguation) to support long-horizon decision-making.
3. Performance Gains: Demonstrated improvements on the VL-LN IIGN benchmark without task-specific policy training, establishing dialogue-to-state conversion as an effective mechanism for interactive instance search.

4. Experimental Results

The framework was evaluated on the VL-LN IIGN benchmark in the Habitat simulator and validated on a real wheeled robot.

• Benchmark Performance: Compared to the strongest dialogue-enabled baseline (VLLN-D), SAIN improved:
  • Success Rate (SR): from 20.2 to 25.4.
  • Success weighted by Path Length (SPL): from 13.07 to 14.17.
  • Navigation Error (NE): reduced from 8.84 to 8.06.
• Ablation Studies:
  • Removing the Similarity Verifier caused a sharp drop in SPL (11.02) and increased steps, indicating that persistent instance state is crucial for efficiency.
  • Removing the Route Bonus caused the largest drop in SR (to 19.0) among value-prior ablations, highlighting the importance of route-grounded memory.
  • Question Budget Analysis: A single information question recovered most of the dialogue benefit (SR 20.6), but the full budget (unlimited route/disambiguation) yielded the best absolute performance.
• Real-World Deployment: SAIN successfully executed the full pipeline on a physical robot (Intel RealSense D435i, RTX 4090M GPU) to find a specific wooden table, demonstrating the transferability of the dialogue-to-state approach beyond simulation.

5. Significance and Claims

The paper claims that treating dialogue as persistent navigation state rather than transient language context or direct action commands yields a stronger zero-shot framework for interactive instance search.

• Zero-Shot Capability: SAIN achieves state-of-the-art results without training a task-specific policy, relying instead on the structured compilation of oracle answers into internal state representations.
• Long-Horizon Effectiveness: The framework effectively resolves the "forgetting" problem in long-horizon navigation by grounding answers into topological corridors and object labels that persist across observations.
• Limitations and Future Work: The authors note that final instance-level verification remains the primary bottleneck, with ambiguity accounting for over 50% of failures in the best configuration. Future work is suggested to focus on stronger candidate verification and finer disambiguation questions for high-similarity candidates.