🤖 AI

ProCompNav: Proactive Instance Navigation with Comparative Judgment for Ambiguous User Queries

ProCompNav is a two-stage framework that resolves ambiguous instance navigation queries by iteratively constructing a candidate pool and employing binary comparative questions to efficiently distinguish the target from distractors, thereby outperforming existing methods in success rate while significantly reducing user response length.

Original authors: Junhyuk Kwon, Seungjoon Lee, Hyejin Park, Kyle Min, Jungseul Ok

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junhyuk Kwon, Seungjoon Lee, Hyejin Park, Kyle Min, Jungseul Ok

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a robot tasked with finding a specific object in a giant, messy warehouse full of identical-looking items. The boss (the user) gives you a vague command: "Find the cabinet."

The problem? There are 50 cabinets in the warehouse. They all look somewhat similar. Some are in bathrooms, some in bedrooms, some are wood, some are metal. If you just guess the first one you see, you might grab the wrong one. If you ask the boss, "What does the cabinet look like?" and they give you a long, rambling description, it takes forever, and you might still get confused because that description fits several cabinets.

This paper introduces a new way for robots to solve this problem, called ProCompNav. Think of it as a game of "20 Questions" played with a twist.

Here is how it works, broken down into simple steps:

1. The Old Way: "Guess and Check" (Independent Matching)

Imagine a robot using the old method. It sees a cabinet and asks, "Is it blue?" The boss says, "Yes." The robot sees another cabinet, asks, "Is it blue?" The boss says, "Yes."

The Flaw: The robot keeps collecting facts (blue, near a mirror, wood) and tries to match them to one cabinet at a time.
The Result: It often picks a "distractor" (a wrong cabinet) too early because that wrong cabinet also happens to be blue and near a mirror. The robot gets stuck in a loop of asking long, confusing questions, or it gives up and picks the wrong thing.

2. The New Way: "The Sorting Hat" (ProCompNav)

ProCompNav changes the strategy entirely. Instead of trying to describe the one right cabinet, it focuses on sorting the whole group of cabinets.

Step 1: Gather the Crowd (Pool Construction)
First, the robot doesn't decide on anything yet. It runs around the warehouse and finds all the cabinets it can. It puts them all in a mental "candidate pool." Now, instead of looking for one needle in a haystack, it has a pile of 10 needles and needs to find the right one.

Step 2: The Magic Split (Comparative Judgment)
Instead of asking, "What color is the target?", the robot looks at the pile and asks a comparative question designed to split the group in half.

Bad Question: "Is the target wood?" (Maybe all 10 are wood. This doesn't help.)
ProCompNav Question: "Is there a red box next to the cabinet?"
- Group A (The "Yes" group): 3 cabinets have a red box next to them.
- Group B (The "No" group): 7 cabinets do not have a red box.

Step 3: The Binary Cut
The robot asks the user a simple Yes/No question: "Does the cabinet you want have a red box next to it?"

If the user says "Yes": The robot instantly throws away the 7 cabinets in Group B. It only keeps the 3 in Group A.
If the user says "No": The robot throws away the 3 cabinets in Group A. It keeps the 7 in Group B.

Step 4: Repeat until One Remains
The robot repeats this process. It looks at the remaining group, finds a new feature that splits them (e.g., "Is there a TV on top?"), asks a Yes/No question, and cuts the group in half again.

Round 1: 10 cabinets $\rightarrow$ 3 left.
Round 2: 3 cabinets $\rightarrow$ 1 left.
Done! The robot has found the target.

Why is this better?

The paper claims this method is a huge improvement for three main reasons:

It avoids "Premature Decisions": By waiting to gather a group before making a choice, the robot doesn't accidentally pick a wrong cabinet just because it was the first one it saw.
It's easier for the user: Instead of writing a long paragraph describing the cabinet ("It's a dark oak cabinet with silver handles, sitting in a room with blue walls..."), the user just has to answer "Yes" or "No". This is much faster and less tiring.
It's smarter about questions: The robot doesn't ask random questions. It specifically looks for a question that will cut the number of suspects in half, like a detective narrowing down a list of suspects by checking who was at the scene.

The Results

The researchers tested this on computer simulations (like a video game world).

Success Rate: ProCompNav found the right object more often than previous methods, even when the user gave very vague instructions.
Efficiency: It required far fewer questions and much shorter answers from the user compared to the old methods.
Versatility: It even worked well in a "non-interactive" setting (where the robot reads a detailed description but still has to find the right object among many), proving that this "compare and split" logic is a powerful tool for finding things.

In short: ProCompNav stops the robot from guessing and starts it from sorting. It turns a confusing search into a simple game of elimination, making it faster for the robot and easier for the human.

Technical Summary: ProCompNav

Problem Definition

The paper addresses Collaborative Instance Navigation (CoIN), a task where an embodied agent must navigate an unknown 3D environment to locate a specific target instance (e.g., "the cabinet") based on an ambiguous natural language query (e.g., "Find the cabinet"). Unlike standard instance navigation which assumes a detailed, unique description is provided upfront, CoIN deals with scenarios where the target cannot be distinguished from same-category distractors using minimal initial information.

Existing approaches typically employ an independent matching strategy, where the agent asks the user about the target's attributes (e.g., color, nearby objects) and scores encountered candidates based on accumulated information. The authors argue this approach suffers from two main flaws:

Premature Decisions: Agents often commit to a distractor that shares attributes with the true target before sufficient evidence is gathered.
Inefficient Disambiguation: Even when multiple candidates are collected, questions are often derived from individual candidates rather than designed to distinguish between the entire pool. This leads to lengthy, open-ended user responses and failure to separate visually similar distractors.

Methodology: ProCompNav

The authors propose Proactive Instance Navigation with Comparative Judgment (ProCompNav), a two-stage framework that shifts the paradigm from "describing the target" to "pruning the candidate pool."

1. Pool Construction Stage

The agent explores the environment to build a candidate pool $C$ of $N$ distinct instances of the requested category.

Representation: Each candidate is represented by a multi-view collage image and a multi-view description generated by a Multimodal Large Language Model (MLLM).
Aggregation: The agent aggregates 3D point clouds and RGB views from different viewpoints to create a comprehensive description of each candidate, capturing attributes that might be missed from a single view.
Trigger: Once the pool size reaches a minimum threshold ( $N_{min}$ ), the agent transitions to the comparison stage.

2. Recursive Comparison Stage (RCJ)

Instead of asking open-ended questions, RCJ iteratively prunes the candidate pool using binary yes/no questions based on Comparative Judgment.

Core Set Selection: At each round $t$ , the current active pool $U_t$ is divided into a Core Set ( $G_c$ ) and a Remainder Set ( $G_r$ ). The core set is selected to maximize intra-set similarity (using a greedy peeling algorithm on text and visual embeddings), ensuring $G_c$ contains semantically similar instances.
Discriminative Attribute (DA) Discovery: The system identifies an attribute $a^*_t$ $a_{t}^{*}$ that is present in $G_c$ $G_{c}$ but absent in $G_r$ $G_{r}$ .
- Attributes are extracted from $G_c$ using an LLM.
- A Natural Language Inference (NLI) model acts as a verifier to score the entailment of each attribute for every instance, quantifying the contrast between $G_c$ and $G_r$ .
- The attribute maximizing the expected difference in entailment scores between the two sets is selected.
Refinement: The remainder set is refined by re-evaluating candidates against the selected DA to ensure the split is clean.
Interactive Pruning: The agent asks the user: "Does the target have [attribute]?"
- If Yes: The pool is updated to $G_c$ .
- If No: The pool is updated to $G_r$ .
- If the pool size becomes 1, the target is identified. If the pool becomes empty (indicating the target was not in the initial pool), the agent resumes exploration to add more candidates.

Key Contributions

Comparative Judgment Strategy: The paper introduces a shift from independent matching to a "collect-then-compare" pipeline. This reframes disambiguation as a process of selecting questions that split the candidate pool, rather than gathering facts about a single hypothesized target.
Recursive Comparative Judgment (RCJ): A specific algorithm that iteratively extracts discriminative attribute-value pairs to prune the candidate pool using binary questions. It requires only that an attribute splits the current pool into two non-empty groups, not that it uniquely identifies the target immediately.
Reduced User Burden: By asking binary questions that eliminate multiple candidates simultaneously, the method significantly reduces the length of user responses compared to open-ended descriptions.

Experimental Results

The method was evaluated on CoIN-Bench (interactive, ambiguous queries) and TextNav (non-interactive, detailed queries).

CoIN-Bench (Interactive): ProCompNav achieved the highest Success Rate (SR) across all splits (Seen, Seen Synonyms, Unseen) compared to interactive baselines like AIUTA and non-interactive baselines like 3D-Mem.
- SR Improvement: Achieved 23.7% SR on Val Seen, outperforming the best interactive baseline (AIUTA*) at 10.5%.
- Efficiency: Substantially reduced the average Response Length (RL) from ~109.5 tokens (AIUTA) to 4.2 tokens, and reduced the Number of Questions (NQ) to 2.2.
TextNav (Non-Interactive): When adapted to a non-interactive setting (extracting attributes directly from detailed descriptions), ProCompNav achieved a state-of-the-art SR of 28.5%, outperforming other training-free methods.
Ablation Studies: Results confirmed the necessity of multi-view aggregation, the NLI-based verifier (over LLM-only scoring), and the similarity-based core set selection (outperforming KMeans).

Significance and Claims

The paper claims that ProCompNav demonstrates that comparative judgment is a broadly useful mechanism for instance-level navigation, particularly when distractors are visually similar.

Robustness: The method is more robust against premature decisions because it defers the final decision until a candidate pool is constructed and compared.
Generalization: The core mechanism of deferring decisions and pruning via discriminative attributes is effective even in non-interactive settings where detailed descriptions are available, suggesting that candidates should always be compared against each other rather than scored independently.
User Experience: The approach significantly lowers the cognitive and verbal burden on users by replacing long descriptions with short binary confirmations.

The authors note limitations, including the current reliance on simulation and MLLM user simulators, and the potential inefficiency of the pool construction stage in environments with very few same-category instances. They suggest future work in real-world robotic deployment and dynamic pool size adaptation.