💻 computer science

User-Centric Object Navigation: A Benchmark with Integrated User Habits for Personalized Embodied Object Search

This paper introduces User-centric Object Navigation (UcON), the first large-scale benchmark that incorporates user-specific object placement habits to evaluate and improve personalized embodied object search, demonstrating that integrating these habits significantly enhances navigation success rates compared to existing methods.

Original authors: Hongcheng Wang, Jinyu Zhu, Hao Dong

Published 2026-02-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hongcheng Wang, Jinyu Zhu, Hao Dong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've lost your sunglasses. You know you wore them yesterday while eating lunch. If you ask a standard robot to find them, it might say, "Okay, sunglasses are usually on a table or a shelf," and start checking the living room. But what if your specific habit is to leave them on the kitchen counter because that's where you eat? A standard robot, relying only on general rules, would likely miss them.

This paper introduces a new way to teach robots to find things by learning your specific habits, not just general rules. Here is a simple breakdown of their work:

1. The Problem: The "One-Size-Fits-All" Robot

Most robots today are like tourists who know the general rules of a city (e.g., "Beds are in bedrooms"). They work great in hotels or showrooms where everything is perfectly organized. But real homes are messy and personal.

The Analogy: Imagine trying to find a friend's house. A tourist might look for a house with a red door (the general rule). But your friend might live in a house with a blue door because they hate red. If the robot only knows the "red door rule," it will never find your friend.
The Issue: Current robots don't know that you always leave your keys in the fridge or that you read your newspaper on the sofa, not at the dining table.

2. The Solution: The "Habit Library" (UcON)

The researchers built a new testing ground called UcON (User-centric Object Navigation). Think of this as a massive library of "User Habits."

Instead of just saying "Where are the books?", the robot is given a cheat sheet that says: "This specific user reads on the sofa," or "This user keeps bananas in the fridge to keep them fresh."
They created over 22,600 different habits for 489 different objects. It's like giving the robot a diary of how a specific person lives, rather than just a map of a generic house.

3. The Challenge: Too Much Information

Giving the robot a whole diary is overwhelming. If you ask a robot to find "newspapers" and you give it a list of 300 habits (including how they store coffee, where they put their shoes, and what they eat for breakfast), the robot gets confused. It's like trying to find a needle in a haystack while someone is shouting 300 different facts at you.

The Bottleneck: The robot's "brain" (a Large Language Model) gets tired and makes mistakes when the instructions are too long.

4. The Trick: The "Habit Retriever"

To fix the confusion, the authors added a special tool called a Habit Retrieval Module.

The Analogy: Imagine you are looking for your lost keys. Instead of reading your entire 500-page diary, you have a smart assistant who instantly flips to the page that says, "I usually leave my keys by the door." The assistant ignores the pages about what you ate for lunch last Tuesday.
How it works: When the robot needs to find an object, this module quickly scans the "Habit Library" and pulls out only the habits relevant to that specific object. This makes the robot's job much easier and smarter.

5. What They Found

They tested this on robots (both in computer simulations and with a real robot dog in a house).

Without Habits: The robots were often wrong because they relied on "common sense" (e.g., "Books are in the library") which didn't match the user's actual habits.
With Habits: The robots got much better at finding things.
With the "Retriever": The robots performed the best. By filtering out the noise and focusing only on the relevant habits, they found objects faster and more accurately.

The Bottom Line

This paper shows that for robots to be truly helpful in our homes, they can't just follow general rules. They need to learn our specific quirks. However, they can't read our entire life story at once; they need a smart way to look up the specific habit that matters for the task at hand. The authors proved that when robots do this, they become much better at finding lost items.

Technical Summary: User-Centric Object Navigation (UcON)

Problem Statement

Existing Object Navigation (ON) benchmarks primarily rely on general scene priors, assuming that object placement adheres to common-sense norms (e.g., beds are in bedrooms, sofas near TVs). However, in real-world household environments, object placement is often dictated by individual user habits rather than universal rules. For instance, a user might habitually leave a book on a sofa rather than in a study. This discrepancy limits the adaptability of navigation agents in personalized settings. Current state-of-the-art (SOTA) methods, which often leverage Large Language Models (LLMs) for reasoning based on scene priors, fail to account for these specific, habit-driven placements, leading to significant performance degradation when objects are not located where common sense dictates.

Methodology

1. The UcON Benchmark

The authors introduce User-centric Object Navigation (UcON), a novel benchmark designed to evaluate and improve object search in personalized environments.

Scale and Scope: UcON covers 489 distinct object categories and incorporates approximately 22,600 user habits. It is the first benchmark to explicitly formalize and evaluate habit-conditioned object navigation at this scale.
Task Generation: The benchmark utilizes a three-step pipeline using GPT-4 to generate natural language user habits and corresponding object placements. These placements are validated for physical plausibility and semantic consistency within the Omnigibson simulator.
Environment: Each episode begins with a "habit-shaped scene" where specific objects are repositioned according to a sampled User Habit Knowledge Base (UHKB). The agent must navigate to a target object using a set of actions (Move, Rotate, Look, Open, Done) within a 300-step limit.
Challenge: The core difficulty lies in resolving spatial conflicts where multiple habits may exist, and the agent must use exploration evidence to prioritize the correct habit-driven location over general scene priors.

2. Habit Retrieval Module (HRM)

To address the limitations of LLMs in processing long prompts containing hundreds of habits, the authors propose a Habit Retrieval Module (HRM).

Mechanism: Inspired by Retrieval-Augmented Generation (RAG), the HRM uses the BGE-M3 model to calculate similarity between a query template (e.g., "Retrieve habits about {TargetObject}") and the stored habits in the UHKB.
Function: It selects the top- $p$ most relevant habits to include in the prompt for the foundation model. This simplifies the reasoning process by filtering out irrelevant information and focusing the LLM on task-relevant user behaviors.

3. Experimental Setup

Simulations: Conducted on 22 scenes in Omnigibson.
Real-World Validation: Experiments were performed on a Unitree GO2 Edu robot equipped with an Astra Pro Plus RGB-D camera, running on consumer-grade hardware (RTX 3090 workstation).
Baselines: The study evaluates SOTA methods including PixelNav, LGX, L3MVN, VTN, and ZSON. These methods were modified to accept user habits as input, with some components (like object detectors) replaced by ground truth to establish upper bounds.

Key Results

Performance Degradation of SOTA: Current SOTA methods exhibit substantial performance drops in the UcON benchmark compared to traditional ON benchmarks. End-to-end methods (VTN, ZSON) and foundation model-based approaches (PixelNav, LGX) struggle because their reasoning is grounded in common sense, which is violated by habit-driven object placement.
Benefit of User Habits: Integrating user habits significantly improves the Success Rate (SR) and Success weighted by Path Length (SPL) for most methods. For example, the LGX method with GPT-4 saw its SR increase from 25.1% (No Habits) to 27.8% (Full Habits) and 30.6% (Retrieval Habits).
Effectiveness of HRM: The proposed Habit Retrieval Module outperforms even the "Ground Truth" habit setting (where only the direct habit for the target object is provided). The authors hypothesize this is because the retrieval mechanism surfaces indirectly related habits (e.g., habits about a "breakfast table" when searching for a "tea bag") that provide crucial contextual landmarks, enabling more efficient exploration strategies.
Real-World Viability: Case studies in both small and large real-world environments demonstrate that agents utilizing retrieved habits can successfully locate objects obscured from immediate view by reasoning about user behavior (e.g., searching near a sofa because the user reads there), whereas agents without habits fail or search aimlessly.

Significance and Claims

The paper claims that UcON is the first comprehensive evaluation framework for habit-integrated navigation, uniquely combining a wide range of object categories with personalized behavioral patterns.

The authors emphasize that:

User habits are a critical signal: Effective home service agents must not only recognize but leverage user habits to facilitate efficient object search, especially when objects defy common-sense placement.
Reasoning limitations: Relying solely on LLMs for reasoning over long lists of habits is insufficient; retrieval mechanisms are necessary to enhance inference accuracy, particularly for smaller models.
Future Directions: While the work focuses on habit utilization (reasoning over a known habit set), it identifies habit acquisition (learning habits from long-term observation) and context-aware retrieval as important directions for future research.

The paper concludes that integrating user habits and utilizing retrieval modules are essential steps toward creating robust, personalized embodied agents capable of serving humans in dynamic, real-world household environments.