STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation
STEGNav is a training-free framework that enhances multimodal lifelong object navigation by extending conventional scene graphs into spatio-temporal event graphs, which integrate query-conditioned spatial instance grounding with trajectory-aware temporal memory to improve instance distinction, frontier representation, and cross-subtask experience reuse.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a robot entering a room it has never seen before, tasked with finding a specific object. In the simplest version of this challenge, the robot is told to find "a chair." But in the real world, rooms are full of chairs, and the robot needs to know which one is the right one based on a description like "the red chair near the window" or a photo of a specific piece of furniture. This is the realm of embodied navigation, where artificial agents must understand their surroundings, interpret complex instructions, and move through space to achieve a goal. For years, researchers have tried to teach robots to build mental maps of these environments, often using a tool called a scene graph. Think of a scene graph as a digital list that records what objects are in a room and how they relate to one another. While useful, these traditional lists have a blind spot: they often treat all chairs as the same generic item and forget the path the robot took to get there. They are static snapshots of a room rather than a record of the journey, which causes robots to get confused between identical objects or to wander in circles, revisiting areas they have already explored.
A team of researchers at Nanjing University has developed a new approach to solve these problems, creating a system they call STEGNav. Instead of relying on a static list of objects, this system builds a dynamic, event-driven map that remembers both the layout of the room and the history of the robot's movements. The researchers realized that to navigate effectively over a long period, an agent needs to understand not just what is in front of it, but also how it got there and what it has already tried. Their solution involves two main improvements to how the robot perceives the world. First, they added a spatial layer that helps the robot distinguish between individual items. If the robot is looking for a specific table, this system helps it tell the difference between the table in the dining room and the one in the kitchen, even if they look similar. It also connects these objects to the empty spaces around them, allowing the robot to see not just the furniture, but the open paths it can take to reach them.
The second improvement is a memory system that tracks the robot's recent decisions and past successes. This system works like a dual-layered notebook. One part of the notebook records the immediate past, keeping a close watch on the last few steps the robot took, the paths it walked, and the areas it explored. This prevents the robot from getting stuck in loops or wasting time revisiting places it has already checked. The other part of the notebook stores verified successes from earlier tasks. If the robot successfully found a specific object in a previous task, that information is saved and linked back to the current map, helping the robot remember where similar items might be found in the future. By combining these spatial and temporal layers, the robot creates a rich, living representation of its environment that evolves as it moves.
To test this new system, the researchers put it through a series of rigorous challenges in a simulated environment designed to mimic real-world navigation. They used a benchmark called GOAT-Bench, which requires the robot to complete a sequence of different tasks, such as finding a specific object based on a photo, a sentence description, or a category name, all within the same continuous journey. The results were significant. The new system successfully completed 66.3% of these complex, multi-step navigation tasks, a notable improvement over previous methods that struggled with the same challenges. It also managed to find the shortest paths more often, scoring 39.7 on a metric that balances success with the efficiency of the route taken. When tested on a different, larger set of environments called HM3D, the system continued to perform well, achieving success rates of 64.0% and 69.4% on two different versions of the test.
The researchers analyzed where the system succeeded and where it still faced difficulties to understand why it worked so well. They found that the biggest gains came from the system's ability to stop the robot from exploring the same areas repeatedly and from confusing one object for another. By explicitly remembering which paths had been taken and which objects had already been checked, the robot avoided many of the dead ends that plagued older systems. The analysis showed that the new method reduced the number of times the robot got lost or confused by nearly half compared to the previous best methods. While the system still faces some challenges with the very low-level mechanics of moving and stopping, the core logic of navigation—knowing where to go and what to look for—has been substantially improved. This work demonstrates that by giving robots a better way to remember their journey and distinguish between similar objects, we can make them much more reliable partners in exploring the unknown.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.