← Latest papers
💻 computer science

GLAM: Training a latent world model over global spatiotemporal memory for active exploration and navigation

The paper presents GLAM, a goal-conditioned latent world model trained on global spatiotemporal memory that predicts future map representations and navigation waypoints to enable improved active exploration and semantic navigation, demonstrated by the GLAM NAV system outperforming the BSC-Nav baseline on HM3D-ObjectNav tasks.

Original authors: I-Tak Ieong, Ruizhi Feng, Zhaoyang Lu, Yifei Cao, Jiayao Zhao, Leon Li, Senhua Zhu, Wenbo Ding

Published 2026-09-16
📖 7 min read🧠 Deep dive

Original authors: I-Tak Ieong, Ruizhi Feng, Zhaoyang Lu, Yifei Cao, Jiayao Zhao, Leon Li, Senhua Zhu, Wenbo Ding

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot entering a room it has never seen before, tasked with finding a specific object, like a red chair, that it cannot see from where it stands. To succeed, the robot cannot simply react to what is immediately in front of its camera. It must build a mental picture of the space as it moves, remember where it has been, and guess what lies around the next corner. This ability to hold a map in memory, anticipate how that map will change as the robot moves forward, and use that anticipation to plan the next step is the core challenge of active exploration. For years, researchers have tried to teach machines this skill by having them reconstruct the world in high-definition detail, pixel by pixel, or by relying on pre-made maps. However, a new approach suggests that a robot does not need to see every brick and shadow to navigate; it only needs to understand the structure of the space and the likely path to its goal.

A team of researchers has developed a system called GLAM, which stands for a goal-conditioned latent world model trained over global spatiotemporal memory. In simpler terms, this is a navigation system that learns to predict the future layout of a room and the best path to a destination without needing to recreate the visual world in perfect detail. Instead of trying to generate a new video frame of what the room will look like in the next second, the system predicts a compressed, abstract representation of the map and a hidden plan for where to go next. This approach is inspired by how biological brains, particularly the hippocampus, organize spatial memories and use them to simulate future scenarios. The researchers built a complete navigation system around this model, named GLAM NAV, which combines online mapping, memory retrieval, and predictive planning into a single loop.

The system works by constantly updating a sparse, digital memory of the environment as the robot moves. This memory is not a full 3D photograph but a collection of key landmarks and spatial relationships. When the robot needs to find a target, it uses its current position and the memory of what it has already seen to ask a simple question: "If I move in this direction, what will the map look like, and will I be closer to my goal?" The GLAM model answers this by predicting two things simultaneously. First, it forecasts how the map tokens—the abstract building blocks of its memory—will evolve as the robot explores. Second, it predicts a latent waypoint, which is a hidden representation of the next step the robot should take. These predictions happen in a shared space, meaning the model learns to link the changing map directly to the necessary movement, all without ever trying to reconstruct the raw visual images of the future.

To train this system, the researchers did not send robots out into the real world to make mistakes. Instead, they used a high-fidelity simulator called Habitat, which contains hundreds of detailed 3D scans of real indoor environments like apartments and offices. They replayed expert trajectories—paths taken by a perfect, computer-controlled agent that knew exactly where the objects were—and sliced these paths into thousands of training samples. The model learned to look at a history of map tokens and a goal, and then predict the future map and the next waypoint. Crucially, the system was taught to ignore the task of recreating the visual world. It focused entirely on predicting the structure of the space and the plan to navigate it. Before training the main model, the researchers also pre-trained a separate component to understand how to translate these hidden waypoint plans into actual physical movements, ensuring that the predictions could be turned into real commands.

When tested in the simulator, the results showed that this predictive approach worked better than previous methods that relied on different types of memory structures. On a specific set of fifty percent of the test scenes, the new system, GLAM NAV, successfully found the target object in 86.89 percent of the attempts, a significant improvement over the 78.50 percent success rate of the baseline system it was compared against. It also managed to reach the target more efficiently, with a metric called Success weighted by Path Length rising from 47.70 percent to 48.35 percent. These numbers suggest that by predicting the future structure of the map and the path together, the robot became more reliable at finding its way, even when the target was initially out of sight. The system did not just memorize a route; it learned to anticipate the layout of the environment and adjust its plan based on that anticipation.

To prove that this was not merely a result of the simulator, the researchers deployed the system on a physical robot, a wheeled humanoid with dual arms, in a real office environment. In the first real-world test, the robot was asked to find a potted plant in a room it had never seen, with no pre-built map provided. As it moved, it built its own memory of the space, connecting new visual observations to its growing map. It successfully navigated to the plant, demonstrating that the system could construct a useful spatial memory from scratch in a real, physical setting. In a second test, the robot was asked to recall where it had seen the plant after the object had moved out of view. The system retrieved the stored location from its memory and successfully navigated back to that spot. These demonstrations confirmed that the abstract predictions made by the model could guide a real robot through a real environment, using memory to bridge the gap between what is seen and what is needed.

Despite these successes, the researchers are careful to note the boundaries of their work. The system learns from the paths taken by expert agents in the simulator, so it is most effective when the robot's behavior stays within the patterns it has seen during training. It is not a general simulator that can predict the outcome of any random action; rather, it is a goal-directed tool that estimates the most likely map and path for a specific objective. The model also produces a single, definite prediction rather than a range of possibilities, meaning it does not explicitly calculate the uncertainty of its guesses. If the robot encounters a layout that is very different from what it learned, or if its sensors drift and lose track of its position, the system may struggle. The researchers emphasize that the robot still relies on real-time observations to verify its location and avoid collisions, using the predictions only to guide its search.

The work represents a shift in how we think about robot navigation. Instead of trying to build a perfect, high-resolution replica of the world inside a computer, the system learns to work with a functional, abstract version of the map that is sufficient for planning. By treating the prediction of the future map and the planning of the next move as a single, connected task, the researchers have created a system that is both more reliable at finding targets and capable of operating in real-world settings. The findings suggest that for a robot to explore effectively, it does not need to see everything; it needs to understand the structure of the space well enough to imagine what comes next.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →