The Objective Is the Bottleneck: Latent World Models Encode What Their Planners Cannot Use

Imagine you are teaching a robot to navigate a maze. To do this, the robot builds a "mental map" of the world inside its brain. This map isn't a picture; it's a compressed, mathematical summary of what the world looks like at any given moment. Scientists call this a "latent world model." The robot learns this map by watching videos of itself moving and trying to predict what happens next. If it can predict the future accurately, we assume it understands the world. Once it has this map, the robot uses a "planner"—a decision-making algorithm—to figure out the best path to a goal, like finding a treasure chest. The big question in this field has always been: if the robot fails to reach a goal far away, is it because its mental map is blurry and inaccurate, or is it because its decision-making tool is broken? This paper dives into that mystery, testing whether the problem lies in the robot's vision or its compass.

The researchers in this paper investigated a specific robot navigating a simple two-room environment. They found that the robot's "mental map" was actually incredibly sharp and accurate, even for looking far into the future. However, the robot kept failing to reach goals that were far away. The surprising discovery is that the robot wasn't failing because it couldn't see the path; it was failing because the "compass" it was using to measure distance was broken. The robot was trying to minimize a specific mathematical score (the squared distance between its current mental state and the goal), but this score stopped making sense after a certain point. It was like a GPS that tells you you're getting closer to your destination when you are actually driving in the opposite direction.

The paper shows that the robot's internal map could still predict the state of the world 75 steps ahead with very little error—far beyond the 25 steps the robot ever tried to plan for. The problem wasn't the map; it was the objective function, the rule the planner used to judge success. The researchers found that the standard way of measuring distance in the robot's mind "saturated" and then "inverted." This means that once the goal was more than about 120 units away, moving away from the goal actually lowered the robot's cost score, tricking it into thinking it was making progress. The information about where the goal was and how to get there was still fully present in the robot's brain, but the robot's planner was ignoring it because it was looking at the wrong metric.

To fix this, the researchers didn't retrain the robot or give it more powerful hardware. Instead, they simply swapped out the broken compass for a new one. They taught the planner to care about "reachability" (can I actually get there?) rather than just "proximity" (how close does it look?). They did this by training a small helper to predict how many steps it would take to get from one frame to another, rather than just measuring the distance between two points in a mathematical space. This new approach was a massive success. On the original robot's settings, the success rate for reaching goals 100 steps away jumped from a dismal 26% to a stellar 98%. Even more impressively, the robot could reach 92% of those far-away goals with only one-third of the usual time budget.

However, this repair isn't a magic bullet that works everywhere. When the researchers tried the same fix on the original authors' released weights, it only improved success from 14% to 34%, falling short of the 70% achieved by a simpler linear method. The reason for this failure is that the original model's "mental map" drifts too far from reality when imagining the future. The new compass was trained on real video frames, but the planner was scoring imagined frames. Because the original model's predictions were so inaccurate, the imagined frames were too different from the real ones for the new compass to work correctly. This highlights a crucial rule: a learned planning cost must be trained on the exact type of "imagined" data the planner will actually use, or it will fail when the predictor drifts.

The study also revealed a strange quirk: the "best" planner wasn't the one with the most accurate map. In fact, a planner using a cost function that was worse at predicting exact spatial distance actually planned better. This is because the new cost function learned to charge extra for crossing a wall, whereas the old one didn't care about walls at all. The old planner thought two rooms were close because they were physically near each other, ignoring the door. The new planner understood that you have to go through the door, making it a better navigator. The paper concludes that for long-horizon planning, the bottleneck isn't the ability to predict the future, but the ability to define what "getting there" actually means. If your compass points the wrong way, no amount of perfect vision will help you find your destination.