💻 computer science

Do Better Imagined Rollouts Mean Better Robot Control? A Controlled Study of World-Model Evaluation Under Feedback

This paper demonstrates that for robotics applications relying on feedback, offline predictive model evaluations using open-loop rollouts are less reliable than trajectory replay or closed-loop testing, as they often fail to correlate with actual control performance unless the evaluation schedule explicitly mirrors the system's measurement-update pattern.

Original authors: Dharini Raghavan, Amritpal Singh

Published 2026-09-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dharini Raghavan, Amritpal Singh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots that move through the real world rely on a constant, invisible conversation between their eyes, their wheels, and their brains. To navigate, a robot must first guess where it is, then decide how to move, and finally check its surroundings to see if the guess was right. This cycle happens again and again, thousands of times an hour. If the robot's internal guess is slightly wrong, the correction it makes might send it off course. If the guess is wrong in a different way, the robot might correct itself perfectly. For years, engineers have tried to judge how good a robot's "brain" is by testing it in a quiet, controlled environment where it predicts the future without any new information. They ask the model to imagine a path forward and see how far off it gets. But this paper asks a simple, crucial question: does being good at imagining the future in a vacuum actually mean a robot will be good at driving a car or walking through a forest when real sensors are constantly updating its view?

The researchers at Georgia Tech and Emory University set out to answer this by building a controlled experiment with a small, wheeled robot. They gave the robot a specific path to follow, a set of wheels that sometimes slipped, and a gyroscope that drifted slightly off course. To help the robot know where it was, they placed known landmarks around the room that the robot could see occasionally, but not constantly. The robot had to use its wheel sensors to guess its position between these sightings. The team tested six different ways the robot could estimate its location. Some relied purely on math and wheel data, while others used learned patterns to fix the errors in those math models. They then compared three different ways of testing these robots. First, they replayed a recorded journey where the robot saw every landmark exactly as it happened in the past. Second, they asked the robot to imagine a twenty-step journey into the future without seeing any landmarks at all, relying only on its internal guess. Third, they let the robot drive in real-time, where its guesses directly controlled the wheels, and it received landmark updates whenever they were available.

The results revealed a surprising disconnect. When the researchers looked at how well the robots performed in the real-time driving test, the method that best predicted the winner was the one where the robot simply replayed a past journey with all the landmark updates included. This method correctly identified the best robot in most cases. However, the popular method of asking the robot to imagine a long, twenty-step journey without any new information was much worse at predicting the real-world winner. In fact, this "imagined rollout" method picked the wrong robot as the best performer in eighteen out of twenty-four different testing scenarios. The paper shows that a model can be very good at predicting where a robot will be if it never gets corrected, but that skill does not translate to a robot that is constantly being corrected by new sensor data. The ability to drift accurately in a vacuum is not the same as the ability to navigate with help.

The study went deeper to understand why this happened. The researchers realized that the problem wasn't just the length of the imagined journey, but the lack of updates during that journey. When they tested the robots with a long imagined path but allowed them to receive a sensor update at every single step, the test became accurate again. It was only when they combined a long prediction time with a total lack of updates that the test failed to match reality. This suggests that for robots operating in a feedback loop—where they act, sense, and correct—the way we test them must mimic how they actually work. If a robot gets frequent updates in the real world, testing it by asking it to guess for a long time without any updates is misleading.

The team also explored whether they could train the robots to be better at these long, uncorrected guesses. They trained some of the learning-based robots to handle longer periods without seeing any landmarks. In some cases, this helped. For the robots that used a strong mathematical foundation to start, training them to handle long gaps reduced their errors significantly, cutting the distance they wandered off course from over one and a half meters down to just over one meter. However, this improvement was not universal. For robots that started with a weaker mathematical foundation, training them to handle long gaps did not help at all; in some cases, it made them slightly worse. This finding suggests that simply exposing a robot to more difficult training conditions does not guarantee it will become more robust. The underlying structure of the robot's brain matters just as much as the training it receives.

Ultimately, the paper argues that the way we evaluate predictive models in robotics needs to change. We cannot just measure how far off a model gets after a long time without data. Instead, we must measure how well the model performs under the specific schedule of updates it will face in the real world. If a robot receives a camera image or a sensor reading every second, its evaluation should include those updates every second. If we test it by asking it to guess for a minute without any help, we are testing a different skill entirely. The most useful benchmark is one that reflects the rhythm of the robot's actual life: the act of moving, the arrival of new information, and the correction of the path. By aligning our tests with the reality of how robots operate, we can choose the right tools for the job and build machines that truly understand the world they move through.

Technical Summary: Do Better Imagined Rollouts Mean Better Robot Control?

Problem Statement

Predictive models (world models) are increasingly central to robotics for state estimation, planning, and control. However, these models are typically evaluated using offline metrics, such as multi-step rollout accuracy or state-prediction error, often under open-loop conditions where the model propagates state without external correction.

The core problem addressed is the disconnect between these offline evaluation metrics and actual closed-loop performance. In a real robotic system, a controller acts based on a state estimate, the robot moves, new sensor measurements arrive, the state is updated, and a new control command is computed. The paper argues that an offline metric (like a long, measurement-free rollout) may not accurately predict which model will perform best in a feedback loop because it ignores the frequency and timing of measurement corrections that occur during actual operation. The relationship between rollout accuracy and closed-loop utility depends heavily on the sensing and update schedule, a factor often omitted in standard evaluations.

Methodology

The authors conduct a controlled study using a differential-drive robot in a planar environment to isolate the effects of prediction, measurement correction, and feedback.

Experimental Setup

Task: Path tracking of a lemniscate reference trajectory using a pure-pursuit controller.
Sensing: The robot relies on biased wheel odometry and intermittent range-bearing measurements from known landmarks.
Conditions: 24 distinct sensing conditions were created by varying landmark availability, range noise, gyroscope bias, and the duration of sensing outages (periods with no landmark data).
Models: Six state estimators were evaluated:
1. Dead Reckoning (DR).
2. Extended Kalman Filter (EKF).
3. Four recurrent residual estimators (GRU-DR, SSM-DR, GRU-EKF, SSM-EKF) that learn a correction term on top of the analytic DR or EKF reference.
4. Variants of the recurrent models were also trained with "extended blackouts" (longer periods without landmark data) to test robustness.

Evaluation Protocols

Each estimator was tested under three distinct protocols to compare offline metrics against closed-loop reality:

Trajectory Replay: The estimator processes a fixed, recorded trajectory with all available measurements. This measures standard offline localization accuracy (Replay Position RMSE).
Measurement-Free Rollout: The estimator is initialized from a checkpoint and propagated forward for $H=20$ steps using only odometry, with all landmark measurements withheld. This tests open-loop state propagation accuracy.
Closed-Loop Path Tracking: The estimator runs online, providing state estimates to the controller. The controller's actions affect the robot's trajectory and subsequent sensor readings. Performance is measured by Cross-Track RMSE.

The study also varied the rollout horizon ( $H$ ) and the measurement-update interval ( $k$ ) to determine how these factors interact.

Key Results

1. Replay Metrics Correlate Better with Closed-Loop Performance than Rollouts

Contrary to the intuition that a multi-step rollout better simulates the challenges of prediction, the Replay Position RMSE showed a stronger correlation with closed-loop cross-track RMSE ( $\rho = 0.923$ ) than the 20-step Measurement-Free Rollout error ( $\rho = 0.774$ ).

In 5 out of 24 sensing conditions, the Replay metric selected a different "best" estimator than the one that actually performed best in closed-loop.
The Measurement-Free Rollout metric selected a suboptimal estimator in 18 out of 24 conditions.
This indicates that a model's ability to propagate state without correction (rollout) does not necessarily predict its ability to function in a feedback loop where corrections are frequent.

2. The Interaction of Horizon and Update Frequency

The study systematically varied the prediction horizon ( $H$ ) and the frequency of measurement updates ( $k$ ) during the offline evaluation.

High Agreement with Updates: When measurements were incorporated at every step ( $k=1$ ), even long horizons ( $H=80$ ) maintained high rank correlation with closed-loop performance ( $\rho \approx 0.90$ ).
Low Agreement without Updates: When measurements were withheld entirely ( $k=\infty$ ), the correlation dropped significantly. The worst disagreement occurred when a long horizon was combined with no corrections ( $H=80, k=\infty$ ), where rank correlation fell to $\rho = 0.32$ .
Conclusion: The degradation in predictive power is not caused by the length of the horizon alone, but by the combination of a long horizon and the absence of measurement updates.

3. Training for Long Outages is Context-Dependent

Training recurrent estimators on data with longer sensing outages (32–58 steps vs. standard 6–27 steps) yielded mixed results:

EKF-Anchored Models: Training with extended blackouts improved performance under combined sensing degradation (e.g., GRU-EKF cross-track RMSE dropped from 1.72m to 1.06m).
DR-Anchored Models: The same training procedure did not improve the DR-anchored SSM model; in fact, performance slightly degraded.
Isolated Outages: Improvements were not uniform across isolated sensing outage conditions. For instance, the extended-training SSM-EKF performed worse than the standard version across the entire range of outage durations.
Implication: Simply matching the training data distribution (blackout duration) to the evaluation condition does not guarantee improved closed-loop robustness; the underlying analytic estimator quality and architecture matter significantly.

Significance and Claims

The paper does not claim that short-horizon prediction is universally superior to long-horizon prediction. Instead, it establishes a practical principle for evaluating predictive models in robotics:

Evaluation Must Match Operation: Offline prediction metrics should be evaluated under the sensing schedule of the intended closed-loop system. A measurement-free rollout is a poor proxy for performance when the deployed robot receives regular measurement corrections.
Horizon is Not the Only Variable: The usefulness of a rollout metric depends on the measurement-update schedule. Evaluation protocols must specify both the prediction horizon and the frequency of state corrections.
Feedback Structure Matters: For models used inside a feedback loop, the evaluation protocol should reproduce the timing of sensing, state correction, and control updates. Treating prediction horizon as the sole measure of temporal difficulty is insufficient.

The authors conclude that while offline rollouts provide useful information about open-loop state propagation, they can produce misleading model rankings if their sensing and correction patterns differ substantially from the closed-loop operation they are intended to support. The study emphasizes that the relationship between prediction accuracy and control performance is mediated by the feedback loop's update frequency.