Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models

Robots that can plan their own movements often rely on an internal map of how the world works. Imagine a robot arm trying to push a block across a table. Instead of just reacting to what it sees in the moment, a smart robot builds a mental model that predicts what will happen if it moves its arm a certain way. It learns to say, "If I push here, the block will slide there." This mental model, often called a world model, allows the robot to simulate future actions in its mind before it actually moves, helping it avoid mistakes and reach its goals efficiently. These models are usually trained in a controlled environment, like a computer simulation, where the lighting is perfect and the physics are consistent. However, the real world is messy. When a robot trained in a simulation is placed in a new room with different lighting, or when the table surface becomes slippery, the robot's internal map can become wrong. The predictions it makes no longer match reality, and the robot fails to complete its task. For years, the standard solution to this problem has been to retrain parts of the robot's brain while it is working, adjusting millions of internal settings to fit the new conditions. This process is heavy, slow, and requires the robot to constantly rewrite its own understanding of how things move.

A team of researchers has discovered a much lighter way to fix this problem. Instead of asking the robot to rewrite its entire internal map, they found that it is often enough to simply adjust the edges where the robot reads its inputs and writes its outputs. In a new study, the researchers introduced a method they call "Sandwich-Residuals." They took a robot that had already learned how to move in a simulation and froze its internal brain completely, preventing any of its millions of learned settings from changing. Then, they added two very small, flexible layers of software: one that tweaks the information coming into the robot's brain, and another that tweaks the predictions coming out of it. These small layers act like a sandwich filling, wrapping around the frozen core. As the robot tries to move and makes mistakes, these small layers learn to correct the errors on the fly, without ever touching the heavy, pre-trained brain inside. The robot learns to say, "My brain thinks the block will go here, but my new correction layer knows the floor is slippery, so I will adjust my plan to send it there instead."

The researchers tested this idea on a variety of challenging tasks, including navigating mazes and pushing objects of different shapes. They compared their method against the standard approach, which involves updating the robot's internal brain, and against a robot that made no adjustments at all. In twenty-one different test scenarios, the robot using the new "sandwich" method succeeded in reaching its goal 65 percent of the time. This was a significant improvement over the unadjusted robot, which succeeded only about 49 percent of the time. More importantly, the new method performed nearly as well as the standard approach that rewrites the robot's brain, which succeeded about 68 percent of the time. The crucial difference lies in efficiency. The standard method had to update nearly ten million settings to adapt to the new conditions. The new method, by contrast, only needed to adjust about one hundred thousand settings. This means the new approach adapts using less than three percent of the computational effort required by the old method, yet it achieves almost the same level of success.

The study also looked at what happens when the robot faces multiple problems at once, such as a change in lighting combined with a change in how heavy the objects feel. In these difficult "compound" situations, the new method was even more impressive. It succeeded 1.9 times more often than the unadjusted robot, while remaining just as effective as the heavy, brain-updating method. The researchers found that the type of correction needed depended on the specific problem. When the robot was navigating a maze and the physics of movement changed, the layer that corrected the robot's predictions after they were made did most of the work. When the robot was pushing objects and the controller became more sensitive, the layer that adjusted the robot's input commands was more important. By keeping both layers, the system could handle a wide variety of unexpected changes without needing to know in advance what kind of trouble it would face.

To prove that this idea works beyond simple maze games, the researchers also applied it to a more complex task involving a real-world style robot arm moving a cube in three-dimensional space. They used a different type of robot brain for this task, one that relies on visual patterns rather than the previous design. Even with this different architecture, the same "sandwich" principle worked. The robot was able to adapt to changes in lighting, camera angles, and the strength of its own motors. In every single test case where the conditions changed, the new method improved the robot's performance compared to doing nothing, whereas the other methods sometimes made the robot perform worse. This suggests that the ability to adapt does not always require a robot to fundamentally change how it understands the world. Sometimes, it is enough to simply add a small, flexible filter that helps the robot interpret its current situation correctly.

The findings suggest a shift in how we might build robots for the future. For a long time, the assumption was that if a robot's environment changed, its internal model of physics had to be rewritten to match. This paper shows that assumption is not always necessary. If the robot's core understanding of the world is still mostly correct, it can simply learn to adjust its inputs and outputs to fit the new reality. This approach is not just faster and cheaper; it is also more stable, because it avoids the risk of the robot forgetting what it already knows while trying to learn something new. While the experiments were conducted in computer simulations, the results point toward a future where robots can enter new environments and immediately start working, using tiny, efficient adjustments to handle the surprises of the real world without needing a massive overhaul of their intelligence.