PRISM: Projection-Integrated Sampling-Based MPC with Bayesian Cost Tuning for Bimanual Manipulation
The paper presents PRISM, a GPU-accelerated Model Predictive Control framework for bimanual manipulation that combines QP-guided projection for feasible trajectory sampling, a custom efficient solver, and Bayesian cost tuning to achieve robust, real-time performance in contact-rich environments with successful sim-to-real transfer.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Robots have long been masters of the factory floor, repeating the same precise movements thousands of times a day. But when the environment changes, or when a task requires two arms to work together in a cluttered space, traditional programming often fails. The challenge lies in coordination: getting two mechanical arms to move in perfect unison while avoiding obstacles, holding objects steady, and reacting to the physical world in real time. For years, researchers have tried to teach robots this skill by showing them thousands of examples, hoping the machine learns the pattern. However, this approach struggles when the robot encounters a situation it has never seen before, such as a new arrangement of obstacles or a slightly different object. To solve this, a team of researchers has developed a new way for robots to think on their feet, using a method that plans every move from scratch based on the laws of physics rather than relying on past memories.
The researchers, working with dual robotic arms, created a system called PRISM. Instead of trying to memorize a specific solution for every possible scenario, the system acts like a super-fast simulator running inside the robot's computer. At every fraction of a second, the robot considers thousands of different ways it could move its arms in the next few moments. It then uses a powerful physics engine to instantly predict what would happen if it tried each of those movements. Would the arms collide with a wall? Would the object slip? Would the motion be too jerky for the motors to handle? By testing these possibilities in a virtual world that mirrors reality, the system can instantly discard the bad ideas and keep the ones that work.
The core innovation of this work is how the system handles the sheer number of possibilities. In the past, when robots tried to explore many different paths at once, the resulting movements were often too erratic or physically impossible, causing the robot to shake or break its own rules. The new method solves this by acting as a filter. Before the robot even simulates a movement, it mathematically forces every potential path to be smooth and safe. It ensures that the arms will not move too fast, accelerate too sharply, or jerk in a way that would damage the machine. This allows the robot to be bold in its exploration, trying out wild and creative movements, while guaranteeing that the final choice is always smooth, safe, and executable.
To make this process even more effective, the team used a technique called Bayesian optimization to automatically tune the robot's priorities. Think of this as a way for the robot to learn how to balance its own internal goals, such as moving quickly versus moving carefully, without a human engineer spending hours adjusting dials. The system ran thousands of simulations, each time slightly changing how much the robot cared about avoiding collisions versus reaching the target, until it found the perfect balance for success. This automated tuning meant the robot could adapt to different tasks, from lifting a heavy box to passing a cube between arms, without needing a new set of instructions for each job.
The researchers tested their system in a virtual environment filled with obstacles and then successfully transferred those skills to real-world robots. In the real world, two robotic arms had to work together to pick up a tray, navigate through a crowded workspace, and place it between two obstacles without dropping it. They also tested scenarios where the arms had to lift a ball, pass a cube to each other, and carry a heavy box. In every case, the system proved more reliable than previous methods. While older approaches often failed when the environment was cluttered or the task required precise coordination, this new system succeeded in the vast majority of attempts. It managed to keep the tray level, hold the ball steady, and pass the cube smoothly, all while reacting to the physical constraints of the real world.
One of the most significant findings was how the system handled situations it had never seen before. When the researchers placed a new obstacle in the path of a robot that had been trained on a different setup, the old learning-based methods often froze or crashed because the new situation did not match their training data. The new system, however, simply simulated the new obstacle and found a way around it on the fly. It did not need to be retrained or shown a new example; it simply reasoned through the physics of the new scene and found a solution. This ability to adapt to the unknown is a crucial step toward robots that can work in dynamic, unstructured environments like warehouses or homes, where the layout is never exactly the same twice.
The study confirms that combining a fast physics simulator with a smart way of filtering movements allows robots to perform complex, coordinated tasks with a level of robustness that was previously difficult to achieve. The system runs fast enough to make decisions in real time, updating its plan dozens of times per second. While the researchers note that the system still requires a human to define the general goal and the basic structure of the task, the heavy lifting of figuring out how to move is done automatically. This approach offers a practical alternative to methods that rely on massive datasets, showing that with the right planning tools, robots can learn to navigate the physical world by understanding it, rather than just memorizing it.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.