💻 computer science

Object-Informed Model Predictive Path Integral Control for Non-Prehensile Robot Manipulation

This paper proposes a hierarchical Model Predictive Path Integral (MPPI) control framework that leverages a simplified object-level plan to guide robot-level planning, significantly improving success rates and computational efficiency for long-horizon non-prehensile manipulation tasks in both simulation and real-world hardware.

Original authors: Nikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nikola Raicevic, Bharath Raam Radhakrishnan, Chenbin Yu, Ki Myung Brian Lee, Nikolay Atanasov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to push a heavy, awkwardly shaped box across a cluttered room to get it to a specific spot on the other side. You can't pick it up; you can only push it. This is what robots call "non-prehensile manipulation."

The paper describes a new way for robots to solve this problem, which is notoriously difficult because the physics are tricky. If you push the box slightly wrong, it might get stuck, slide into a wall, or spin out of control.

Here is the simple breakdown of their solution, using everyday analogies:

The Problem: The "Myopic" Robot

Standard robot planning (called MPPI) is like a person trying to push that box while wearing blinders that only let them see a few feet ahead. They look at the immediate spot in front of them and try to find the best push to get closer to the goal right now.

The problem is that sometimes, to get to the goal, you have to push the box away from it first to go around a chair or a table. A "short-sighted" robot sees that move as a bad idea because it increases the distance to the goal immediately, so it refuses to do it. It gets stuck trying to push straight through obstacles.

The Solution: The "Architect" and the "Builder"

The authors propose a two-step team approach, splitting the job into a Planner (The Architect) and a Doer (The Builder).

The Architect (Object-Level Plan): First, the robot ignores the fact that it has a physical arm with joints. It pretends the box is magical and can move itself directly. It asks, "If I could just teleport this box around the obstacles to the goal, what is the perfect path?" It draws a map of where the box should go, ignoring the robot's limitations for a moment.
The Builder (Robot-Level Plan): Now, the robot looks at that map. It says, "Okay, the box needs to go here first, then there." The robot then figures out how to move its own arm to push the box along that specific path.

By giving the robot a "big picture" map of where the object needs to go, the robot stops making short-sighted mistakes. It is willing to push the box away from the goal temporarily because the "Architect" told it that's the only way to get around the obstacle.

The Two Variations

The paper tests two ways this team can work together:

The "One-and-Done" Method (SOI): The Architect draws the whole map at the very beginning. The Builder then tries to follow that map step-by-step. If the box gets bumped or the floor is slippery, the Builder has to guess how to stay on the original map.
The "Live-Update" Method (CLOI): The Architect and Builder work in a loop. The Architect draws a short segment of the map, the Builder follows it, then checks where the box actually ended up. The Architect then redraws the next part of the map based on the box's actual new position. This is more robust if things go wrong, but it takes a bit more computing power to keep redrawing the map.

The Results: Did it Work?

The researchers tested this on a real robot arm (an xArm6) and in a computer simulation.

Success Rate: The new method was much better at finishing the task. In the computer simulation, it succeeded 40% more often than the standard robot. In real-life experiments with a physical robot, it succeeded 20% more often.
Speed: Interestingly, the new method didn't slow the robot down. In fact, in the simulation, it calculated moves 26% faster because the "Architect" simplified the problem, making it easier for the "Builder" to find a solution.
Real-World Performance: Even with imperfect cameras and slight errors in how the robot pushed, the new method handled obstacles much better than the old method, which often just gave up or got stuck.

The Bottom Line

The paper claims that by separating the "where the object should go" from "how the robot moves," robots can think longer-term and avoid getting stuck. It's like giving a driver a GPS route that tells them to take a detour, rather than just telling them to "drive straight toward the destination," which might lead them into a dead end.

The authors note two current limits: if the robot's model of the object is wrong, the map might be impossible to follow, and the system requires a lot of computer power to run the simulations. But overall, it makes robots much better at pushing things around cluttered rooms.

Technical Summary: Object-Informed Model Predictive Path Integral Control for Non-Prehensile Robot Manipulation

Problem Statement

Non-prehensile manipulation, such as pushing objects, expands robotic capabilities beyond stable grasping but presents significant challenges for long-horizon planning. The primary difficulties stem from two factors:

Discontinuous Contact Dynamics: Small variations in robot motion can lead to qualitatively different object behaviors depending on how contact is established or broken. This renders large portions of the robot's configuration space ineffective for task progress.
Implicit Actuation: The object is not directly actuated; the robot must infer the necessary motion to produce desired object displacements through contact. This creates a massive search space where the planner must simultaneously reason about robot kinematics, contact geometry, and object response.

Standard sampling-based Model Predictive Control (MPC) methods, such as Model Predictive Path Integral (MPPI) control, utilize fast parallel simulators to handle complex dynamics. However, in non-prehensile tasks, standard MPPI often struggles with long-horizon planning, frequently stalling near obstacles or failing to discover feasible detours due to short-sighted cost minimization.

Methodology

The authors propose a hierarchical formulation of MPPI that decomposes the joint robot-object planning problem into two coupled optimal control problems operating at different abstraction levels: an object-level problem and a robot-level problem.

Hierarchical Decomposition

The core idea is to treat the object as if it were directly actuated via virtual control inputs to generate a reference trajectory, which then guides the robot-level planner.

Object-Level Planning: The system solves an optimal control problem for the object alone (Eq. 13), assuming the object can be actuated directly. This planner operates over a longer horizon ( $H_o$ ) in the smaller configuration space of object motions, generating a nominal object reference trajectory $x_o^*$ .
Robot-Level Planning: The robot-level planner (Eq. 14) solves for actual robot control inputs over a shorter horizon ( $H_r$ ). It minimizes a cost function that includes the standard robot dynamics and a coupling term $\ell_c$ , which penalizes the deviation between the actual object state and the reference trajectory generated by the object-level planner.

Control Variants

The paper introduces two specific implementations of this hierarchy:

Sequential Object-Informed (SOI) MPPI: The object-level planner computes a global reference trajectory once (offline or at the start of the episode). The robot-level planner then executes this trajectory in a receding horizon manner without updating the object reference.
Closed-Loop Object-Informed (CLOI) MPPI: The object-level planner operates in a closed loop. At each time step, it re-computes the object reference trajectory based on the latest observed object state. This provides robustness to deviations between the simplified object model and actual interaction dynamics.

Cost Functions

Object-Level: Costs track the squared distance to the goal and penalize excessive contact forces with static obstacles (using a hinge loss on contact force magnitude).
Robot-Level: Costs include terms for end-effector proximity to the object, alignment of the end-effector relative to the goal (encouraging a "pushing" orientation), and maintaining a favorable vertical orientation.

Key Contributions

The paper makes the following contributions:

Hierarchical MPPI Variants: The introduction of CLOI and SOI, which decompose non-prehensile manipulation into object-level and robot-level planning problems, effectively reducing the planning horizon at the robot level while allowing for long-horizon reasoning at the object level.
Mathematical Formulation: A clear formulation of this decomposition based on general dynamic models where coupling occurs exclusively through external force terms.
Empirical Validation: Demonstration of the method's efficacy in both simulation and real-world hardware experiments using a 6-DoF xArm6 manipulator.

Experimental Results

The authors evaluated their methods against standard MPPI using a 6-DoF xArm6 manipulator in both IsaacGym simulation and real hardware experiments.

Simulation Results

In five distinct non-prehensile tasks (ranging from simple pushing to navigating cluttered environments):

Success Rate: SOI achieved an 82% success rate, and CLOI achieved 72%, compared to 42% for standard MPPI.
Performance: The object-informed methods showed lower average position errors and significantly reduced execution times (approx. 183s for SOI and 195s for CLOI vs. 236s for MPPI).
Frequency: CLOI demonstrated a 26% faster control frequency (3.01 Hz vs. 2.38 Hz) compared to standard MPPI, attributed to the reduced complexity of the robot-level search space.

Hardware Results

In real-world experiments involving pushing a can to a goal with and without obstacles:

Success Rate: Both CLOI and SOI achieved a 90% success rate, outperforming standard MPPI at 70%.
Execution Time: SOI yielded the lowest overall execution time, even when accounting for the overhead of offline object trajectory planning.
Robustness: While standard MPPI showed slightly lower position error in successful trials (likely due to simpler environments in the test set), the object-informed methods demonstrated superior ability to navigate obstacles and avoid stalling. The authors note that failures in the proposed methods occurred when the planned reference trajectory passed too close to obstacles, leaving insufficient clearance for hardware execution errors.

Significance and Claims

The paper claims that the proposed object-informed MPPI approach significantly improves the reliability and sample efficiency of long-horizon non-prehensile manipulation. By injecting task structure through a separate object-level plan, the method guides the robot-level planner toward promising control actions, enabling non-myopic reasoning that standard MPPI often lacks.

The authors acknowledge two limitations:

Model Mismatch: Discrepancies between the simplified object model and the actual robot-object dynamics can lead to dynamically infeasible reference trajectories.
Computational Cost: The hierarchical approach increases the total number of parallel simulations required, as it involves solving both object and robot problems.

Future work is proposed to address these limitations through uncertainty-aware guidance and contact-aware object modeling, such as explicitly accounting for feasible contact wrenches and randomizing parameters like friction during rollouts.