🤖 AI

Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints

This paper proposes a unified Task and Motion Planning (TAMP) approach that leverages object-centric abstractions of motion constraints within a single heuristic search to generate physically feasible plans efficiently, thereby eliminating the need for costly sub-symbolic geometric reasoning and iterative re-planning.

Original authors: Alejandro Agostini, Justus Piater

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alejandro Agostini, Justus Piater

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to make a sandwich. You give it a simple list of instructions: "Get the bread," "Get the cheese," "Put cheese on bread." This is the Task Plan.

However, the robot's brain has a problem. The instruction "Get the bread" is vague. It doesn't say where the bread is, how to hold it, or if the robot's arm will bump into the toaster while reaching for it.

Traditionally, robots handle this by guessing. They try to grab the bread. If they miss or hit the toaster, they say, "Oops, that didn't work," and try again. They might try a different angle, then another, and another. This is like trying to solve a maze by running into every wall until you find the exit. It takes a long time, and the robot wastes a lot of energy hitting walls (or in this case, running expensive computer simulations that fail).

The Problem with the Old Way
The paper describes a system where the "Task Planner" (the brain that decides the order of steps) and the "Motion Planner" (the brain that figures out the physical movement) don't talk to each other well. The Task Planner says, "Pick up the cup," and the Motion Planner tries to figure out the physics. If the cup is in a weird spot, the Motion Planner fails, tells the Task Planner, "I can't do that," and the Task Planner has to start over. This back-and-forth is slow and clunky.

The New Solution: The "Object-Centric" Map
The authors, Alejandro Agostini and Justus Piater, propose a new way called U-TAMP. Instead of treating the robot's movements as a separate, messy math problem, they translate the physical world into a simple, symbolic map that the Task Planner can understand immediately.

They do this using Object-Centric Abstractions. Think of every object (like a cup or a block) not just as a 3D shape, but as a box with six distinct sides: Top, Bottom, Front, Back, Left, and Right.

The Analogy: Imagine the robot doesn't see a complex 3D cup. Instead, it sees a "Top Face," a "Bottom Face," and "Side Faces."
The Magic: The robot learns that to pick up a cup, it needs to touch the "Top Face" with its palm and the "Side Faces" with its fingers. It doesn't need to calculate complex angles or run physics simulations to know this is possible. It just checks the map: "Is the Top Face clear? Yes. Is the Side Face clear? Yes. Okay, I can grab it."

How It Works in Practice
The paper introduces two main ways the robot thinks about the world:

Object-Support (The Shelf): This is for things sitting on top of other things, like a book on a table. The robot checks: "Is the top of the book clear? Is the table strong enough to hold it?"
Object-Container (The Box): This is for things inside other things, like a spoon in a drawer. The robot checks: "Is the inside of the drawer empty? Is the spoon inside the drawer?"

By turning complex 3D geometry into simple "Yes/No" questions about these faces (e.g., "Is the 'Top' face clear?"), the robot can use a very fast, standard search engine (like the ones used in video games or chess) to find the perfect plan. It finds the solution before it even tries to move its arm.

The Results: Speed and Success
The authors tested this on two difficult scenarios:

Arranging Blocks: A robot had to move green blocks behind blue ones in a crowded room. This is tricky because the robot often has to move a block away from its goal just to get to another block, then move it back later.
Cooking Dinner: A robot had to wash vegetables, cook them, and set a table, all while moving other ingredients out of the way.

The Outcome:

Old Way (State of the Art): Took about 44 to 135 seconds to plan the moves, and only succeeded about 72-76% of the time. It was like a student taking a test, getting stuck, guessing, and sometimes failing.
New Way (U-TAMP): Took less than 1 second (0.31s and 0.94s) to plan the moves and succeeded 100% of the time. It was like a student who studied the map perfectly and walked straight to the answer.

In Summary
This paper doesn't invent a new robot or a new type of hand. Instead, it invents a new language for the robot to speak. By translating complex physical rules (like "don't hit the wall") into simple, object-based rules (like "the 'Right' side of the box is clear"), the robot can plan its entire day in a flash, without needing to run expensive, slow simulations to see if it will crash. It turns a slow, trial-and-error process into a fast, confident decision.

Technical Summary: Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints

Problem Statement

In Task and Motion Planning (TAMP), a fundamental challenge arises from the ambiguity and under-determination of abstract symbolic descriptions used by task planners. These descriptions often fail to characterize the specific physical constraints required to successfully execute a task in the real world. The traditional approach addresses this by decoupling task planning from motion planning: task planners generate sequences of abstract instructions while overlooking physical constraints, and subsequent sub-symbolic geometric reasoning techniques attempt to ground these actions.

This decoupling leads to significant inefficiencies. The geometric reasoning phase often relies on expensive, trial-and-error strategies (e.g., sampling-based methods) that require multiple calls on unfeasible actions. If a geometric solution cannot be found, the system must trigger task plan correction and re-planning, initiating another intensive search process. Furthermore, many existing TAMP approaches rely on "black-box" sub-symbolic mechanisms to assess motion feasibility outside the task planning loop or require complex robot dynamics models and optimal control computations, which are computationally prohibitive for long-horizon tasks.

Methodology

The authors propose U-TAMP, a unified approach that integrates task and motion planning into a single heuristic search. The core innovation is the use of object-centric abstractions to represent motion constraints directly within the symbolic task planning domain. This allows the planner to evaluate geometric feasibility using standard AI heuristic search techniques, eliminating the need for separate, expensive sub-symbolic reasoning loops.

1. Object-Centric Abstractions

The method defines symbolic variables based on the functional parts of an object's bounding box and orientation. An object $o$ is characterized by parts: on (top), under (bottom), left, right, front, back, and in (inside).

Grasping Constraints: Defined by variables representing the interaction between the object and the robot hand's palm and fingers (e.g., ?o-h-p, ?o-h-f1, ?o-h-f2). Assigning values to these variables uniquely determines the hand-object pose ( $^1\xi_h$ ), including relative position and orientation (roll, pitch, yaw).
Placement Constraints: Defined by variables representing the interaction between the object and its support (e.g., ?o1-o2, ?o2-o1). This includes interactions with support surfaces and container spaces. Assigning values determines the object-support pose ( $^2\xi_1$ ).
Kinematic Constraints: The global hand pose ( $r\xi_h$ ) is derived mathematically from the grasping and placement configurations using transformation matrices, ensuring kinematic feasibility without explicit inverse kinematics solvers during the search.

2. Constraint Encoding in PDDL

The authors encode these abstractions into Planning Domain Definition Language (PDDL) predicates to guide the heuristic search. Two primary task types are addressed:

Object-Support Tasks: Focus on interactions between an object and a support surface. Predicates check if the interacting parts are clear (e.g., oc ?o1-h-p ?o1 air) and if the support surface can stably hold the object (e.g., force ?o2 ?o2-o1).
Object-Container Tasks: Focus on placing objects inside container spaces (e.g., a cup in a box). Here, the container space acts as the support. Constraints are evaluated based on the surrounding space of the container rather than the object's surface directly, allowing for reasoning about clutter and adjacent objects.

3. Unified Action Templates

The approach defines specific PDDL action templates (Pick, Place, Pick-space, Place-space) that incorporate these constraints as preconditions.

Preconditions: Ensure that a grasping configuration is legal (stable grasp) and kinematically reachable (not opposite to the robot base) before the action is considered.
Effects: Update the symbolic state to reflect changes in object locations, clear surfaces, and force distributions (e.g., an object picked up no longer supports other objects).
Execution: Once a plan is generated, the symbolic variables are mapped back to real-valued parameters (poses and trajectories) using the defined geometric relationships. A pre-grasping/pre-placing point is calculated to generate collision-free trajectories using cubic spline interpolation.

Key Contributions

Unified Heuristic Search: The paper presents a method that unifies task and motion planning into a single search process, leveraging the computational efficiency of off-the-shelf AI planners (specifically Fast Downward).
Object-Centric Abstractions: It introduces a novel representation where motion constraints (grasping, placement, kinematics) are abstracted into symbolic variables based on object parts, allowing geometric feasibility to be assessed at the task planning level.
Elimination of Sub-Symbolic Loops: The approach removes the need for iterative trial-and-error geometric reasoning or "black-box" feasibility checks outside the planner. Feasible plans are generated directly.
Dynamic Tabletop Generation: The system can dynamically generate tabletop parts based on object poses and available spaces, enabling robust handling of disturbances and random object placement, unlike previous approaches restricted to handcrafted scenarios.
Scalability to Multi-Object Interactions: Unlike previous object-centric works limited to single interactions, this approach encodes constraints for multiple object interactions (e.g., placing an object to the right of one and in front of another).

Experimental Results

The U-TAMP framework was evaluated on two challenging, non-monotonic benchmark tasks adapted from Garrett et al. [23]:

Arranging Green Blocks: A task requiring moving green blocks behind blue blocks in a cluttered environment, necessitating the undoing of goal conditions.
- Performance: U-TAMP generated a 46-action plan in 0.31 seconds and executed it without failure.
- Comparison: This significantly outperformed the state-of-the-art HFFRob, HA algorithm, which required 135 seconds with a 0.72 success rate.
Cooking Dinner for Two: A complex task involving picking cabbages, cleaning/cooking them, washing cups, and setting a table, involving both object-support and object-container interactions.
- Performance: U-TAMP generated a 57-action plan in 0.94 seconds and executed it without failure.
- Comparison: The state-of-the-art approach required 44 seconds with a 0.76 success rate.

In both cases, U-TAMP demonstrated a reduction in computation time by approximately two orders of magnitude compared to existing methods.

Significance and Claims

The paper claims that U-TAMP represents a significant advancement in TAMP by bridging the gap between symbolic task planning and geometric motion planning. By moving motion planning constraints into the task planning level via object-centric abstractions, the approach yields physically feasible plans that can be directly transformed into execution parameters.

The authors state that their approach:

Avoids expensive sub-symbolic reasoning: It does not require intensive geometric backtracking or sampling-based retries.
Does not require robot dynamics models: Unlike optimal control-based TAMP approaches, it relies on state-based planning efficiency.
Scales to long-horizon tasks: It successfully solved complex, non-monotonic scenarios that are difficult for hierarchical or decoupled approaches.
Generalizes well: The object-centric representation allows for generalization across objects with similar physical laws, and the dynamic generation of support spaces allows for robustness in unstructured environments.

The authors conclude that while their current implementation uses bounding box parts, the framework is designed to be easily extended to richer sets of functional object parts, suggesting a pathway for broader applicability in robotic manipulation.