Technical Summary: METAPUSHER

Problem Statement

Nonprehensile manipulation of previously unseen objects remains a significant challenge in robotics. Unlike grasping, pushing outcomes depend not only on an object's geometry but also on latent physical properties—such as friction, mass distribution, and inertia—that cannot be inferred from visual perception alone. While prior experience can offer initial estimates, these often degrade during sim-to-real transfer. Existing approaches typically address this problem in isolation: generalizable methods rely on geometric features but fail to capture latent dynamics; adaptive methods update models via interaction but often require dedicated data collection phases or short-horizon control; and kinodynamic planners provide long-horizon reasoning but assume a fixed forward model, rendering planned trajectories invalid when the model evolves. Consequently, there is a need for a framework that couples rapid online dynamics adaptation with long-horizon planning to manipulate unseen objects without prior object-specific interactions.

Methodology

The paper introduces METAPUSHER, a framework integrating meta-learning and adaptive kinodynamic planning. The system operates in two distinct phases:

1. Offline Meta-Training

A forward dynamics model is trained across a diverse set of objects ($O_{train}$) using Model-Agnostic Meta-Learning (MAML).

• Input Representation: The model takes the object's 2D contour point cloud ($g_o$), encoded via a PointNet-style architecture into a 1024-dimensional global feature vector, concatenated with the push action vector $u = [c_x, c_y, \sin\phi, \cos\phi, d]$.
• Output: The model predicts the planar displacement $\Delta x = [\Delta p_x, \Delta p_y, \Delta \psi]$ in $SE(2)$.
• Objective: The meta-learning objective seeks an initialization $\theta_0$ that allows the model to rapidly adapt to a new object's dynamics using only a few gradient updates on limited interaction data. This is achieved by sampling support and query sets for each training object and optimizing the shared parameters to minimize the loss after inner-loop adaptation.

2. Online Deployment and Adaptation

Upon encountering a previously unseen object ($o \in O_{test}$) with zero prior interaction data:

• Initialization: The task begins using the meta-learned initialization $\theta_0$ and the observed geometry of the new object.
• Online Dynamics Adaptation: As the robot executes pushes, the model parameters are updated ($\theta_k \to \theta_{k+1}$) using the observed transitions from the current task. This is a direct gradient update based on the most recent interactions, allowing the model to specialize to the specific latent properties of the unseen object in real-time.
• Adaptive Kinodynamic Planning: Instead of replanning from scratch after every model update, METAPUSHER employs an adaptive planner that retains and refines the existing search tree. The process involves three operations:
  1. Prune: Remove branches of the search tree that are no longer reachable from the current state.
  2. Refinement: Reconstruct the remaining subtree using the updated dynamics model to ensure consistency.
  3. Expansion: If the refined tree no longer reaches the goal, new nodes are added to search for a valid path under the new dynamics.

This coupling ensures that the long-horizon plan evolves alongside the dynamics model, maintaining feasibility without a separate data collection phase.

Key Contributions

1. Geometry-Conditioned Meta-Learning: A dynamics model that enables manipulation of unseen objects without prior object-specific interactions, capable of rapid adaptation from recent transitions during task execution.
2. Adaptive Kinodynamic Planning: A strategy that updates the existing search tree as the dynamics model evolves, avoiding the computational cost of replanning from scratch while maintaining long-horizon feasibility.
3. Unified Framework: The integration of these components allows for efficient manipulation and adaptation in a single loop, requiring no separate object-specific data collection phase.

Experimental Results

The framework was evaluated on 14 unseen objects in both simulation and real-world (sim-to-real) scenarios, comparing against fine-tuning baselines, active learning, MPPI-based control, and reinforcement learning (HACMan).

• Dynamics Adaptation: METAPUSHER achieved lower prediction error with significantly fewer interactions compared to pooled fine-tuning and active learning methods. It demonstrated effective sim-to-real transfer, adapting rapidly to physical objects despite being trained entirely on simulated data.
• Manipulation Performance:
  • In simulation, combining meta-learning with adaptive planning achieved a 100% task success rate, outperforming MPPI-based controllers and fine-tuning baselines.
  • The method improved the task success rate by up to 20% compared to alternatives.
  • In real-world experiments, METAPUSHER achieved a 100% success rate across all evaluated trials, successfully handling perception noise and execution uncertainties.
• Generalization: When compared to models trained with access to the evaluation objects' data, METAPUSHER (trained only on disjoint training objects) achieved comparable or superior performance in terms of tracking error and push efficiency, confirming its ability to generalize without object-specific pre-training.

Significance and Claims

The paper claims that METAPUSHER addresses the critical gap between dynamics adaptation and long-horizon planning. By coupling rapid online adaptation with a planning strategy that reuses and refines existing search trees, the framework enables robots to manipulate unseen objects immediately upon encounter. The authors emphasize that this approach eliminates the need for a dedicated data collection phase for new objects, a common bottleneck in adaptive manipulation. The results demonstrate that a meta-learned initialization, when combined with adaptive planning, can effectively transfer to unseen objects and latent physical properties, achieving robust performance in both simulation and real-world settings.

Limitations

The authors acknowledge current limitations, including the reliance on accurate contour point cloud observations and state estimation. The current formulation is restricted to planar manipulation ($SE(2)$) using normal-direction push actions. Future work is proposed to extend the framework to full $SE(3)$ manipulation, more complex contact interactions, and the incorporation of force sensing.