🤖 AI

GOAG: Generative and Object-Agnostic Grasp Planner for Dexterous Robotic Manipulation

GOAG is a novel object-agnostic deep generative model that leverages the geometric compatibility between gripper and object contact surfaces to efficiently generalize grasp planning to unseen objects without requiring object-specific training data, achieving state-of-the-art performance in both simulation and real-world experiments.

Original authors: Julien Merand, Boris Meden, Mathieu Grossard, Liming Chen

Published 2026-08-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Julien Merand, Boris Meden, Mathieu Grossard, Liming Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots have long been masters of repetitive tasks, moving with precision along fixed paths in factories. Yet, when asked to reach into a cluttered drawer and pick up a random object, they often stumble. The challenge lies not in the hardware, which has advanced to include hands with multiple fingers capable of delicate manipulation, but in the software that tells those hands how to move. For a robot to grasp an object, it must calculate exactly where its fingers should touch the surface to hold it securely without dropping it or crushing it. Traditional methods rely on vast libraries of pre-calculated examples, teaching the robot how to hold specific shapes like a coffee mug or a screwdriver. However, this approach hits a wall when the robot encounters a new, unseen object. Because the training data is tied to specific shapes, the robot struggles to generalize, often failing to find a stable grip on anything it has not seen before.

A team of researchers has proposed a fundamentally different way to solve this problem, shifting the focus from the object to the hand itself. Instead of teaching a robot how to hold thousands of different objects, they taught it how its own hand works. The core idea is that for a grasp to succeed, the surface of the robot's fingers and the surface of the object must match perfectly at the points where they touch. If you were to press a robot finger against a wall, the shape of the contact area on the finger is identical to the shape of the contact area on the wall. The researchers realized that by studying only the geometry of the robot's hand and the ways its joints can move, they could learn a universal set of rules for making contact. This approach allows the robot to learn a "grip strategy" without ever seeing a single object during its training phase.

The researchers developed a system they call GOAG, which stands for a Generative and Object-Agnostic Grasp Planner. In the training phase, the system ignores objects entirely. Instead, it generates millions of random configurations for a robotic hand, simulating how the fingers might curl, spread, or close. For each of these hand positions, the system identifies the specific areas on the fingers that would be available to touch something. It learns the statistical patterns of these contact zones, essentially memorizing the "fingerprint" of a successful grip based solely on the hand's own mechanics. This is a departure from previous methods that required massive datasets pairing specific objects with specific grasps. By training only on the hand, the system avoids the bias of learning only how to hold the specific shapes found in its training database.

When the system is ready to use, it is presented with a new object it has never seen before. The robot scans the object to understand its shape and then asks the trained model to predict where the fingers should touch. Because the model learned the intrinsic capabilities of the hand, it can instantly generate a list of valid contact points on the new object that are compatible with the hand's movement. The system then calculates the precise joint angles needed to move the fingers to those points. Crucially, before the robot attempts to pick up the object, the software runs a quick check to ensure the proposed grip is physically stable, verifying that the forces involved would prevent the object from slipping. If the first guess is unstable, the system quickly generates a new set of contact points and tries again, repeating this process only a few times until a secure grip is found.

The team tested this approach in both computer simulations and with a real robot arm equipped with a multi-fingered hand. They evaluated the system on a wide variety of objects, including those from standard test sets used by other researchers. In simulations, the method achieved an average success rate of 86.93 percent when tested on objects from the MultiDex dataset, outperforming other leading methods that were specifically trained on that same data. Perhaps more impressively, the system generated grasps significantly faster than its competitors, especially when asked to produce many different ways to hold a single object. The researchers also tested the system on a real robot, successfully grasping eleven different objects from the YCB dataset, which includes everyday items like bananas, tools, and containers. These real-world trials confirmed that the strategy learned in the simulation could transfer directly to physical hardware.

The results suggest that by decoupling the learning process from specific objects, robots can achieve a level of flexibility that was previously difficult to attain. The system does not need to be retrained every time a new object is introduced; it simply applies its understanding of its own hand to the new shape. While the researchers note that the method works best for objects that fit comfortably within the robot's reach, the ability to generalize to unseen shapes without prior exposure marks a significant step forward. The work demonstrates that a robot can learn to grasp the unknown by first understanding itself, offering a path toward more adaptable and capable machines that can operate in the unpredictable environments of the real world.

Technical Summary: GOAG – Generative and Object-Agnostic Grasp Planner

Problem Statement

Dexterous grasping with multifingered robotic hands remains a significant challenge. While sophisticated hardware exists (e.g., Allegro, ShadowHand), current deep-learning grasp planners struggle to generalize to novel, real-world object shapes. Traditional data-driven approaches are typically trained on limited, object-specific datasets, creating a bias that hinders their ability to handle unseen geometries. Furthermore, generating large-scale, physically validated grasp databases for specific object-gripper pairs is computationally expensive and time-consuming. The core problem addressed is the "generalization bottleneck": how to create a grasp planner that can adapt to arbitrary object shapes without requiring retraining on object-specific data.

Methodology

The authors propose GOAG (Generative and Object-Agnostic Grasp Planner), a novel framework that shifts the learning paradigm from an object-centric or grasp-centric view to a gripper-centric perspective. The fundamental observation driving this approach is that at the point of contact, the gripper and the object share identical surface geometry. Therefore, the distribution of valid contact zones is an intrinsic property of the gripper's kinematics and geometry, independent of the specific object being grasped.

Core Architecture and Training

Object-Agnostic Training: Unlike traditional methods that learn mappings from object point clouds to grasp poses, GOAG is trained exclusively on the gripper's geometry.
- Data Generation: The system samples valid kinematic configurations ( $Q$ ) of the gripper and maps them to specific grasp taxonomy labels (e.g., C6, F27). It then randomly samples contact points strictly within the admissible regions defined by these taxonomies on the gripper's surface. This creates a dataset of feasible contact distributions without ever using object geometry.
- Basis Point Set (BPS) Encoding: To handle variable point clouds, the gripper's workspace is discretized into a fixed-size Basis Point Set ( $W$ ). Input data (gripper surface points during training, object points during inference) are projected onto this basis to compute an aligned distance field.
- Generative Model: A Conditional Variational Autoencoder (CVAE) is trained to learn the conditional distribution of contact points. The encoder takes the distance field and contact labels to predict a latent variable $z$ . The decoder reconstructs the contact distribution.
- Links Mapper: A parallel PointNet++ network is trained to associate predicted contact points with specific gripper links (phalanges), providing the kinematic context necessary for execution.
Inference Pipeline:
- Input: A novel object point cloud $O$ is transformed into the gripper's canonical frame using an inverse pose $[R, T]^{-1}$ .
- Generation: The CVAE decoder, conditioned on the object's distance field and a sampled latent vector $z$ , generates a continuous contact field $\hat{C}(O)$ . High-probability points are extracted to form a discrete set of target contact locations.
- Stability Check: A rapid Force-Closure (FC) estimation is performed on the predicted contact barycenters. If the configuration fails the FC check (indicating instability), the latent variable is resampled (up to 20 iterations) to generate a new candidate.
- Optimization: A final optimization step solves for the optimal joint configuration $Q^*$ $Q^{*}$ . This minimizes an energy function comprising:
  - Contact distance (aligning links to targets).
  - Object penetration (avoiding intersection with the object).
  - Self-penetration (avoiding collisions between gripper links).
  - Joint limits.

Key Contributions

Object-Agnostic Learning Strategy: The paper introduces a training paradigm that decouples the learning phase from object data. By modeling the gripper's intrinsic contact capabilities, the method eliminates bias toward specific object datasets and enables zero-shot generalization to arbitrary shapes.
Efficiency and Generalization: The approach demonstrates that a model trained solely on gripper kinematics can achieve state-of-the-art performance on novel objects. It offers significantly faster processing times when generating multiple grasps compared to methods that optimize each grasp independently.
Robust Physical Validity: The pipeline integrates explicit force-closure estimation and a constrained optimization step to ensure that generative predictions result in physically feasible, non-penetrating, and stable grasps.

Experimental Results

The authors validated GOAG through extensive experiments in simulation and on real-world hardware.

Multidex Dataset Performance: On the standard Multidex test set, GOAG achieved an average success rate of 86.93% across three different grippers (Barrett, Allegro, ShadowHand). This outperformed several baselines, including GenDexGrasp (70.96%) and DRO-Grasp (72.47%), despite GOAG not being trained on object-specific data.
Generalization: In a cross-dataset evaluation involving five different benchmarks (including real-world and human-hand retargeted datasets), GOAG achieved the second-highest average success rate (53.97%). Notably, while competing methods were retrained specifically for each dataset, GOAG was trained only once on the Shadow Hand kinematics.
Efficiency: GOAG demonstrated superior efficiency in batch generation. While some baselines are faster for a single grasp, GOAG's vectorized optimization allows it to generate numerous grasps significantly faster (e.g., ~0.18s per grasp vs. >9s for GenDexGrasp).
Real-Robot Deployment: The method was successfully deployed on a 7-DoF robot arm with an Allegro Hand, successfully grasping 11 objects from the YCB dataset in a real-world setting.

Significance and Claims

The paper claims that GOAG represents a fundamental shift in dexterous grasp planning. By leveraging the observation that contact geometry is a shared property between the gripper and the object, the authors demonstrate that object-specific training data is not strictly necessary for high-performance grasping.

The significance lies in:

Scalability: Eliminating the need for costly, object-specific grasp database generation.
Generalization: Providing a solution that naturally adapts to unseen object shapes without retraining.
Competitiveness: Achieving performance comparable to, and in some metrics exceeding, state-of-the-art methods that rely on massive, object-specific datasets.

The authors conclude that their gripper-centric, object-agnostic approach effectively addresses the generalization challenges faced by traditional data-driven planners, offering a robust and efficient solution for dexterous robotic manipulation.