💻 computer science

Grasp Synthesis Matching From Rigid To Soft Robot Grippers Using Conditional Flow Matching

This paper proposes a Conditional Flow Matching framework that bridges the representation gap between rigid and soft grippers by learning a continuous mapping from Anygrasp poses to stable Fin-ray gripper configurations, significantly improving grasp success rates for both seen and unseen objects compared to direct rigid-to-soft adaptation.

Original authors: Tanisha Parulekar, Ge Shi, Josh Pinskier, David Howard, Jen Jen Chung

Published 2026-02-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tanisha Parulekar, Ge Shi, Josh Pinskier, David Howard, Jen Jen Chung

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, rigid robot hand (like a human hand made of steel) that is excellent at picking up objects. It knows exactly where to grab a cup, a ball, or a box. Now, imagine you want to use a different kind of hand: a soft, squishy robot hand (like a rubber glove or a flexible fin). This soft hand is amazing at handling fragile things like strawberries or interacting safely with humans, but it moves and bends differently than the steel hand.

The problem? The steel hand's "brain" doesn't know how to talk to the soft hand. If you tell the soft hand to grab exactly where the steel hand would, it often fails because the soft hand needs to wrap around things differently. It's like trying to give a recipe for a stiff cake to a baker who only makes fluffy soufflés; the instructions just don't translate.

The Solution: A "Translator" for Robot Hands

This paper introduces a clever new method to act as a translator between these two types of hands. They call it Conditional Flow Matching (CFM).

Here is how it works, using some simple analogies:

1. The "GPS Route" Analogy

Think of the rigid hand's grasp as a starting point on a map (Point A). The perfect soft hand grasp is the destination (Point B).

Old Way: You might try to guess the route by drawing a straight line, but the terrain (the physics of the soft hand) is too bumpy and complex.
The New Way (CFM): The researchers built a smart GPS that doesn't just draw a line; it learns a smooth, flowing river connecting Point A to Point B. It takes the rigid hand's idea and gently "flows" it through a transformation until it becomes the perfect soft hand pose. It learns that to grab a ball, the soft hand needs to sink deeper and wrap around more, so the "river" guides the hand there naturally.

2. The "Chef's Secret Sauce" (The U-Net)

To make sure this translation works for any object (not just the ones it practiced on), the system uses a special camera brain called a U-Net.

Imagine the robot looks at an object through a depth camera. The U-Net is like a master chef tasting the object and instantly understanding its "shape flavor."
It compresses the complex 3D shape of an apple or a remote control into a simple "secret code" (a mathematical vector).
This code is fed into the GPS (the CFM model) to say, "Hey, we are dealing with a round, slippery apple today, so adjust the flow accordingly!"

3. The Training Process: "Practice Makes Perfect"

The researchers didn't just guess the rules. They built a robot arm with a soft "Fin-ray" gripper (named after the flexible fins of a fish).

Step 1: They let the rigid-hand AI (called AnyGrasp) suggest a grab.
Step 2: A human (or the robot itself) manually tweaked that grab until the soft hand held the object perfectly.
Step 3: They showed the CFM model thousands of these "Before" (rigid) and "After" (soft) pairs. The model learned the pattern: "When the rigid hand suggests a shallow pinch, the soft hand needs a deep, wrapping hug."

The Results: Does it Work?

The team tested this on a real robot arm with 7 moving joints. They threw all sorts of objects at it: cylinders (like cans), spheres (like oranges), and flat things (like boxes).

The Rigid Hand's Advice: When the soft hand tried to follow the rigid hand's advice directly, it failed miserably (only about 6% success rate on new objects).
The CFM Translator: When the soft hand used the "flow" translated by the new model, the success rate jumped to 46% on new objects it had never seen before!

The Magic Numbers:

Cylinders (Cans): The rigid advice failed 100% of the time on new cans. The CFM model succeeded 100% of the time.
Spheres (Oranges): The rigid advice failed completely. The CFM model succeeded about 31% of the time.

Why This Matters

This is a big deal because it means we don't need to start from scratch every time we build a new type of robot hand. We can take the massive libraries of data we already have for rigid hands and translate them to work with soft, squishy hands.

It's like taking a library of driving manuals written for sports cars and using a smart AI to rewrite them for tractors, so the tractor can drive just as safely without needing a whole new library of books.

In short: The paper teaches a robot how to take a "stiff" idea of how to grab something and gently stretch and bend that idea until it fits perfectly in a "soft" hand, making robots much better at handling the messy, fragile, and unpredictable real world.

1. Problem Statement

The paper addresses a critical representation gap between grasp synthesis for rigid parallel grippers and soft robot grippers.

The Mismatch: State-of-the-art grasp synthesis methods (e.g., AnyGrasp) are designed for rigid grippers, relying on precise joint control and antipodal grasping assumptions. These methods fail to capture the unique compliant behaviors of soft grippers (e.g., Fin-ray grippers), which rely on envelopment and distributed forces rather than precise point contacts.
The Consequence: Directly applying rigid grasp strategies to soft grippers results in low success rates, data-intensive retraining requirements, and inaccurate models because soft grippers exhibit non-linear deformation that is difficult to model analytically or via simulation.
The Goal: To develop a data-efficient framework that maps grasp poses generated for rigid grippers (specifically AnyGrasp outputs) to stable, successful grasp poses for soft Fin-ray grippers.

2. Methodology

The authors propose a framework utilizing Conditional Flow Matching (CFM), a generative model class, to learn a continuous transformation between two data distributions: rigid grasp poses and soft grasp poses.

A. Data Collection Pipeline

Setup: A 7-DOF Franka Emika Panda robot equipped with a Fin-ray gripper and an Intel RealSense D415 depth camera.
Process:
1. Rigid Pose Generation: For a given scene (depth image $D$ and point cloud $P$ ), the AnyGrasp algorithm generates a ranked list of rigid gripper poses ( $G_{Anygrasp}$ ).
2. Soft Pose Adjustment: The robot approaches the object using the rigid pose but manually adjusts the position and orientation to find a successful grasp ( $G_{CFM}$ ) for the soft gripper.
3. Dataset Creation: This creates paired data $(G_{Anygrasp}, G_{CFM})$ conditioned on the same scene geometry. The dataset includes 254 pairs from 8 objects for training and validation, covering various geometries (cylindrical, spherical, rectangular, flat).

B. Conditional Flow Matching (CFM) Framework

The core of the method models the transformation from a rigid pose to a soft pose as the solution to a conditional Ordinary Differential Equation (ODE).

State Representation: A grasp pose $G = \{R, t, w\}$ is represented as a 7-dimensional vector (4D quaternion for orientation $R$ , 3D position $t$ ). The width $w$ is fixed to the max opening.
The Flow: The model learns a time-dependent vector field $v_\theta$ that guides the trajectory from an initial state $G(0) = G_{Anygrasp}$ to a target state $G(1) = G_{CFM}$ .
$\frac{dG(t_c)}{dt_c} = v_\theta(G(t_c), t_c, c)$
Where $t_c \in [0, 1]$ is a progression parameter and $c$ is the condition vector.
Conditioning (U-Net Autoencoder): To ensure the model adapts to different object geometries, the condition vector $c$ is derived from the raw depth image. A pre-trained U-Net autoencoder compresses the depth image into a 128-dimensional latent vector, which serves as the condition $c$ for the CFM model.
Network Architecture:
- Velocity Field ( $v_\theta$ ): A fully connected MLP with four hidden layers (sizes 128, 256, 256, 128) using SiLU activations and Batch Normalization.
- Inputs: Interpolated grasp pose $G(t_c)$ , time step $t_c$ , and condition vector $c$ .
Training Objective: The model minimizes the Mean Squared Error (MSE) between the predicted velocity field and the target velocity field ( $u_{t_c} = G_{CFM} - G_{Anygrasp}$ ) derived from linear interpolation between the paired data.

C. Inference

During deployment, given a new scene and an AnyGrasp pose, the system solves the ODE numerically (using the Dormand-Prince method) to integrate the learned vector field from $t_c=0$ to $t_c=1$ , generating the final soft gripper pose.

3. Key Contributions

Novel Framework: Introduction of Conditional Flow Matching (CFM) to bridge the representation gap between rigid and soft grasp synthesis, offering a more data-efficient alternative to traditional deep learning or simulation-heavy approaches.
Data Pipeline: Development of a paired data collection pipeline that generates matched rigid-soft grasp poses, enabling the learning of complex, non-linear transformations without explicit physical modeling of the soft gripper.
Conditional Generation: Integration of a U-Net autoencoder to condition the generative model on object geometry, allowing the system to generalize to unseen objects.
Empirical Validation: Comprehensive validation on a real-world 7-DOF robot demonstrating significant performance improvements over baseline rigid strategies.

4. Results

The system was evaluated on a 7-DOF robot with 15 scenes involving 12 distinct objects (including seen and unseen objects).

Overall Success Rate:
- CFM (Soft): ~38% overall success rate.
- Baseline (AnyGrasp on Soft): ~15% overall success rate.
- Specifics: For seen objects, CFM achieved 34% vs. 6% for baseline. For unseen objects, CFM achieved 46% vs. 25% for baseline.
Performance by Geometry:
- Cylindrical Objects: CFM showed massive improvement (50% seen, 100% unseen) compared to the baseline (0% seen, 75% unseen).
- Spherical Objects: CFM achieved 25% (seen) and 31% (unseen), whereas the baseline failed completely (0%).
- Rectangular Objects: CFM achieved 50% for both seen and unseen, outperforming the baseline (25% and 0%).
- Flat Objects: The baseline performed better (100% success) on flat objects, while CFM achieved 25-50%. The authors attribute this to a bias in the training data toward top-down approaches, which are less optimal for side-pinch grasps on flat items.
Generalization: The model successfully generalized to unseen objects (e.g., upside-down fruits), proving it learned the underlying transformation policy rather than memorizing specific object poses.

5. Significance and Discussion

Data Efficiency: CFM achieves high performance with significantly less training data compared to other generative models (like Diffusion models) or rigid-to-soft transfer methods that require extensive simulation.
Physical Insight: The model learned to correct rigid-grasp strategies by:
- Increasing grasp depth: Allowing the soft fingers to envelop the object rather than just pinching the surface.
- Correcting orientation: Shifting from tilted or corner-targeting grasps (good for rigid caging) to centered, top-down approaches that ensure balanced force distribution for soft grippers.
Scalability: This work demonstrates a scalable methodology for transferring grasp strategies across different actuation paradigms (rigid to soft), potentially applicable to other soft robotic systems.
Limitations: The model's performance is constrained by the diversity of the training data. For instance, the bias toward top-down grasps limited performance on flat objects where side-pinch strategies are superior. Future work suggests integrating grasp centering methods and improving geometry perception.

In conclusion, the paper establishes CFM as a powerful, data-efficient tool for adapting established rigid grasping algorithms to the complex, compliant world of soft robotics, significantly improving grasp success rates on real-world hardware.