💻 computer science

PartialBiGrasp: Inferring Hidden Local Geometry for Bimanual Grasping from Partial Views

This paper introduces PartialBiGrasp, a novel framework that enables robust dual-arm robotic grasping of complex objects from partial point cloud views by implicitly learning hidden local geometry through convolutional occupancy networks to generate and refine force-closure compliant grasp pairs.

Original authors: Ayush Kaura, Vignesh Vembar, Md Faizal Karim, Keshab Patra, K Madhava Krishna

Published 2026-08-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ayush Kaura, Vignesh Vembar, Md Faizal Karim, Keshab Patra, K Madhava Krishna

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots have long been masters of the assembly line, repeating the same precise motion thousands of times without error. Yet, when we ask them to perform the fluid, adaptable tasks of daily life—picking up a heavy box, lifting a chair, or carrying a tray—they often stumble. The difficulty lies not in the strength of the machine, but in the uncertainty of its vision. A robot arm sees the world through a camera, and like any camera, it has a limited field of view. When a robot looks at a large object, it sees only the side facing it; the back, the bottom, and the hidden corners remain a mystery. This is particularly problematic for dual-arm robots, which are designed to work together to lift heavy or awkward items. To lift a large object safely, two arms must find a pair of spots that are not only strong enough to hold the weight but also positioned so the robot does not drop the object or crash its own hands into the item. If the robot cannot see the full shape, it might guess a spot that looks good on the surface but is actually too thin, too curved, or blocked by a hidden part of the object.

For years, researchers have tried to teach robots to grasp objects by feeding them complete 3D maps of the world, assuming the robot has already filled in all the missing pieces. But in the real world, robots rarely get a perfect, complete view. They get partial, noisy glimpses. A new approach called PartialBiGrasp, developed by researchers at the Robotics Research Center in Hyderabad, addresses this gap directly. Instead of trying to rebuild the entire hidden shape of an object before acting, this system teaches the robot to reason about what is hidden based on what is visible. The team found that by learning to infer the local geometry of an object—such as its thickness and how a surface continues behind an edge—a robot can generate stable, two-handed grasps even when it can only see a fraction of the object.

The core challenge the researchers tackled is that a valid two-handed grasp is not simply two independent grabs added together. The two arms must work in concert, satisfying strict physical rules to ensure the object does not slip or rotate. In a single view, a robot might see a flat surface that looks perfect for a grip, only to discover that the object is too thin to hold, or that the back of the object curves away in a way that makes the second arm's position impossible. Previous methods often failed here because they relied on reconstructing the entire object first, a process that frequently introduced errors around thin edges and complex curves. If the reconstruction was slightly wrong, the robot would plan a grasp that looked good on the computer screen but would result in a collision or a drop in reality.

To solve this, the researchers built a system that skips the full reconstruction step and goes straight to understanding the geometry relevant to the grasp. The robot's vision system first creates a cloud of points representing the visible surface. Instead of trying to guess the whole shape, the system uses a specialized neural network to learn a continuous map of the object's presence. This map allows the robot to "ask" questions about any point in 3D space, even those it cannot see, to determine if there is solid material there or empty space. The system operates on two levels. First, a global view helps the robot understand the overall shape and identify broad areas where a grasp might be possible. Second, a local view zooms in on specific candidate spots to check fine details, such as whether there is enough space for the robot's fingers to close without hitting the object, or if the surface is thick enough to support the weight.

This two-tiered approach allows the robot to predict a pair of grasps that are physically stable. The system generates many potential pairs and then uses a learned "critic" to evaluate them, checking if they satisfy the laws of physics required to hold the object securely. Crucially, the system does not stop at the initial guess. It runs a refinement process that simulates small adjustments to the robot's hands. By checking the hidden geometry around the contact points, the system can nudge the grasp slightly to avoid a collision or to ensure the fingers are pressing firmly against a solid surface. This refinement happens without needing to see the whole object, relying instead on the system's ability to infer the hidden local structure.

The researchers tested this method extensively, comparing it against other leading techniques that either tried to reconstruct the full object first or relied on simpler pairing strategies. In simulations using a large dataset of objects, the new method achieved a success rate of nearly 68 percent in generating physically stable grasps, significantly outperforming other approaches which often struggled with collision rates or failed to find valid pairs. When tested on real-world objects using a dual-arm robot setup, the system maintained a high success rate of over 81 percent, even when the input data was noisy and incomplete. The results showed that the system could successfully lift heavy items like briefcases and chairs, avoiding the unstable or colliding grasps that plagued other methods.

One of the most significant findings was that the system's ability to infer hidden geometry was essential. When the researchers removed the component responsible for local refinement, the success rate dropped, and the number of collisions increased. This proved that knowing the general shape of the object was not enough; the robot needed to understand the specific, fine-grained details of the surface it was touching. Furthermore, the study demonstrated that trying to reconstruct the entire object before grasping was not only computationally expensive but also prone to errors that made the final grasp less reliable. By focusing directly on the geometry needed for the grasp, the system remained efficient and robust.

The work also highlighted the limitations of current approaches. The system does not yet account for the physical reachability of the robot's arms or the complex motion planning required to move them into position without hitting each other. It generates the ideal grasp points, but a separate planner is still needed to ensure the robot can actually get there. However, by solving the problem of finding the right spots on an object from a partial view, this research removes a major barrier to deploying dual-arm robots in unstructured environments. It suggests that robots can learn to "feel" the shape of an object through inference, allowing them to handle the heavy, awkward, and complex tasks of the real world with a level of confidence that was previously out of reach.

Technical Summary: PartialBiGrasp

Problem Statement

Dual-arm robotic grasping is critical for manipulating large, heavy, and geometrically complex objects that single manipulators cannot handle reliably. However, existing learning-based bimanual grasp generation methods face significant challenges in real-world scenarios where only partial point cloud observations (e.g., from a single RGB-D sensor view) are available.

The core difficulty lies in the fact that partial views obscure hidden local geometry essential for determining grasp feasibility. Large objects often possess sparse graspable regions defined by fine-grained properties such as local thickness, edge structure, and gripper clearance. Incomplete observations can introduce deceptive grasp cues, such as artificial edges or thick surfaces that appear geometrically valid but are physically infeasible, leading to unstable grasps or collisions. Furthermore, unlike single-arm grasping, dual-arm grasps must satisfy force-closure constraints jointly, meaning two grasps must be individually valid and collectively stable, a requirement that is difficult to infer without complete object geometry.

Methodology: PartialBiGrasp

The authors propose PartialBiGrasp, a framework that generates stable, force-closure-compliant dual-arm grasp pairs directly from single-view partial point clouds without requiring explicit shape reconstruction or complete object geometry. The approach relies on implicit geometric reasoning via convolutional occupancy networks.

1. Implicit Geometry Representation

The framework utilizes a continuous feature field derived from the partial point cloud to reason about both global structure and hidden local details.

Global Geometry Encoder: Processes the entire object point cloud to learn object-level structure, identifying feasible grasp regions and spatial relationships. It uses a point-based encoder projected onto a triplane representation and processed by a 2D U-Net to aggregate multi-scale context.
Local Geometry Encoder: Focuses on the vicinity of candidate grasp points within a grasp-centered coordinate frame. It implicitly captures fine-grained properties (thickness, surface continuation, gripper clearance) crucial for contact reasoning. This encoder is conditioned on global features via a residual connection to maintain awareness of the overall object structure.
Training: Both encoders are trained using occupancy supervision (Binary Cross-Entropy loss) on sampled 3D query points, learning to predict whether a point is inside or outside the object.

2. Dual-Arm Grasp Synthesis Pipeline

The generation process occurs in three stages:

Single-Grasp Candidate Generation: The Global Encoder evaluates spherical neighborhoods around contact points to predict graspability and regress grasp parameters (approach vector, baseline vector, gripper offset).
Force-Closure Pairing: A learned Dual-Grasp Critic evaluates candidate pairs of single-arm grasps. It concatenates the latent representations of two grasps and classifies whether they form a force-closure stable pair, trained on the DG16M dataset.
Local Occupancy-Guided Refinement: To address geometric inconsistencies caused by occlusions, a lightweight optimization step refines individual grasp poses. Using the Local Encoder, the system samples perturbations in the Lie algebra $se(3)$ and minimizes an objective function $J$ that penalizes collisions (outer gripper points) and encourages stable contact (inner gripper points), while regularizing the update magnitude. This step implicitly reasons about missing geometry to correct alignment and eliminate collisions.

Key Contributions

First Framework for Partial Bimanual Grasping: PartialBiGrasp is the first method, to the authors' knowledge, designed specifically for generating bimanual grasps from single-view partial point clouds. It decomposes the task into single-grasp generation, force-closure pairing, and local geometry-conditioned refinement.
Occupancy-Conditioned Reasoning: The paper introduces a pipeline leveraging convolutional occupancy networks and grasp-aligned local features to implicitly infer graspability, local geometry, and collision-free regions without explicit reconstruction.
Robust Evaluation: The approach is validated through analytical force-closure metrics, large-scale simulations, and real-world robot deployments on noisy partial point clouds of novel objects.

Experimental Results

The method was evaluated on the DG16M dataset and a real-world dataset captured with a Realsense D455, comparing against baselines including ContactGraspNet (with various pairing strategies), DAGDiff (adapted for partial views), and reconstruction-based pipelines (RecGen + DAGDiff).

Quantitative Performance: PartialBiGrasp significantly outperformed baselines. On the DG16M test set, it achieved a Force Closure (FC%) of 55.16% and a Grasp Success Rate (GS%) of 67.87%, compared to the next best FC% of 22.35% (DAGDiff). On the real-world dataset, it achieved an FC% of 51.06% and GS% of 81.54%.
Collision Reduction: The method demonstrated a substantial reduction in Pair Collision Rate (PC%), achieving 17.48% on DG16M and 38.80% on real-world data, significantly lower than baselines which often exceeded 50%.
Ablation Studies: Removing the local refinement step or the residual connection between global and local features resulted in significant performance degradation, confirming that both global context and fine-grained local reasoning are necessary for handling partial observations.

Significance and Claims

The paper claims that PartialBiGrasp addresses a critical gap in robotic manipulation by enabling robust dual-arm grasping under realistic, incomplete observation conditions. By reasoning about hidden local geometry rather than relying on potentially inaccurate shape reconstruction, the framework avoids the "deceptive cues" that lead to unstable grasps in prior methods.

The authors emphasize that their unified framework allows for efficient, physically stable grasp generation without the computational overhead of dense reconstruction or the need for complete object meshes. The results demonstrate that implicit geometric reasoning can successfully infer grasp-relevant properties (like thickness and clearance) from sparse data, leading to higher success rates and fewer collisions in both simulation and real-world heterogeneous dual-arm setups.

Limitations: The authors note that the framework does not explicitly model grasp reachability or motion feasibility for specific robot embodiments during generation. Consequently, predicted grasp pairs may require validation by a motion planner to ensure kinematic feasibility and avoid arm collisions during execution.