💻 computer science

TransGraspNet: Physically and Geometrically Consistent Manipulation of Transparent Labware

TransGraspNet is a novel framework that ensures physically and geometrically consistent robotic manipulation of transparent liquid-filled glassware by unifying perception and execution through boundary, surface, and physics consistency, thereby achieving high success rates and zero spillage in real-world scenarios.

Original authors: Hailing Hu, Mingyi Zhu, Yiquan An, Yifei Tian, Tianyou Zuo, Lifeng Zhou

Published 2026-08-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hailing Hu, Mingyi Zhu, Yiquan An, Yifei Tian, Tianyou Zuo, Lifeng Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot trying to do the dishes, but instead of ceramic plates, it's handling delicate glass beakers filled with bubbling, colorful chemicals. This is the world of "robot scientists," machines designed to run complex lab experiments without human help. But there's a catch: glass is tricky. To a robot's camera, a clear glass beaker looks like a ghost. It doesn't have its own color or texture; instead, it acts like a funhouse mirror, bending light, showing the background through itself, and creating confusing shiny spots. If a robot tries to grab a glass beaker based on a blurry or wrong picture, it might squeeze too hard, tip the beaker, or worse, spill dangerous liquid everywhere. The core challenge isn't just seeing the glass; it's understanding its shape perfectly and then holding it in a way that keeps it upright and stable, even when the robot is moving fast.

Enter TransGraspNet, a new "brain" for robots designed specifically to solve this glassy puzzle. The researchers behind this system realized that most robots fail because they treat seeing, measuring, and grabbing as three separate steps. It's like trying to build a house by having one person draw the blueprints, a different person build the walls without looking at the drawings, and a third person try to hang the door without checking if the frame is straight. If the first step is slightly off, the whole house collapses. TransGraspNet fixes this by making sure every step agrees with the others, from the first glance at the object to the final grip.

The system works in three magical stages that talk to each other. First, the Perception stage acts like a detective with a magnifying glass. Instead of just guessing where the glass is, it uses special attention tools to trace the exact, sharp edges of the beaker, ignoring the confusing reflections that usually fool robots. It's like learning to see the outline of a clear plastic cup against a busy background by focusing only on the rim.

Next comes the Depth stage, which is where the robot figures out how far away the glass is. Usually, cameras get confused by transparent objects, seeing "holes" in the glass or bleeding colors from the background. TransGraspNet uses the sharp edges it found in the first step to act as guardrails. It tells the robot, "Don't let the depth measurement leak out of the glass; keep the shape smooth and true." This creates a perfect 3D map of the beaker, ensuring the robot knows exactly how round and tall it is.

Finally, the Grasp stage is the moment of truth. Most robots just pick a spot that looks easy to grab. But TransGraspNet is smarter; it thinks like a physicist. It calculates exactly where the center of the liquid is and how the robot's fingers should align to keep the beaker from tipping over. It rejects grabs that might work for a solid rock but would spill a beaker of water. It ensures the robot grabs the glass straight up, right in the middle, so it can lift and move it without spilling a drop.

The researchers tested this system in a real lab with a real robot arm. They didn't just simulate it on a computer; they put it to the test with 50 trials in simple setups and 50 in messy, cluttered ones. The results were impressive: the robot succeeded in grabbing and moving the glass 96% of the time in simple scenes and 86% in the messy ones. But the real magic happened during a "stress test." The robot had to carry a beaker filled with liquid at a speed of 0.5 meters per second (about 1.1 miles per hour) with a sharp acceleration. Despite the speed and the sloshing liquid, the robot achieved zero spillage.

This paper argues that the old way of doing things—where each part of the robot's brain works in isolation—is the reason robots struggle with glass. By forcing the robot to be consistent from the very first image to the final movement, TransGraspNet proves that robots can handle fragile, transparent, and dangerous liquids safely. It's a big step toward a future where robots can truly take over the lab, handling the glassware with the steady hands of a master chemist.

Technical Summary: TransGraspNet

Problem Statement

Autonomous laboratory systems, or "robot scientists," face significant challenges when manipulating transparent laboratory glassware (e.g., beakers, cylinders, test tubes) containing liquids. Unlike structured industrial parts, transparent objects suffer from severe optical ambiguity, including refractions, specular highlights, and background distortions that dominate RGB observations. Furthermore, commodity depth sensors often fail to capture valid returns from glass surfaces, leading to missing data and boundary corruption.

The core issue identified is that existing systems typically employ a cascaded pipeline where detection, depth reconstruction, and grasp planning are optimized independently. This modular approach leads to cross-stage geometric inconsistency:

Imperfect object boundaries induce "depth bleeding" during reconstruction.
Distorted depth surfaces corrupt surface normal estimation.
Task-agnostic grasp scoring produces tilted or off-center grasps that fail under dynamic motion.

In safety-critical scenarios involving liquids, even minor geometric errors can cause unstable grasps, hazardous spillage, or tipping. The paper argues that the solution requires a framework that enforces geometry–physics consistency from perception to execution, rather than simply improving individual modules.

Methodology: TransGraspNet

TransGraspNet is a unified framework designed to enforce structural coherence across three coupled stages: perception, depth reconstruction, and grasp execution.

1. Edge-Guided Boundary Consistency (Perception)

To address the weak internal texture of transparent objects, the perception module (TransGraspNet-Det) enhances boundary reliability:

Architecture: Built on a ResNet-101 + FPN backbone.
Enhanced CBAM (E-CBAM): An attention module is introduced to refine features, suppressing background-induced noise and reflection artifacts.
Dual-Stream Boundary Head: A lightweight Edge Branch predicts contour probability ( $P_{edge}$ ) and is fused with mask features.
Supervision: Edge supervision is generated via morphological erosion of the ground truth mask. The total loss function combines classification, bounding box, mask, and a Dice-style edge loss ( $L_{edge}$ ).
Output: The predicted mask and edge map are explicitly forwarded as geometric priors for the downstream depth reconstruction stage.

2. Surface Consistency via Geometry-Aware Depth Completion

To prevent depth bleeding and preserve surface fidelity, the depth module (TransGraspNet-Depth) utilizes a TDCNet backbone with specific modifications:

Edge-Guided Attention Gate (EGAG): This mechanism adaptively learns RGB feature confidence. It suppresses guidance in texture-rich but geometrically flat regions (where background textures leak through glass) while preserving interior smoothness.
Masked Geometric Retention (MGR) Loss: A custom loss function is applied to preserve object geometry and prevent over-smoothing. It penalizes gradients in valid object regions while allowing smoothing in background areas.
Output: The refined depth map is back-projected to generate a point cloud and accurate surface normals, critical for force-closure analysis.

3. Geometry–Physics Aware Grasp Refinement

The grasp planning stage refines candidates generated by a pre-trained 6-DoF model (e.g., GraspNet-1Billion) by integrating geometric alignment and physical stability constraints:

Geometric Alignment: Scores are adjusted based on radial alignment with the object's principal axis, angular matching of the approach vector, and centroid alignment to ensure the grasp is centered.
Physics Constraints:
- Antipodal Condition: Ensures contact points satisfy friction cone constraints.
- Wrench-Space Robustness: Evaluates the radius of the convex hull of wrenches to ensure stability against external disturbances.
Final Scoring: A weighted sum of raw scores and these physics-aware metrics ( $S_{final}$ ) selects the optimal grasp pose, prioritizing upright, dynamically stable configurations suitable for liquid transport.

Key Contributions

Unified Framework: The establishment of a geometry–physics consistent framework that systematically addresses cross-stage inconsistency in transparent object manipulation.
Boundary-Enforced Reconstruction: The design of a reconstruction strategy (E-CBAM + Edge Branch + EGAG) that preserves geometric structure and surface normal fidelity under severe optical ambiguity.
Physics-Aware Refinement: The development of a grasp refinement mechanism that explicitly incorporates task-level stability constraints (centroid alignment and wrench-space analysis) to ensure safe execution.
Empirical Validation: Comprehensive evaluation on public benchmarks, a dedicated dataset (RobotSci-Glass), and a real robotic platform, demonstrating robust performance in cluttered environments and high-speed liquid transport.

Experimental Results

The system was evaluated on the RobotSci-Glass dataset (constructed for this work) and public benchmarks like Trans10K and ClearGrasp.

Perception: On the RobotSci-Glass dataset, the full method achieved an APmask of 78.5% and a Boundary F-score of 65.3%, outperforming baselines like Mask R-CNN and TransLab, particularly in boundary precision. (Note: The 65.1% score corresponds to the ClearGrasp benchmark in Table IV).
Depth Completion: The proposed method reduced RMSE by 7.3 mm compared to the baseline TDCNet and halved the surface normal error (from 15.2° to 8.4°), indicating superior recovery of physically plausible curvature.
Grasp Quality: Top-1 grasp poses showed a significant reduction in angular error (3.8° vs. 22.5° for baseline) and center offset (8.5 mm vs. 35.2 mm), ensuring grasps are vertically aligned and centered.
Real-World Performance:
- In Scenario-II (Hard) conditions, the system achieved an 86.0% grasp success rate (96.0% in Scenario-I Simple).
- Dynamic Stability: During high-speed liquid transport (0.5 m/s), the system achieved zero spillage, confirming that wrench-constrained scoring effectively suppresses slippage and maintains orientation alignment.

Significance

The paper posits that the core challenge in transparent glassware manipulation is not merely the need for stronger individual models, but the lack of consistency across the perception-action pipeline. By enforcing structural and physical consistency throughout the entire system, TransGraspNet enables reliable real-world operation in safety-critical laboratory automation. The results demonstrate that integrating geometric consistency restoration and physical constraints into the planning layer significantly enhances robustness when handling transparent, fragile, and high-risk chemical vessels, laying a foundation for fully autonomous "robot scientist" capabilities.