💻 computer science

From Transparent Labware Segmentation to Collision Avoidance: A Real-Time Edge-Aware Perception Pipeline

This paper introduces a lightweight, real-time edge-aware instance segmentation framework and a corresponding dataset that enable robotic collision avoidance with transparent laboratory glassware by leveraging boundary contours to achieve high accuracy and efficiency in 3D perception.

Original authors: Shijun Ding, Chen Qian, Weiwei Shang, Junlin Xiong

Published 2026-08-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shijun Ding, Chen Qian, Weiwei Shang, Junlin Xiong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to walk through a room full of invisible ghosts. In the world of robotics, "seeing" is usually easy if objects are solid and colorful; a camera can spot a red box or a blue chair instantly. But when those objects are made of clear glass—like the beakers and flasks used in chemistry labs—they become a nightmare for robot eyes. Glass doesn't just sit there; it bends light (refraction), flashes it back (reflection), and often looks exactly like the background behind it. Standard cameras, which rely on depth sensors or simple colors, often get confused, seeing nothing but empty space or a distorted mess. This is a huge problem because robots need to move safely around these fragile tools without smashing them. If a robot can't "see" the invisible glass, it can't avoid crashing into it. This paper tackles that specific headache: how to give a robot the superpower to spot clear glass in real-time, just by looking at it, so it can dance around the lab equipment without breaking a sweat.

The researchers behind this study, Shijun Ding, Chen Qian, Weiwei Shang, and Junlin Xiong, decided to stop trying to see the inside of the glass (which is messy and confusing) and instead focus entirely on the edges. Think of it like trying to find a clear plastic bag in a pile of laundry. You can't see the plastic itself, but you can see the crinkly outline where the light bends. The team built a new "brain" for robots that acts like a detective who only looks for outlines. They took a standard, fast image-recognition system and gave it two special upgrades. First, they added a "boundary detective" that ignores the confusing inside of the glass and zooms in on the sharp lines where the glass meets the air. Second, they added a "focus filter" that helps the robot ignore the shiny, distracting reflections that usually trick cameras.

To train this new brain, the team couldn't just use old photos because they didn't have enough of the right kind. So, they created their own massive library called LabGlass-IS. It's like a photo album containing 3,485 pictures of 21 different types of lab glassware—beakers, test tubes, flasks, and droppers—taken in real, messy laboratories. They manually drew outlines around every single piece of glass in these photos, teaching the robot exactly what to look for.

When they tested their new system, the results were impressive. The robot's "brain" could spot and outline the glass in just 7.1 milliseconds per frame. That's faster than a human eye can blink! In terms of accuracy, the system scored a 97.80 on a scale that measures how well it traced the edges (called the Boundary F-score). This was a huge leap forward, beating the next best method by nearly 19 points. Even better, the system was incredibly lightweight, using only 2.85% of the computer power required by its closest competitor, meaning it could run on small, portable robot computers rather than needing a giant supercomputer.

But seeing the glass is only half the battle; the robot also needs to know where the glass is in 3D space to avoid hitting it. The team solved this by having the robot move its camera around the glass from different angles, like a person tilting their head to see a shiny object better. By combining these different views, the robot could calculate the exact center point of each glass object in 3D space. They then wrapped each object in a "safety bubble"—a simple, conservative 3D box that was slightly larger than the glass itself. This ensured that even if the robot's guess was slightly off, it would still steer clear.

In real-world tests with an actual robot arm, this system worked like a charm. The robot successfully avoided collisions in 93.3% of the trials, even when the glass was small, overlapping, or in tricky lighting. The whole process, from seeing the glass to planning a safe path around it, happened so quickly that the robot didn't even have to pause to think. The authors conclude that while this system is currently a master at avoiding crashes, it isn't yet ready to pick up the glass gently. However, it proves that with the right mix of edge-detecting vision and smart 3D guessing, robots can finally navigate the invisible world of transparent glassware without breaking a single beaker.

Technical Summary: From Transparent Labware Segmentation to Collision Avoidance

Problem Statement

Transparent laboratory glassware (e.g., beakers, flasks, test tubes) presents a significant challenge for robotic perception systems. Unlike opaque objects, these vessels lack stable interior textures and exhibit complex optical phenomena such as refraction, specular reflection, and transparency. Consequently, conventional depth sensors (RGB-D, LiDAR) often fail, producing missing depth data or measurement distortions. Furthermore, existing segmentation methods face a trade-off: high-accuracy models (e.g., Transformer-based or foundation models like SAM) are computationally too heavy for real-time deployment, while lightweight models often struggle with boundary precision on transparent objects. Additionally, there is a scarcity of real-world, instance-level datasets specifically for laboratory glassware, with existing benchmarks relying heavily on synthetic data or coarse semantic annotations insufficient for precise robotic manipulation.

Methodology

The authors propose a complete perception-to-action pipeline consisting of three core components:

1. Edge-Aware Instance Segmentation Framework

The core perception module is built upon a YOLOv5-Seg backbone, enhanced with three specific modifications to address the unique challenges of transparent objects:

Multi-scale Edge Detection Branch: An auxiliary branch extracts multi-scale feature maps ( $P_2$ to $P_5$ ) from the backbone. These are aligned, fused, and processed through a lightweight Atrous Spatial Pyramid Pooling (ASPP) module to generate an edge-aware feature representation ( $F_e$ ). This branch provides explicit boundary supervision during training but does not add inference overhead.
Edge-Guided Attention Fusion: The edge features are integrated into the network neck via a Bottleneck Attention Module (BAM). These features act as attention inputs to guide the network to emphasize boundary regions and suppress background noise during inference, rather than directly participating in loss computation.
Parameter-Free SimAM Module: To improve feature discrimination without increasing parameter count, the SimAM attention module is inserted after C3 modules. It uses an energy-based measure to assign 3D attention weights, suppressing misleading high-energy responses caused by refraction and specular highlights while enhancing informative features.

2. Multi-view 3D Centroid Estimation

To enable collision avoidance without dense 3D reconstruction, the system estimates the 3D position of each segmented vessel:

2D Centroid Extraction: The geometric centroid of the predicted segmentation mask is calculated for each view.
Triangulation: Using the camera intrinsic matrix and the robot's pose, 2D centroids are back-projected into 3D rays. A least-squares multi-ray triangulation method estimates the 3D centroid position by minimizing the distance from the point to the collection of rays.
Robustness: Mask centroids are preferred over bounding box centers as they are more stable under partial occlusion and reduce triangulation jitter.

3. Conservative Obstacle Modeling

For motion planning, the estimated 3D centroid is used to construct an axis-aligned cubic bounding volume. This volume is conservatively sized to fully enclose the object, accounting for segmentation uncertainty and geometric variations. These volumes serve as dynamic obstacles in the robot's workspace, allowing for safe trajectory planning using standard collision checking algorithms without requiring precise shape reconstruction.

Key Contributions

Perception Framework: A lightweight, edge-aware instance segmentation model that achieves high boundary precision for transparent objects while maintaining real-time inference speeds.
System Integration: A complete pipeline combining the segmentation model with multi-view 3D centroid estimation and conservative bounding-volume modeling to enable real-time robotic collision avoidance.
Dataset Construction (LabGlass-IS): The introduction of a real-world instance segmentation dataset containing 3,485 images, 21 categories of laboratory glassware, and 6,099 annotated instances. This dataset addresses the lack of fine-grained, real-world data for transparent objects in laboratory settings.

Experimental Results

The proposed method was evaluated on the LabGlass-IS dataset and in real-robot trials:

Segmentation Performance: The model achieved a Boundary F-score (BF) of 97.80, outperforming the YOLO-prompted FastSAM framework by 18.93 BF points and surpassing other state-of-the-art methods (e.g., PointRend, YOLACT++) in boundary precision. For slender objects (glass rods, droppers), the BF reached 98.21.
Efficiency: The system maintains an inference speed of 7.1 ms per frame (140.85 FPS) and requires only 2.85% of the parameters of the closest accuracy competitor (PointRend).
3D Estimation: In 3D centroid estimation trials across 5 scenes, the system achieved a mean Euclidean error of 38.0 mm (max 60.5 mm). The authors note that errors are primarily along the depth axis but are mitigated by the conservative sizing of the bounding volumes.
Collision Avoidance: Real-robot trials across three difficulty levels (1 to 3 objects) resulted in a 93.3% success rate (14/15 trials) for collision-free manipulation. The average re-planning time was under 20 ms.

Significance and Claims

The paper claims that the proposed approach provides a practical, feasible solution for robot collision avoidance in cluttered environments containing fragile transparent objects. By explicitly leveraging the reliability of boundary contours over unstable interior textures, the system bridges the gap between high-accuracy perception and real-time robotic action. The authors emphasize that their method avoids the computational overhead of dense 3D reconstruction while ensuring safety through conservative geometric modeling.

The work is presented as a step toward reliable perception-to-action pipelines for chemical laboratories, though the authors modestly note that the current system is designed primarily for collision avoidance and does not yet support fine-grained geometric understanding or precise manipulation tasks, which are identified as directions for future work.