💻 computer science

TADNet: Transparency-Aware Feature Aggregation with Structural Enhancement for Transparent Object Depth Completion

This paper proposes TADNet, a hierarchical encoder-decoder network that integrates a cascaded Swin-Transformer encoder with Transparency-Aware Attention and Structural Feature Enhancement modules to effectively address depth noise and geometric distortion in transparent object depth completion, achieving superior performance on the ClearGrasp and TransCG datasets.

Original authors: Xinyang Cai, Yiwen Qi, Hedan Liu

Published 2026-09-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xinyang Cai, Yiwen Qi, Hedan Liu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to teach a robot to pick up a glass of water. To a human, the task is simple: see the glass, reach out, and grasp it. To a robot equipped with a standard depth camera, however, the glass is a ghost. These cameras work by bouncing light off surfaces to measure distance, but transparent objects like glass or clear plastic do not behave like solid walls. Instead of reflecting the light back cleanly, they bend it, scatter it, or let it pass straight through to the table behind. The result is a digital map of the world that is full of holes, static, and confusing distortions right where the robot needs to see most clearly. This inability to "see" through clear materials has long been a major bottleneck for robots trying to navigate our everyday environments, from sorting recycling bins to serving drinks in a home.

For years, researchers have tried to solve this by using complex math to guess where the missing pieces of the puzzle should go, or by training computers to recognize the shape of glass from thousands of photos. While these methods have improved, they often struggle to balance two conflicting needs: understanding the big picture of the room to figure out where a transparent object might be, while also keeping the tiny, sharp edges of that object crisp and accurate. A new study by researchers at Fuzhou University and Xiamen University Tan Kah Kee College introduces a fresh approach called TADNet. This system is designed specifically to fill in the missing depth information of transparent objects, allowing a robot to see a glass not as a glitch in the camera, but as a solid, graspable object with a clear shape and position.

The core of this new system is a digital architecture that mimics how a human might look at a scene. When a person looks at a glass on a table, they use the background—the table, the wall, the light—to infer the shape of the glass, while also focusing on the specific edges where the glass meets the table. TADNet does something similar by using two different types of digital processing. First, it uses a powerful engine based on a technology called a Transformer, which is excellent at looking at the entire image at once to understand the context. This helps the system realize, "Ah, that blurry patch is likely a glass because of the table behind it." At the same time, it uses traditional, precise processing to hold onto the fine details, ensuring the edges of the glass don't get smeared out.

What makes this approach unique is how it treats the transparent parts of the image differently from the solid parts. The researchers realized that a standard computer vision model tries to apply the same rules to everything, which fails when light behaves strangely. TADNet, however, uses a special "transparency-aware" filter. If the system detects a region that is likely transparent, it switches to a mode that gathers information from the whole scene to guess the depth. If it sees a solid object, it focuses on preserving the local details. This dual strategy allows the system to repair the broken depth data without losing the sharpness of the object's outline. To further sharpen the result, the system includes a module that specifically looks for sudden changes in the surface, like the rim of a cup or the corner of a box, ensuring these critical structural lines are reconstructed accurately.

The team tested their creation on two major collections of data: one containing thousands of computer-generated images of plastic objects and another with over 57,000 real-world photos taken by a robot in a lab. In these tests, TADNet proved highly effective. When compared to the best existing methods, it produced depth maps that were significantly more accurate, with fewer errors in the distance measurements. In one specific test involving real-world objects the robot had never seen before, the system correctly estimated the depth for nearly 99 percent of the pixels within a very tight margin of error. This level of precision is crucial because even a small mistake in depth perception can cause a robot to miss a grasp or knock an object over.

Perhaps most importantly, the system showed it could learn from simulated data and apply that knowledge to the real world. The researchers trained the model on synthetic images and then tested it on real photos, a common challenge in robotics where real-world data is hard to collect. TADNet maintained its high performance, suggesting that the way it separates transparency from solidity is a robust solution that works even when the lighting and objects change. The study does not claim to have solved every problem in robot vision, but it demonstrates that by teaching machines to distinguish between what is see-through and what is solid, and by treating them differently, we can give robots a much clearer view of the transparent world they are increasingly asked to navigate.

Technical Summary: TADNet for Transparent Object Depth Completion

Problem Statement
Accurate depth perception of transparent objects is critical for robotic grasping and scene understanding. However, RGB-D sensors struggle with transparent materials due to complex optical phenomena such as refraction and reflection. These phenomena cause severe depth noise, missing measurements, and geometric distortion in captured depth maps. Existing learning-based depth completion methods often fail to balance global contextual reasoning with local structural modeling, leading to suboptimal reconstruction quality in transparent regions.

Methodology
The paper proposes TADNet, a hierarchical encoder-decoder network designed specifically for transparent object depth completion. The architecture integrates the following core components:

Swin-Transformer Encoder: To capture long-range contextual dependencies essential for inferring depth in corrupted transparent regions, the model employs a cascaded Swin-Transformer encoder. This extracts multi-scale global contextual features from the input RGB-D image (concatenated RGB and raw depth).
Transparency-Aware Attention (TAA) Module: This module addresses the divergent feature modeling needs of transparent versus non-transparent regions. Guided by pixel-level transparency masks, it performs region-differentiated feature aggregation:
- Transparent Regions: Utilizes a Global Context (GC) Block to aggregate scene-level information from surrounding stable objects to repair depth surfaces.
- Non-Transparent Regions: Uses dense convolutional blocks to preserve local geometric details and structural integrity.
Structural Feature Enhancement (SFE) Module: Designed to optimize the representation of structural transitions, this module employs a parallel attention mechanism combining Channel Attention (CAM) and Spatial Attention (SAM). It enhances sensitivity to depth discontinuities and fine-grained geometric changes caused by refractive blurring.
CDCU Decoder: The decoder utilizes cascaded Convolutional Dense Convolutional Upsampling (CDCU) units. Unlike standard transposed convolutions which can introduce checkerboard artifacts, CDCU blocks use dense feature reuse and pixel rearrangement (via DUC blocks) to progressively upsample features while maintaining spatial continuity and anti-aliasing properties.

Key Contributions
The authors highlight four primary contributions:

Network Architecture: The proposal of TADNet, which combines a Swin-Transformer encoder for global context with a CDCU-based decoder for dense depth recovery, effectively preserving structural details during depth restoration.
Region-Aware Mechanism: The design of the TAA module, which utilizes transparency masks to guide feature aggregation, allowing the network to trade off global context for local details based on the specific region (transparent vs. non-transparent).
Structural Optimization: The introduction of the SFE module, which enhances feature sensitivity to structural transitions and depth discontinuities through parallel attention mechanisms.
Empirical Validation: Comprehensive experiments demonstrating competitive performance against state-of-the-art methods on standard datasets.

Experimental Results
The method was evaluated on the ClearGrasp, TransCG, and Omniverse Object (OOD) datasets.

ClearGrasp Dataset: On the "Real-known" subset, TADNet achieved an RMSE of 0.020 and REL of 0.024, performing comparably to the top-performing CAGT method and outperforming other baselines. On the "Real-novel" subset (testing on unseen objects), TADNet achieved an RMSE of 0.013 and a $\delta_{1.10}$ accuracy of 99.23%, demonstrating strong generalization. Notably, TADNet's model size (12.5 MB) is significantly smaller than CAGT (40.7 MB).
TransCG Dataset: In real-world scenarios, TADNet achieved the best performance in RMSE (0.017), REL (0.028), and MAE (0.013) among evaluated reproducible methods, attributed to the balance between global reasoning and local structural enhancement.
Cross-Domain Performance: When trained on synthetic data (ClearGrasp + OOD) and tested on real-world TransCG data, TADNet demonstrated robust sim-to-real transfer capabilities, ranking second only to CAGT in most metrics, with a superior $\delta_{1.25}$ score (99.94%).
Ablation Studies: Removing the TAA module increased RMSE from 0.020 to 0.024, confirming its importance for transparent region modeling. Similarly, removing the SFE module or replacing CDCU units with standard convolutions led to performance degradation across all metrics.

Significance and Claims
The paper claims that TADNet effectively improves depth completion accuracy for transparent objects by addressing the specific challenge of balancing global context with local structural details. The authors assert that the proposed method achieves competitive performance compared to existing approaches and exhibits good generalization in real-world scenarios. The work is presented as a solution to the limitations of current models that struggle to simultaneously capture global context and fine-grained structural information in transparent regions. The authors do not claim to solve all depth perception issues but position TADNet as a significant step forward specifically for the domain of transparent object depth completion.

More like this