Technical Summary: Enhanced Seam Segmentation for Automated Welding Robot in Construction Through Transfer Learning

Problem Statement

Autonomous robotic welding in construction environments faces significant challenges due to the inherent characteristics of metallic surfaces, specifically strong specular reflections, saturated highlights, and dynamic illumination. These factors frequently obscure thin weld seam boundaries, leading to fragmented predictions, discontinuous masks, and unstable centerline extraction. Such segmentation failures can propagate into unreliable robotic trajectories, causing endpoint detection failures or unsafe welding execution.

While existing deep learning-based segmentation models (e.g., BiSeNetV2, DeepLabV3+) offer a balance between accuracy and efficiency, they often struggle with reflection robustness and geometric continuity in real-world construction scenarios. Previous approaches have primarily attempted to solve this by increasing architectural complexity (e.g., adding attention mechanisms or deeper encoders) or relying on specialized hardware. However, these methods often fail to address the core issue of learning instability under highly reflective conditions without incurring computational overhead that is prohibitive for edge-deployed robotic platforms.

Methodology

The proposed study introduces a reflection-robust seam segmentation framework that enhances the BiSeNetV2 backbone through transfer learning and a hybrid Cross-Entropy–Lovász (CE–Lovász) loss, rather than modifying the network architecture.

Core Components

1. Backbone Architecture: The framework utilizes the standard BiSeNetV2, a lightweight, real-time architecture featuring a dual-branch design (Detail Branch and Semantic Branch) integrated via a Bilateral Guided Aggregation module. No structural changes are made to the network, preserving identical FLOPs, parameter count, and inference speed.
2. Transfer Learning Strategy:
   • Initialization: The model is first pretrained for 1,000 epochs using Online Hard Example Mining (OHEM) loss to establish a stable initialization capable of handling class imbalance and reflective noise.
   • Fine-Tuning: A subsequent transfer learning stage involves fine-tuning the pretrained model for a single epoch using the proposed hybrid loss. Three fine-tuning configurations were evaluated: decoder-only, partial, and full fine-tuning. Full fine-tuning yielded the best results.
3. Hybrid Loss Function:
   • The framework employs a weighted combination of Cross-Entropy (CE) loss and Lovász-Softmax loss: $L_{Total} = \lambda L_{CE} + (1-\lambda) L_{Lovasz}$.
   • CE Loss: Provides stable pixel-wise supervision.
   • Lovász-Softmax Loss: Directly optimizes the Intersection over Union (IoU) at the region level, encouraging connected seam regions and preserving global structural continuity.
   • Optimization: The weighting parameter $\lambda$ was tuned via sensitivity analysis, with the optimal value found at 0.04. This indicates that a Lovász-dominant regime (96% weight) is crucial for maintaining seam continuity under severe reflection, while a small CE component (4% weight) aids in local boundary refinement.
4. Dataset: The study utilizes the WJ3600 dataset, an expansion of the WJ1000 dataset, containing 3,600 images with diverse backgrounds, illumination variations, and specular reflection intensities. The semantic classes were simplified to three: Background, Plate, and Joint.

Key Contributions

• Reflection-Aware Framework: Development of a framework that significantly improves robustness against specular metallic reflections and fragmented seam structures without architectural expansion.
• Learning-Stability-Oriented Optimization: Demonstration that combining transfer learning with a hybrid CE–Lovász loss improves both pixel-level accuracy and region-level seam continuity. This approach reframes reflection robustness as a learning-stability problem rather than a model-capacity problem.
• Trajectory-Stable Representation: The method produces geometrically stable seam representations suitable for downstream robotic tasks (seam tracking, waypoint generation) by recovering severe failure cases that typically result in zero-IoU predictions.
• Architecture-Dependent Efficacy: Comprehensive analysis across BiSeNetV2, DeepLabV3+, UNet, and SegFormer-B0 reveals that the proposed optimization strategy is particularly effective for lightweight, real-time architectures (like BiSeNetV2) which prioritize spatial continuity over large-scale semantic context.
• Zero-Overhead Improvement: The framework achieves substantial performance gains while maintaining identical computational complexity (FLOPs, parameters, latency) to the baseline.

Experimental Results

Extensive experiments were conducted under identical training and evaluation conditions.

• Quantitative Performance:
  • The proposed framework (BiSeNetV2 + Full Fine-tuning + CE–Lovász) achieved a Joint IoU of 81.76% and an mIoU of 90.73%.
  • This represents a +22.36 percentage point improvement in Joint IoU over the OHEM-based baseline (59.40%).
  • Recall improved from 69.86% to 94.10%.
• Recovery of Failure Cases:
  • A critical metric, Recovery Rate, was introduced to measure the ability to recover from severe zero-IoU failures. The proposed method achieved a 96.33% Recovery Rate for BiSeNetV2, successfully restoring seam continuity in cases where the baseline model completely failed.
  • In contrast, heavier architectures like DeepLabV3+ showed significantly lower recovery rates (4.84%) despite moderate IoU improvements.
• Computational Efficiency:
  • The proposed method maintained the baseline's 22.26 GFLOPs, 3.05M parameters, and 161.91 FPS (on an NVIDIA A100).
  • Latency remained at 6.18 ms, and GPU memory usage was unchanged at 171.13 MB.
• Qualitative Analysis:
  • Visual comparisons demonstrated that the proposed method reconstructs geometrically stable, continuous seam structures even under extreme glare, whereas baseline predictions often resulted in fragmented or missing seams.
  • Confidence map analysis showed that after fine-tuning, prediction confidence became concentrated along true seam regions rather than spreading into reflective artifacts.

Significance and Claims

The paper posits that robust seam segmentation in robotic welding is fundamentally a learning-stability issue rather than an architectural capacity limitation. The primary significance of this work lies in demonstrating that:

1. Optimization Strategy > Architectural Complexity: Substantial gains in reflection robustness and seam continuity can be achieved through principled transfer learning and hybrid loss optimization without increasing model size or computational cost.
2. Criticality of Failure Recovery: In robotic welding, recovering isolated catastrophic failures (zero-IoU cases) is more critical than marginal improvements in average overlap metrics, as a single failure can disrupt the entire robotic trajectory.
3. Practical Deployability: The framework offers a practical, lightweight solution for edge-deployed robotic systems operating under strict real-time constraints and reflective construction environments.
4. Limitations: The authors acknowledge limitations including the specific diversity of the welding scenes, potential annotation uncertainty under severe reflection, and the lack of closed-loop robotic welding experiments in the current study. However, the improved geometric continuity suggests strong applicability for robotic trajectory generation.

The study concludes that by focusing on learning stability and region-level consistency, the proposed framework provides a computationally efficient path to reliable robotic welding perception in challenging industrial environments.