Technical Summary: Enhanced Seam Segmentation for Automated Welding Robot in Construction Through Transfer Learning

Problem Statement

Robotic welding in construction environments faces significant challenges due to the harsh visual conditions inherent to metallic surfaces. Strong specular reflections, saturated highlights, and dynamic illumination often obscure thin weld seam boundaries, leading to fragmented segmentation predictions, discontinuous masks, and unstable centerline extraction. These failures propagate into unreliable robotic trajectories, posing risks to safety and weld quality.

While deep learning-based semantic segmentation (e.g., BiSeNetV2) offers a balance between accuracy and real-time inference, existing approaches often struggle with reflection robustness and seam continuity. Previous studies have primarily addressed these issues by increasing architectural complexity (e.g., adding attention mechanisms or deeper encoders) or relying on specialized hardware. However, this often introduces computational overhead unsuitable for edge-deployed robotic platforms. Furthermore, standard pixel-wise optimization (e.g., Cross-Entropy) often fails to preserve the geometric continuity required for stable robotic trajectory generation, particularly when dealing with severe zero-IoU failure cases where the seam is completely missed.

Methodology

The proposed framework addresses these limitations not by modifying the network architecture, but through learning-stability-oriented optimization using transfer learning and a hybrid loss function.

1. Backbone and Architecture

The study utilizes BiSeNetV2, a lightweight, real-time semantic segmentation architecture featuring a dual-branch design (Detail Branch for spatial information and Semantic Branch for context). The architecture remains unmodified to preserve identical FLOPs, parameter count, and inference speed as the baseline.

2. Dataset

The experiments utilize the WJ3600 dataset, an expansion of the WJ1000 dataset containing 3,600 images with diverse backgrounds, camera distances, and illumination conditions. For this study, the original six semantic classes were simplified into three: Background, Plate, and Joint (seam). This simplification focuses the model on robust seam localization rather than fine-grained joint-type classification.

3. Transfer Learning Strategy

The training process involves a two-stage approach:

• Initialization: The BiSeNetV2 model is first pretrained for 1,000 epochs using Online Hard Example Mining (OHEM) loss to establish a stable initialization capable of handling class imbalance and reflective noise.
• Fine-Tuning: A transfer learning phase is applied for a single epoch using a hybrid Cross-Entropy (CE) – Lovász–Softmax loss. Three fine-tuning configurations were evaluated:
  • Decoder-only fine-tuning.
  • Partial fine-tuning (unfreezing later semantic/decoder layers).
  • Full fine-tuning (unfreezing all parameters), which yielded the best results.

4. Hybrid Loss Function

The core innovation is the CE–Lovász hybrid loss, formulated as:
$$L_{Total} = \lambda L_{CE} + (1-\lambda) L_{Lovasz}$$

• Cross-Entropy ($L_{CE}$): Provides stable pixel-wise supervision.
• Lovász–Softmax ($L_{Lovasz}$): Directly optimizes the Intersection over Union (IoU), encouraging region-level consistency and connected seam structures.
• Weighting ($\lambda$): A sensitivity analysis determined that a Lovász-dominant regime (specifically $\lambda = 0.04$) provides the optimal balance, preserving global seam continuity while stabilizing local boundaries.

Key Contributions

1. Reflection-Aware Framework: Development of a framework that improves robustness against specular metallic reflections without altering the network architecture.
2. Learning-Stability Optimization: Introduction of a hybrid CE–Lovász loss combined with transfer learning to enhance both pixel-level accuracy and region-level seam continuity.
3. Trajectory-Stable Representation: Demonstration that the proposed method recovers severe failure cases (zero-IoU), providing geometrically stable seam representations suitable for downstream robotic trajectory generation.
4. Architecture-Dependent Efficacy: Comprehensive analysis showing that optimization strategies significantly impact lightweight real-time models (like BiSeNetV2) more effectively than heavier semantic models (like DeepLabV3+ or SegFormer) in this specific domain.
5. Computational Efficiency: Achievement of substantial performance gains while maintaining identical computational complexity (FLOPs, parameters, and latency) to the baseline.

Experimental Results

The proposed framework was evaluated against baselines using Joint IoU, mIoU, Recall, and a novel Recovery Rate metric (measuring the percentage of baseline zero-IoU failures successfully recovered).

• Performance Gains: The proposed method (BiSeNetV2 + Full Fine-tuning + CE–Lovász) achieved a Joint IoU of 81.76% and an mIoU of 90.73%.
• Improvement over Baseline: This represents a +22.36 percentage point improvement in Joint IoU over the OHEM-based baseline (59.40%).
• Failure Recovery: The framework achieved a Recovery Rate of 96.33%, successfully reconstructing seam geometries in cases where the baseline model produced completely fragmented or missing predictions (zero-IoU).
• Architectural Comparison: While DeepLabV3+, UNet, and SegFormer-B0 showed some improvement, they exhibited unstable optimization behavior and significantly lower recovery rates (e.g., DeepLabV3+ recovered only 4.84% of failures). BiSeNetV2 proved most suitable for this task due to its superior spatial-preservation capabilities.
• Efficiency: The improvements were achieved without increasing FLOPs (22.26 G), parameter count (3.05 M), or inference latency (6.18 ms).

Significance and Claims

The paper posits that robust seam segmentation in reflective environments is fundamentally a learning-stability problem rather than an architectural capacity limitation. By reframing the challenge from "adding complexity" to "stabilizing optimization," the study demonstrates that:

1. Optimization Strategy > Architecture: A lightweight model with a tailored optimization strategy (hybrid loss + transfer learning) outperforms heavier architectures in terms of both accuracy and, crucially, the recovery of catastrophic failures.
2. Criticality of Continuity: In robotic welding, preserving the geometric continuity of thin seams is more critical for trajectory planning than maximizing average pixel-wise overlap. The Lovász component is essential for this global consistency.
3. Practical Deployment: The framework offers a practical solution for edge-deployed robotic systems, providing substantial robustness gains without the computational overhead of architectural expansion or additional hardware sensors.
4. Failure Awareness: The study highlights the importance of evaluating "Recovery Rates" for rare but critical failure cases (zero-IoU), as a single missed seam can lead to unsafe robotic execution, making the recovery of these cases more significant than marginal improvements in average metrics.

The authors conclude that their approach provides a computationally efficient, reflection-robust perception solution that enables more reliable robotic welding in challenging construction environments.