Technical Summary: SteelDS Dataset

Problem Statement

The recycling of metals, particularly the separation of copper impurities from shredded E40-grade steel scrap, is a critical bottleneck in industrial post-magnetic sorting. While Sensor-Based Sorting (SBS) using X-ray Transmission (XRT) or Hyperspectral Imaging (HSI) is effective, these modalities are capital-intensive and offer lower spatial resolutions. Conversely, standard vision-based sorting using deep learning and RGB cameras offers a cost-effective alternative but lacks specialized, high-fidelity datasets for this specific domain.

Existing waste datasets (e.g., TACO, TrashNet, ZeroWaste) focus on macroscopic, intact household or municipal waste, failing to address the mechanically shredded, micro-scale, and stochastically deformed nature of industrial scrap. Furthermore, previous attempts to classify shredded metal scrap relied on coarse bounding boxes or image-level annotations, which cannot delineate the highly irregular boundaries and fine strands typical of deformed scrap. There is a significant gap in publicly available, instance-level segmentation benchmarks for steel-copper separation in a heterogeneous, post-magnetic sorting stream.

Methodology

Data Acquisition

The SteelDS dataset was captured in a controlled laboratory environment at the Technical University of Leoben to replicate industrial conveyor belt conditions.

• Material Source: 497 shredded metal objects (396 steel, 101 copper) sourced from Scholz Austria GmbH. The material represents the post-magnetic fraction of E40-grade steel scrap (fragmented old steel from end-of-life vehicles and white goods).
• Physical Setup: A 1.0 m wide conveyor belt operated at 0.4–0.5 m/s. Objects were arranged in two configurations: isolated particles (minimum 1 cm spacing) and dense clusters (no forced spacing) to simulate varying industrial densities and occlusions.
• Imaging: A GoPro Hero 11 camera captured 4K Ultra HD (3680 × 2160) video at 100 fps from a strict nadir (top-down) perspective. Diffuse LED lighting minimized reflections on metallic surfaces, and an enclosure blocked ambient light.
• Limitations: The single-view 2D setup inherently occludes copper components located on the underside of particles, a standard constraint in industrial 2D optical sorting.

Annotation and Pre-processing

• Frame Extraction: Raw 100 fps video was downsampled to 20 fps, and frames were extracted as lossless PNG images.
• Annotation Tool: A custom semi-autonomous tool (FOST) generated mask proposals, which were manually corrected and verified by human annotators.
• Annotation Rules: Strict pixel-level guidelines were enforced. Masks traced physical boundaries, excluding cast shadows and motion blur trails. Fragments resulting from transport breakage were labeled as independent instances.
• Classes: Objects were categorized into two primary classes: Steel (ferrous components) and Copper (containing visible copper components). Additionally, objects were heuristically grouped by size (S1–S5 for steel, C1–C3 for copper) based on visual inspection and handling weight, though these size groups are not distinct ground-truth classes.

Dataset Structure

The dataset comprises 24,297 labeled frames across five subsets (a1–a5) with increasing complexity:

• Subsets a1–a3: Cumulative object inclusion with >1 cm spacing.
• Subset a4: Dense clusters with no spacing (high occlusion).
• Subset a5: A holistic evaluation set combining all materials and both spacing configurations.
• Splits: Strict isolation is enforced between training, validation, and test sets; objects are never shared across splits.

Key Contributions

1. Instance-Level Segmentation Benchmark: SteelDS provides the first publicly available, high-resolution video dataset specifically for instance segmentation of mechanically shredded E40 steel and copper scrap.
2. Realistic Industrial Complexity: Unlike prior datasets featuring intact objects, SteelDS captures the stochastic deformation, torn edges, and structural fragmentation inherent to industrial shredding.
3. Pixel-Wise Ground Truth: The dataset offers precise pixel-wise segmentation masks, enabling the training of models for fine-grained boundary delineation rather than just coarse detection.
4. Open-World Simulation: By including varying object densities, occlusions, and complex topologies (e.g., copper wires embedded in steel), the dataset addresses the "open-world" challenge where models must handle unpredictable shapes and mixed materials.

Experimental Results

The authors established baselines using state-of-the-art object detection and instance segmentation models: YOLOv8n-seg, YOLO11n-seg, YOLO12n-seg, YOLO26n-seg, and Mask R-CNN (ResNet-50). Models were trained with 5 different random seeds to ensure reproducibility.

• Performance Hierarchy: Mask R-CNN consistently outperformed all YOLO nano architectures across all subsets. For example, on Subset a1, Mask R-CNN achieved a combined Fitness score of 189.65, compared to a maximum of 168.74 for YOLOv8n. This gap is attributed to Mask R-CNN's two-stage architecture and superior spatial feature preservation.
• Detection vs. Segmentation: Bounding box detection (mAP50-95) consistently outperformed instance segmentation (mAP50-95). For Subset a1, detection mAP was 86.63%, while segmentation mAP was 81.67%.
• Material Disparities:
  • Steel: Generally easier to detect due to rigid, bulky structures.
  • Copper: More challenging for segmentation due to high specular reflectance, tangled wire bundles, and mixed-material assemblies (e.g., steel pins in copper nets). However, copper detection performance was comparable to steel in some metrics, suggesting bounding boxes are robust to surface texture, while segmentation struggles with fine boundaries.
• Scale Sensitivity: Performance dropped significantly for medium-sized objects compared to large objects. Small objects were omitted from metrics as the dataset lacks ground-truth instances in that dimension.
• Failure Modes: Models exhibited specific errors, including:
  • Mis-segmenting the internal background of ring-shaped objects as part of the object.
  • Fragmenting long, twisted rods or crumpled sheets into multiple instances.
  • Struggling with copper wire bundles due to thin, multidirectional strands.

Significance and Claims

The paper positions SteelDS as a foundational resource for advancing automated sorting in metal recycling. Its significance lies in:

• Bridging the Data Gap: It addresses the scarcity of specialized resources for industrial metal scrap, particularly for steel-copper separation, which has been hindered by proprietary data restrictions.
• Enabling Advanced Perception: By providing pixel-wise baselines for known targets in a heterogeneous stream, the dataset supports the development of robust open-world sorting architectures capable of handling extreme intra-class variance.
• Benchmarking Utility: The dataset serves as a rigorous testbed for evaluating the limits of current computer vision models in handling stochastically deformed, high-reflectivity industrial materials.

The authors note that while baseline models show promise, the dataset reveals specific challenges (complex shapes, hidden parts, mixed materials) that remain open problems for the field, motivating further research into more robust segmentation and classification algorithms.