Technical Summary: Advancing Metallic Surface Defect Detection via Anomaly-Guided Pretraining

1. Problem Statement

Metallic surface defect detection is critical for industrial quality control but faces severe challenges due to data scarcity and high variability. In real-world scenarios, such as steel pipe production, defects often constitute less than 0.1% of daily images, leading to extreme class imbalance and costly data collection.

Existing solutions face a "critical dilemma":

• Natural Image Pretraining: Models pretrained on large-scale natural datasets (e.g., ImageNet, COCO) suffer from a significant domain gap when applied to industrial images.
• Naive Self-Supervised Learning (SSL): Standard SSL methods (contrastive learning or reconstruction) applied to in-domain industrial data often fail. Existing learning objectives struggle to distinguish subtle, low-contrast defect patterns from complex background noise, textures, and variable lighting. Consequently, simply enlarging in-domain datasets without specific guidance yields suboptimal results, sometimes underperforming ImageNet baselines.
• Detector Initialization: Most SSL approaches only pretrain the model backbone, leaving the detector head (neck and head) randomly initialized. This misalignment degrades performance, particularly in data-scarce scenarios.

2. Methodology: Anomaly-Guided Self-Supervised Pretraining (AGSSP)

The authors propose AGSSP, a novel two-stage pretraining paradigm that explicitly guides representation learning using anomaly priors. The framework operates on a large-scale, unlabeled industrial dataset (120,000 images, 61 categories) and consists of three core components:

A. Knowledge-Enhanced Anomaly Detection (KeAD)

To generate the necessary supervision signals (anomaly maps) without manual labels, the authors developed KeAD, an unsupervised method based on the few-shot, multi-modal WinCLIP model.

• Prompt Engineering: Unlike generic prompts (e.g., "Damaged"), KeAD uses detailed, defect-specific text descriptions generated via GPT-4o and refined by expert knowledge (e.g., "A photo of {object} with {defect type} defect, it appears as {defect features}").
• Visual Enhancement: To improve spatial localization, the method integrates Clip Surgery (V-V attention), which replaces Query and Key components with Value components in the attention mechanism. This captures local spatial information more efficiently than WinCLIP's sliding-window approach.
• Output: KeAD generates pixel-wise anomaly heatmaps (zero-shot and few-shot modes) indicating potential defect regions.

B. Stage 1: Anomaly Map Guided Backbone Pretraining (AGBP)

This stage distills knowledge from the generated anomaly maps into the model backbone.

• Distillation Loss: The backbone's high-level feature maps are aligned with the anomaly maps using an L2 loss (for ViT architectures) or cosine distance loss (for heterogeneous architectures like ResNet).
• Strategy: Low-level features are retained from standard ImageNet pretrained models (as they capture universal patterns), while high-level features are guided by the anomaly maps to focus on defect-salient regions.
• Integration: This distillation loss ($L_{distill}$) is combined with existing pretraining tasks (e.g., classification, MoCov3, SimMIM) via a weighted sum ($L_{total} = L_{distill} + \lambda L_{task}$).

C. Stage 2: Anomaly Box Guided Detector Pretraining (AGDP)

This stage addresses the misalignment of the detector head by pretraining it using pseudo-labels derived from the anomaly maps.

• Pseudo-Box Generation: Images with high anomaly scores are identified. A dynamic thresholding strategy (adapted per object category) converts anomaly maps into binary masks. Connected components are extracted, and the top 10 bounding boxes with the highest anomaly scores are retained as pseudo-defect labels.
• Training: The detector (neck and head) is trained on these pseudo-boxes while the backbone remains frozen. This allows the model to learn the concept of "anomalous objects" and aligns the detection components with the downstream task.

3. Key Contributions

1. AGSSP Framework: A two-stage pretraining paradigm that utilizes anomaly maps to holistically align both backbone representation and detector task alignment, significantly enhancing performance in data-scarce settings.
2. KeAD Method: A knowledge-enhanced anomaly detection technique that leverages detailed defect descriptions and V-V attention to generate high-quality anomaly maps for guided pretraining, achieving strong few-shot performance.
3. Large-Scale Dataset: The collection and release of a large-scale, unlabeled industrial dataset containing 120,000 images across 61 categories, alongside two small-scale, pixel-level labeled metallic surface defect datasets (Casting Billet and Steel Pipe) for validation.
4. Comprehensive Validation: Extensive experiments demonstrating that AGSSP outperforms ImageNet and COCO baselines across multiple backbones (CSPDarknet, ResNet, Swin-Base) and detectors (YOLOv8, Faster R-CNN, DINO).

4. Experimental Results

The authors evaluated AGSSP on four metallic surface defect datasets (Casting Billet, Steel Pipe, GC10-Mini, GC10-DET) and a non-metallic dataset (Fabric Defects).

• Performance Gains: AGSSP consistently improved detection metrics. Compared to ImageNet-based models, it achieved up to a 10% improvement in mAP@0.5 and 11.4% in mAP@0.5:0.95.
• Few-Shot Learning: In the extreme few-shot scenario (Casting Billet-Mini50, 50 training samples), AGSSP yielded a massive gain of +41.0% in mAP@0.5 and +24.5% in mAP@0.5:0.95 over the SimMIM baseline.
• Cross-Domain Generalization: On the non-metallic Fabric Defects dataset, AGSSP improved performance by +3.2% (mAP@0.5) and +5.2% (mAP@0.5:0.95), demonstrating robust generalizability.
• Ablation Studies:
  • KeAD: Detailed defect descriptions and V-V attention significantly improved anomaly detection accuracy (e.g., 93.5% image-level AUROC on Steel Pipe).
  • Pretraining Strategy: Pretraining both the backbone and detector (AGSSP) consistently outperformed pretraining the backbone alone (AGBP) or standard SSL methods.
  • Efficiency: The method maintains real-time inference capabilities (e.g., YOLOv8s with AGSSP achieves 94.5 FPS) while matching or exceeding the accuracy of larger models like YOLOv12s.

5. Significance and Claims

The paper claims that AGSSP resolves the domain gap and objective misalignment inherent in current pretraining strategies for industrial defect detection. By explicitly guiding the model to focus on anomalous regions through a unified, anomaly-guided approach, the method enables the learning of robust, domain-relevant representations without the need for costly bounding-box annotations during the pretraining phase.

The authors emphasize that while standard in-domain SSL often fails due to the inability to distinguish subtle defects from noise, their anomaly-guided approach successfully directs the model's attention to defect-salient features. The framework is presented as architecture-agnostic and capable of being integrated with existing specialized architectures, offering a scalable solution for industrial quality control where data is scarce and imbalanced.

Availability: All code, pretrained models, and datasets are publicly available at the project page provided in the paper.