Technical Summary: Quality Inspection of Printed Circuit Board Pin Insertion via Semantic Segmentation and Board-Level Feature Extraction

Problem Statement

Quality control during Printed Circuit Board (PCB) assembly is critical for ensuring reliable electronic products. While deep learning has become a standard for detecting various board-level and component defects (e.g., open circuits, short circuits), the specific task of detecting misaligned or tilted pins during or after automated pin insertion remains challenging. Existing methods often focus on component detection or rely on specialized hardware and fixed inspection conditions. Furthermore, pin defects can lead to increased rejection rates and costly rework, yet comparatively little work has addressed the detection and evaluation of PCB pins in industrial environments using robust, automated AI methods that do not depend on specific hardware setups.

Methodology

The proposed approach is a multi-stage pipeline designed to classify PCB images as "pass" or "fail" based on the presence of tilted pins. The method combines semantic segmentation, geometric feature extraction, and statistical classification.

1. Semantic Segmentation

The core of the pipeline utilizes semantic segmentation to obtain pixel-accurate representations of individual pins.

• Architecture: A U-Net architecture with a pre-trained ResNet34 backbone was selected as the primary model, balancing segmentation performance and computational complexity. Other architectures (DeepLabV3, FCN, LRASPP) were trained for comparison.
• Training Data: The models were trained on industrial grayscale images (4096×3000 pixels) containing approximately 350 pins per board. To manage memory and annotation complexity, images were split into patches. Manual polygon masks were used to annotate pin contours.
• Optimization: Models were optimized with respect to recall to prioritize the detection of defective pins.

2. Contour Analysis and Feature Extraction

Once segmentation masks are generated, the pipeline extracts geometric features from the predicted contours:

• Process: Thresholding is applied to predicted masks to find contours.
• Features: The following board-level features are computed:
  • Average contour area.
  • Standard deviation of the contour area.
  • Maximum contour area.
  • Mean contour aspect ratio.
  • Total number of found contours.
• Rationale: Misaligned pins typically exhibit a larger visible surface area and different geometric properties compared to correctly inserted pins.

3. Board-Level Classification

The extracted features are used to train a logistic regression classifier to make a binary decision (pass/fail) for the entire board.

• Decision Logic: The classifier combines information from multiple features, as individual feature thresholds were insufficient for reliable separation.
• Threshold Selection: A decision threshold is selected based on specific priorities: maximizing pass precision (to prevent defective boards from passing), maximizing fail recall (to ensure no defective board is overlooked), and balancing precision/recall trade-offs.

4. Baseline Comparisons

To contextualize performance, the authors compared their method against two alternative approaches:

• PatchCore (Anomaly Detection): A memory-bank-based method trained exclusively on defect-free samples. It operates at the "pin nest" level (crops of pin groups) rather than the full board level.
• YOLO Instance Segmentation: A supervised object detection and segmentation approach (using YOLOv8) trained to detect individual pins and classify them as "tilted" or "correct." This operates at the "pin" level.

Datasets

The study evaluated the methods on two distinct datasets:

1. Industrial Dataset: 827 grayscale images from a real-world production environment (4096×3000 resolution). It contains 759 pass images and a smaller number of fail images (manually modified to introduce defects). It features three pin geometries and varied lighting conditions.
2. Public Dataset (Roboflow): A publicly available dataset containing 198 images of pin nests with instance segmentation annotations. This dataset has different visual characteristics and annotation styles (bounding boxes vs. polygons) compared to the industrial data.

Key Results

Segmentation Performance

• U-Net achieved the best overall segmentation performance on the industrial dataset, with an Intersection over Union (IoU) of 0.650, Accuracy of 0.995, and F1-Score of 0.788.
• DeepLabV3 showed superior recall (0.817) but lower precision compared to U-Net.
• On the Roboflow dataset, U-Net achieved an IoU of 0.728 and an F1-Score of 0.843.

Classification Performance

• Industrial Dataset: The proposed pipeline achieved a ROC-AUC of 0.990.
  • Pass Class: 100% Precision, 95.3% Recall.
  • Fail Class: 65.4% Precision, 100% Recall.
• Roboflow Dataset: The pipeline achieved a ROC-AUC of 1.000.
  • Pass Class: 100% Precision, 97.0% Recall.
  • Fail Class: 66.7% Precision, 100% Recall.
• Runtime: Processing a full 4096×3000 board image took approximately 18.0 seconds on an NVIDIA A100 GPU, with semantic segmentation accounting for the majority of the time.

Baseline Comparisons

• PatchCore: Achieved comparable results (e.g., 0.897 Recall for fail on industrial data) but operates at the pin-nest level and requires fewer annotations (only defect-free samples).
• YOLO Instance Segmentation: Showed significant variability. It performed well on the Roboflow dataset (mAP50 of 0.995 for pins) but struggled on the industrial dataset (mAP50 of 0.584 for all classes), likely due to class imbalance and annotation differences.

Significance and Claims

The paper claims that the proposed approach is a promising candidate for automated pin inspection in industrial environments. Key claims include:

1. Robustness to Domain Shift: The method achieved strong performance (ROC-AUC > 0.99) on two datasets with substantially different visual characteristics and annotation styles after dataset-specific training, suggesting robustness to domain changes.
2. Board-Level Decision Making: Unlike methods that focus on individual pin detection, this approach aggregates geometric features to make a holistic board-level pass/fail decision, which is directly applicable to production line quality control.
3. Feature-Driven Interpretability: By using logistic regression on geometric features (like contour area), the method provides insight into why a board is classified as defective (e.g., larger surface areas of tilted pins), unlike "black box" anomaly detection methods.
4. Reproducibility: The authors provide a self-contained reproduction package to allow others to build upon the approach.

The authors maintain modesty regarding the results, noting that the public test set contained a very limited number of defective samples, which limits the statistical significance of the fail-class performance metrics. They also acknowledge that the datasets used were relatively homogeneous, and further work is needed to validate the method on highly heterogeneous datasets or to analyze performance as a function of the specific number of defective pins per board.