💻 computer science

ScratNet: A Swin-Based Multi-Scale Dilated Network with Precision Refinement for Semiconductor Scratch Segmentation

The paper proposes ScratNet, a novel end-to-end framework combining a modified Swin Transformer backbone with a specialized decoder featuring multi-scale aggregation and precision refinement modules to achieve superior accuracy in segmenting irregular, low-contrast semiconductor scratch defects.

Original authors: Sachin Ranjan, Hoon Kim

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sachin Ranjan, Hoon Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to find the tiniest, most stubborn scratches on a shiny, high-tech silicon chip. These aren't just any scratches; they are like invisible spiderwebs or hair-thin lines that can ruin the whole chip if missed. The problem? The background is noisy, the lighting is tricky, and the scratches change shape and size constantly.

For a long time, detectives (or rather, engineers) tried to find these scratches using old-school rules and manual looking. But the paper explains that this is like trying to find a needle in a haystack while wearing thick gloves—it's slow, tiring, and you often miss the needle or mistake a piece of dust for one.

Then came "Deep Learning," a type of computer brain that gets really good at spotting patterns. But even these smart computers had a blind spot: they were great at seeing big blobs of defects but terrible at seeing the super-fine, jagged edges of a thin scratch. It's like having a camera that can see a whole mountain clearly but blurs out the tiny pebbles at the base.

The Big Idea: ScratNet
The authors of this paper built a new tool called ScratNet to fix this. Think of ScratNet as a super-powered magnifying glass combined with a detective's intuition. It's designed specifically to hunt down those tricky, thin scratches on semiconductor wafers.

Here is how ScratNet works, broken down into three cool tricks:

The "Big Picture" and "Small Detail" Team:
The paper uses a special "brain" called a Swin Transformer as its foundation. Imagine this as a team of scouts. Some scouts climb high up a tree to see the whole forest (global context), while others stay on the ground to look at individual leaves (local details). ScratNet mixes these views together so it doesn't miss a scratch just because it's small or just because it's in a weird spot.
The "Stem" Connection:
Sometimes, when you zoom out to see the big picture, you lose the fine texture. ScratNet has a special module called the Stem Integration Module (SIM). Think of this like a "memory lane" that brings the super-sharp, early details back into the mix. It ensures that when the computer decides "Yes, that's a scratch," it remembers exactly what the edge looked like, not just a blurry guess.
The "Edge Sharpening" Tool:
This is the paper's secret weapon. The authors added a Precision Refinement (PR) branch. Imagine you have a sketch of a scratch, but the lines are a bit fuzzy. This module uses special "anisotropic" filters (which just means they stretch in specific directions, like a ruler) to sharpen the edges. It's like taking a blurry photo and using a filter to make the lines crisp and clear, specifically targeting those thin, irregular lines that other methods miss.

What They Ruled Out
The paper is very clear about what doesn't work well enough on its own.

Old Rules: They explicitly say that traditional methods (like manual checking or simple image processing) are too slow and inconsistent.
Standard AI: They argue that standard deep learning models (like regular CNNs) often fail to capture those fine-grained edge details.
The "Magic Bullet" of General AI: The authors tested a famous, general-purpose AI model called SAM (Segment Anything Model). They tried using it in two ways: letting it guess without training (zero-shot) and teaching it with bounding boxes. The results? SAM struggled. In the "zero-shot" test, it got an IoU (a score for how well the shapes match) of only 42.81% on the IC dataset. Even when they fine-tuned it with boxes, it only reached 78.33%. The paper concludes that while SAM is amazing for general photos, it isn't the right tool for these specific, tiny industrial scratches. You need a specialized tool like ScratNet.

How Sure Are They?
The authors didn't just guess; they ran extensive experiments on two real datasets:

IC Images: 749 images of integrated circuits.
Wafer Images: 8,200 images of silicon wafers.

They measured everything using strict math. On the IC dataset, their best model (Swin-Base backbone with ScratNet) achieved an IoU of 87.32% (with data augmentation) and a Dice score of 93.48%. Without augmentation, the IoU was 85.82%. On the Wafer dataset, it hit an IoU of 81.17% and a Dice of 87.96%.

Crucially, they looked at the edges. They measured Boundary IoU (how well the outline matches) and Hausdorff Distance (how far off the edges are). ScratNet reduced the edge error (Hausdorff Distance) down to as low as 23.35 pixels on the IC dataset (specifically with the Swin-Base backbone and data augmentation), which is a massive improvement over other methods that were off by 40+ pixels.

The Verdict
The paper proves that by combining a powerful "scout" system (Swin Transformer) with a "detail-restorer" (Stem Integration) and an "edge-sharpener" (Precision Refinement), you can find scratches that other methods miss.

They measured that ScratNet consistently beats the competition. For example, on the IC dataset, a standard model called UNet got an IoU of 79.48%, while ScratNet got 87.32% (with augmentation). That's a significant jump in accuracy.

The authors are confident in these numbers because they tested them against many other famous models (like ResNet, HRNet, and various decoder types) and found ScratNet to be the winner every time. They suggest that this approach offers a "scalable and robust solution" for the future of chip manufacturing, but they stop short of saying it's a perfect, solved problem for every single scenario in the world. They simply show that, for the data they tested, ScratNet is currently the best tool in the box.

Technical Summary: ScratNet for Semiconductor Scratch Segmentation

1. Problem Statement

Surface scratch defects in semiconductor manufacturing present a critical challenge for quality control due to their irregular shapes, low contrast, and varying scales. Traditional inspection methods, including manual visual inspection and rule-based image processing, are labor-intensive, slow, and prone to inconsistency, particularly as device geometries shrink and integration densities increase. While deep learning approaches based on Convolutional Neural Networks (CNNs) have improved detection accuracy, they often struggle to capture fine-grained edge details necessary for precise pixel-level segmentation of thin and irregular defects. Furthermore, industrial scratch images frequently suffer from significant noise, variable illumination, and severe class imbalance, complicating reliable detection.

2. Methodology

The authors propose ScratNet, an end-to-end trainable scratch segmentation framework designed to address the limitations of existing methods. The architecture integrates a modified Swin Transformer backbone with a custom decoder comprising three specialized modules:

A. Modified Swin Transformer Backbone

To improve the extraction of low-level cues essential for detecting fine scratches, the standard patch partitioning layer of the Swin Transformer is replaced with a lightweight Stem module. This stem consists of two successive $3 \times 3$ convolutional layers with stride-2 pooling. This design reduces resolution while expanding channels, preserving texture and edge information that might otherwise be lost in conventional patch tokenization. The backbone then proceeds through four stages of hierarchical window-based self-attention to extract multi-scale spatial and contextual features.

B. MSP Decoder

The decoder, referred to as MSP, fuses semantic context with high-resolution spatial information through three components:

Multi-Scale Dilated Aggregation (MDA): This module aggregates feature maps from all four stages of the Swin backbone in a top-down manner. It employs a Dilated Block (DiBlock) consisting of two sequential $3 \times 3$ convolutions with specific dilation rates (e.g., $(8,1), (6,1), (4,1), (2,1)$ for stages 1–4). This structure captures both global contextual information (via dilated convolutions) and local spatial details (via non-dilated convolutions), creating a context-rich representation.
Stem Integration Module (SIM): To restore spatial details lost during downsampling, the SIM integrates high-resolution features from the stem network with the aggregated features from the MDA. It applies $3 \times 3$ convolutions to progressively fuse stem cues ( $S_1, S_2$ ) with the decoder output, reinforcing structural details for fine-grained segmentation.
Precision Refinement (PR) Module: This module specifically targets boundary sharpness, which is crucial for thin and elongated defects. It applies four parallel anisotropic convolutions ( $1 \times 3, 3 \times 1, 1 \times 5, 5 \times 1$ ) to the aggregated feature map. These filters capture horizontal, vertical, and elongated patterns, which are then concatenated and projected to generate the final pixel-wise segmentation map.

C. Loss Function

To address the class imbalance between defect (foreground) and background pixels, the model is trained using a composite loss function combining Binary Cross-Entropy (BCE) and Dice loss. This formulation maximizes the overlap between predictions and ground truth while ensuring robust gradient flow for the minority class.

3. Key Contributions

The paper outlines four primary contributions:

ScratNet Framework: A novel, multi-level, end-to-end trainable framework integrating a modified Swin-Base Transformer with multi-scale dilated aggregation and precision refinement, specifically tailored for thin and irregular scratch detection.
Multi-Scale Dilated Aggregation (MDA): A module that integrates multi-scale features from all four Swin stages using both dilated and non-dilated convolutions to capture global and local context simultaneously.
Stem Integration and Precision Refinement: The introduction of a Stem Integration Module (SIM) to restore low-level spatial details and a Precision Refinement (PR) branch using anisotropic convolutions to sharpen boundaries, significantly improving segmentation on challenging scratch patterns.
State-of-the-Art Performance: Demonstration of superior segmentation quality and robustness on semiconductor scratch datasets compared to existing methods.

4. Experimental Results

The authors evaluated ScratNet on two datasets: an IC image dataset (749 images, $180 \times 180$ resolution) and a Wafer image dataset (8,200 images, $224 \times 224$ resolution).

Quantitative Performance: ScratNet consistently outperformed baseline models (UNet, ResNet50, HRNet, and Swin variants combined with FCN, FPN, and UPerNet decoders).
- On the IC dataset with a Swin-Base backbone and data augmentation, ScratNet achieved an IoU of 87.32% and a Dice score of 93.48%, surpassing the next best Swin-Base configuration (UPerNet) by a significant margin.
- On the Wafer dataset, ScratNet with Swin-Base achieved an IoU of 81.17% and Dice of 87.96% (with augmentation), outperforming all other decoder combinations.
Boundary Metrics: The model demonstrated superior boundary alignment, evidenced by higher Boundary IoU scores and lower Hausdorff Distance values compared to CNN-based and standard Transformer-based baselines.
Ablation Studies: Experiments confirmed the incremental value of each component. The MDA module provided the largest initial gain, while the PR module significantly improved boundary precision, and the SIM module further enhanced performance, particularly with transformer backbones.
Efficiency: ScratNet maintained a favorable balance between accuracy and computational cost (latency and FLOPs), outperforming UPerNet in accuracy while remaining competitive in efficiency.
Comparison with SAM: When compared to the Segment Anything Model (SAM) in zero-shot and fine-tuned settings, ScratNet significantly outperformed SAM, highlighting the necessity of domain-specific architectures for micro-defect segmentation.

5. Significance and Claims

The paper claims that ScratNet provides a scalable and robust solution for automated scratch inspection in high-precision manufacturing. By effectively addressing the challenges of irregular shapes, low contrast, and varying scales through stage-adaptive feature aggregation and boundary-aware refinement, the proposed method achieves superior accuracy on thin and irregular defects. The authors position ScratNet as a significant advancement over traditional manual methods and existing deep learning approaches, offering improved product yields and reduced inspection costs. The work concludes that while general-purpose models like SAM show promise in natural images, specialized architectures like ScratNet remain essential for the highly specific demands of semiconductor defect segmentation. Future work is suggested to explore advanced filtering techniques and lightweight decoder architectures for real-world deployment.