🔬 materials science

Overcoming Labelled Data Scarcity for Defect Classification in Scanning Tunneling Microscopy

This paper proposes an automated, flexible approach for segmenting Scanning Tunneling Microscopy (STM) images that combines few-shot and unsupervised learning to overcome labelled data scarcity, enabling high-accuracy defect classification on diverse surfaces with minimal manual annotation.

Original authors: Nikola L. Kolev, Max Trouton, Filippo Federici Canova, Geoff Thornton, David Z. Gao, Neil J. Curson, Taylor J. Z. Stock

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikola L. Kolev, Max Trouton, Filippo Federici Canova, Geoff Thornton, David Z. Gao, Neil J. Curson, Taylor J. Z. Stock

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at a photograph of a vast, perfectly tiled floor made of tiny, shiny tiles. This is what a Scanning Tunneling Microscope (STM) does: it takes a picture of a surface so magnified that you can see individual atoms, which look like little bumps or dimples.

Scientists use these images to study how atoms stick together or how new molecules land on a surface. However, there's a problem: finding and labeling specific "defects" (like a missing tile or a weirdly shaped bump) in these images is like trying to find a needle in a haystack. Doing it by hand is slow, boring, and requires a human expert to stare at thousands of images.

This paper presents a new automated "smart assistant" that can do this labeling job for you, but with a superpower: it doesn't need to be taught with thousands of examples. It can learn from just a handful.

Here is how the system works, broken down into simple steps:

1. The Problem: The "Needle in a Haystack"

Traditionally, to teach a computer to spot these atomic defects, scientists had to manually label thousands of images first. It's like trying to teach a dog to fetch a specific ball by showing it that ball 3,000 times. If the scientist wants to study a new type of ball (a new material), they have to start the training from scratch with thousands of new examples. This is too slow for real-world science.

2. The Solution: The "Few-Shot" Detective

The authors built a system that uses Few-Shot Learning (FSL). Think of this like showing a detective a photo of a specific suspect (the "defect") and asking them to find that same person in a crowd. The detective doesn't need to have seen that suspect 3,000 times; they just need to see them once or twice to recognize the key features.

The system has three main parts, working together like a factory assembly line:

Step A: The "Rough Sketch" Artist (Unsupervised Learning)

First, the system needs to know where the defects are roughly located. It uses a technique called Unsupervised Learning.

The Analogy: Imagine you have a black-and-white photo of a messy room. You don't tell the computer what a "chair" or a "toy" is. Instead, you ask it to group similar-looking pixels together. It might say, "These dark spots look like they belong together, and these bright spots look like something else."
What it does: It creates a rough "map" (a binary mask) that separates the "interesting stuff" (defects) from the "boring background" (the perfect atomic grid). This step requires very little human input.

Step B: The "Cut-and-Paste" Machine (Cropping)

Once the rough map is made, the system cuts out small square pieces of the image that contain the defects.

The Analogy: It's like taking a magnifying glass, finding a bump on the floor, and cutting out just that square inch of the photo to look at it more closely.

Step C: The "Expert Classifier" (Few-Shot Learning)

This is the magic part. The human scientist looks at a few of these cut-out squares and says, "This one is a 'Double Dangling Bond' (Type A), and this one is a 'Siloxane' (Type B)."

The Analogy: You show the computer one or three examples of each type of defect. The computer then uses a mathematical "memory" to compare every other cut-out square against your examples. It asks, "Does this new square look more like your 'Type A' example or your 'Type B' example?"
The Result: It sorts the rest of the image automatically.

3. What Did They Test?

The team tested this "smart assistant" on three different types of surfaces:

Silicon (Si): A very common material.
Germanium (Ge): Similar to silicon, but slightly different.
Titanium Dioxide (TiO2): A completely different material that looks and acts differently.

They wanted to see if the system could learn on Silicon and then instantly recognize defects on Germanium and Titanium Dioxide without needing a massive new training course.

4. The Results

On Silicon: The system was incredibly accurate, getting it right 99% of the time when shown just three examples of each defect type.
On Germanium and Titanium Dioxide: The accuracy dropped (to about 62% and 85% respectively), but it was still much better than just guessing or using older, simpler methods.
The "One-Shot" Test: Even when the system was shown only one example of a defect, it could still find similar ones better than a basic computer program that just looks at raw pixels.

5. Why This Matters (According to the Paper)

Flexibility: If a scientist discovers a weird new molecule on a surface, they don't need to spend weeks labeling data. They can just show the computer a few examples, and it adapts immediately.
Handling "Anomalies": Sometimes, something weird happens in the image that doesn't fit any category (like a giant scratch). The system allows the human to say, "Ignore that one," so it doesn't mess up the statistics. Older systems would force that weird thing into a wrong category.
Speed: It turns a task that takes hours of human staring into a process that happens in seconds.

Summary

The paper describes a new tool that helps scientists analyze atomic images. Instead of needing a massive library of labeled examples to teach the computer, this tool learns from just a few examples (like a human would). It combines a "rough sketch" maker with a "smart classifier" to automatically find and name atomic defects on different materials, making the process of discovering new materials much faster and less labor-intensive.

Technical Summary: Overcoming Labelled Data Scarcity for Defect Classification in Scanning Tunneling Microscopy

Problem Statement
Scanning Tunneling Microscopy (STM) provides atomic-resolution imaging essential for understanding surface physics and chemistry. However, the quantitative analysis of STM data typically requires the manual identification and labelling of atomic-scale features against a uniform background, a process that is labor-intensive and time-consuming. While supervised machine learning (ML) approaches, such as Convolutional Neural Networks (CNNs), have shown success in automating this task, they rely heavily on large, manually annotated datasets specific to each material system. This reliance creates a bottleneck: as STM is frequently applied to novel materials and experiments, the requirement for extensive labelled data for every new surface limits the practical utility of ML. Existing alternatives, such as synthetic data generation via Density Functional Theory (DFT), are computationally expensive and often require expert intervention, while other unsupervised methods may lack rigorous performance metrics or generalizability.

Methodology
The authors propose a hybrid, automated workflow that combines unsupervised learning and Few-Shot Learning (FSL) to segment and classify STM defects with minimal labelled data. The framework operates through a five-step pipeline:

Data Acquisition and Preprocessing: STM images (filled and empty states) are acquired, plane-leveled, scan-line aligned, and corrected for hysteresis.
Unsupervised Coarse Segmentation (Module 1 & 2): To avoid manual labelling for the initial segmentation step, the authors utilize a pre-trained FCNResNet101 (trained on ImageNet) to extract feature vectors from pixels. These vectors undergo k-means clustering to generate a binary map distinguishing "defects" from the "lattice." This unsupervised output is used to train a lightweight U-Net (Module 2) specific to the substrate. The U-Net can then perform coarse segmentation on new scans in seconds on a CPU, replacing the slower, GPU-dependent unsupervised step.
Feature Cropping: The U-Net identifies defect locations, which are then isolated into uniform image crops and resampled to match network dimensions.
User-Defined Support Set Generation (Module 3): The user defines $N$ feature classes and labels a small number ( $K$ ) of examples for each class from the cropped images. This forms the support set. The user can also explicitly exclude anomalous features at this stage.
FSL Classification: The remaining unseen crops (the query set) are classified by an FSL network using the support set. The network embeds images into a latent vector space and classifies query samples based on their proximity to support vectors using specific distance metrics.

The study evaluates four distinct FSL algorithms: Prototypical Networks, Matching Networks, Relation Networks, and Simple Shot. These are tested against a baseline k-Nearest Neighbour (kNN) classifier on raw pixels. The experiments utilize three distinct surface systems to test generalizability:

Si(001): Used for initial training and testing on known substrates with new adsorbates (AsH3).
Ge(001): Tested as a "new" substrate (trained only on Si data).
TiO2(110): Tested as a structurally different substrate (trained only on Si and Ge data).

Key Results
The authors report the following findings across their test cases:

Segmentation Performance: The U-Net, trained on data labelled by the unsupervised pipeline, achieved validation accuracies above 90% for all three surfaces. The Dice Score (DS) for defect-only segmentation was highest for Germanium (0.87) and lowest for Silicon (0.64), though the total DS (including background) remained high (0.96–0.99).
FSL Classification Accuracy:
- Prototypical Networks consistently outperformed other neural-network-based methods. On the Silicon dataset (Si(001):H:AsH3), the Prototypical network achieved 93.6% accuracy in the 1-shot setting and 98.9% in the 3-shot setting.
- Simple Shot networks also performed well, particularly when trained on substrate-specific data, often outperforming larger ResNet18-based approaches despite having fewer parameters.
- Relation Networks performed significantly worse (approx. 54–58% accuracy), likely due to overfitting given the limited training data and the complexity of their learned distance metric.
- Generalization: When applied to unseen substrates (Ge and TiO2) without prior training on those specific materials, accuracy dropped (e.g., 54.5% 1-shot for Ge), but the FSL methods still generally outperformed or matched the kNN baseline in the 1-shot regime.
Ablation Study: The authors demonstrated that artificially increasing the number of training classes (by inverting pixel values to create "negative" defect types) improved classification accuracy. For instance, the Matching Network on Germanium data improved by 12% when the training set was augmented with inverse classes.

Significance and Claims
The paper positions this work as a significant step toward material-agnostic, automatic segmentation of STM images. The primary contribution is the demonstration that high-accuracy defect classification is possible with as few as one or three labelled data points per class, effectively removing the barrier of large annotated datasets.

The authors claim their approach offers greater flexibility than previous supervised methods:

Adaptability: The system can be adapted to unseen surfaces rapidly after initial training.
Anomaly Handling: Unlike rigid supervised classifiers, this workflow allows users to explicitly exclude anomalous features from the support set, preventing them from skewing statistics—a capability lacking in some prior supervised works.
Efficiency: By distilling knowledge from large pre-trained models into a smaller U-Net for coarse segmentation, the method enables rapid, CPU-based processing suitable for laboratory workflows.

The authors remain modest regarding the absolute accuracy on completely novel materials (e.g., 62% on Germanium in the 1-shot case), noting that while this may not be sufficient for final publication-quality analysis, it provides rapid, actionable insights during data acquisition. They conclude that while standard architectures were used to establish a baseline, future work involving hyperparameter optimization and architectural refinement could further enhance performance.