Unsupervised segmentation and clustering workflow for efficient processing of 4D-STEM and 5D-STEM data

This paper introduces a scalable, unsupervised clustering framework that efficiently segments and compresses 4D-STEM and 5D-STEM data by identifying crystallographically distinct domains through local diffraction-pattern similarity, thereby enabling rapid and accurate structural mapping as demonstrated on in situ gold nanoparticle growth.

The Gist

The Big Picture: Finding Order in a Chaotic Crowd

Imagine you are standing in a massive, crowded stadium. Everyone is holding a flashlight and waving it in a specific pattern. Some people are waving in circles, others in squares, and some are just shaking them randomly.

Now, imagine you have a camera that takes a picture of every single person in the stadium, one by one, capturing their unique light pattern. This is what scientists do with 4D-STEM (a super-advanced electron microscope). They scan a tiny sample (like gold nanoparticles) and record a "diffraction pattern" (the light pattern) for every single tiny spot they look at.

The Problem:
If you have a 500x500 grid of spots, that's 250,000 individual pictures.

1. Too much data: It's like trying to read 250,000 pages of a book at once. It's overwhelming for computers and humans.
2. Too much noise: In a liquid environment (like a drop of water under the microscope), the images are grainy and fuzzy, like trying to see through a dirty window.
3. Hard to group: It's hard to tell which people are waving the same pattern just by looking at them individually.

The Solution: The "Marching Squares" Team

The authors of this paper created a smart, automatic system to organize this chaos. Think of it as a team of detectives that walks through the stadium looking for groups of people doing the exact same thing.

Here is how their method works, step-by-step:

1. Cleaning the Glasses (Preprocessing)

Before the detectives start, they put on special glasses.

• The Analogy: Imagine the stadium is foggy. The scientists use a "blur" filter to smooth out the static noise, making the patterns clearer. They also ignore the bright center of the flashlight (which is too bright to be useful) and focus on the interesting edges of the light.
• The Result: The images are now cleaner, and the "real" patterns stand out more.

2. The "Similarity" Game (Clustering)

This is the core of their invention. The detectives start at one person (a "seed").

• The Analogy: The detective asks, "Does the person next to me wave their light the same way I do?"
  • If Yes (high similarity): They join the same group.
  • If No (low similarity): They stay separate.
• The "Marching" Part: The group keeps growing. The new members check their own neighbors. If a neighbor matches, they join the party. This continues until the group hits a wall where the patterns change.
• The Magic: Instead of looking at 250,000 individual people, the computer now sees 10 to 100 distinct groups (clusters). Each group represents a specific type of crystal structure.

3. The "Group Hug" (Averaging)

Once the groups are formed, the scientists take all the pictures from one group and blend them together into one single, perfect picture.

• The Analogy: Imagine 100 people taking a photo of the same sunset, but each photo is a little shaky or blurry. If you stack all 100 photos on top of each other, the shaky parts cancel out, and the final image is crystal clear.
• The Result: The "signal-to-noise" ratio skyrockets. Weak details that were invisible before are now bright and clear.

Why This Matters (The Payoff)

1. Speeding Up the Movie
If you want to know the orientation (direction) of every crystal in the sample, you usually have to analyze every single one of the 250,000 spots. That takes forever.

• With this method: You only have to analyze the 50 groups. It's like reading the summary of 50 chapters instead of the whole book. The computer does the work 1,000 times faster.

2. Seeing the Invisible
Because they blended the images (the "Group Hug"), they can now see the tiny, weak details of the gold nanoparticles growing in the liquid. This helps them understand exactly how the crystals are forming and where the stress (strain) is happening.

3. Handling the "Liquid" Mess
The paper tested this on gold nanoparticles growing in a liquid cell. Liquids are messy and create a lot of "static" (noise). Traditional methods often fail here, but this "detective team" was able to ignore the noise and find the real crystal structures anyway.

The Bottom Line

The authors built a smart, automatic sorting machine for electron microscope data.

• Old way: Look at every single pixel individually, get overwhelmed by noise, and wait hours for results.
• New way: Group similar pixels together, average them to make them clear, and analyze just the groups.

This allows scientists to process massive amounts of data quickly, see details they couldn't see before, and understand how tiny materials behave in real-time, even in messy environments like liquids. It's like turning a chaotic crowd of 250,000 people into a few organized teams, making it easy to understand the whole stadium at a glance.

Technical Summary

1. Problem Statement

Four-dimensional scanning transmission electron microscopy (4D-STEM) and its time-resolved extension (5D-STEM) generate massive datasets by recording a full diffraction pattern at every probe position in a scan. While these techniques offer nanometer-scale resolution of local structure, orientation, and strain, they face significant challenges:

• Data Volume: The sheer size of 4D/5D datasets (e.g., $512 \times 512$ probe positions $\times$ $192 \times 192$ detector pixels) creates bottlenecks in storage, visualization, and computational processing.
• Signal-to-Noise Ratio (SNR): Modern low-dose and in situ experiments (e.g., liquid-cell TEM) often yield diffraction patterns with low SNR, making feature extraction difficult.
• Limitations of Traditional Analysis: Conventional methods often rely on manual region-of-interest selection or global thresholding, which fail to capture subtle, physically meaningful variations in closely spaced nanocrystals or partially coherent domains. Existing unsupervised clustering methods (e.g., K-means, DBSCAN) are often sensitive to user-defined parameters, struggle with irregularly shaped regions, and are not always integrated into unified 4D/5D-STEM workflows.
• Computational Cost: Quantitative analyses like Automated Crystal Orientation Mapping (ACOM) and strain mapping require processing every single probe position, which is computationally expensive and scales poorly with dataset size.

2. Methodology

The authors propose an unsupervised clustering framework based on the marching squares algorithm to segment 4D-STEM data into spatially coherent domains. The workflow consists of four main stages:

A. Preprocessing (Correlative Pixel-Based Filtering)

To enhance SNR before clustering, the authors apply a specific filtering procedure:

• Gaussian Blurring: Diffraction patterns are blurred in reciprocal space to reduce high-frequency noise.
• Radial Weighting: A radial weighting function suppresses the intense central beam and detector edge artifacts while emphasizing intermediate-angle scattering features relevant for structural discrimination.
• Correlative Averaging: Each diffraction pattern is compared to its neighbors within a defined offset range (circular footprint, radius $r=4$). A correlation coefficient is calculated, and patterns are weighted and accumulated based on similarity. This preserves spatial coherence while suppressing uncorrelated background noise.

B. Similarity Matrix Calculation

• A normalized cosine correlation is computed between each diffraction pattern and its eight nearest neighbors in real space.
• This generates a 3D similarity array $S(x, y, n)$, which is then averaged to create a scalar similarity map $\bar{S}(x, y)$.
• A real-space mask is applied to exclude background regions (low signal) from the analysis.

C. Marching-Square Clustering

• Seed Selection: The algorithm iteratively selects the unassigned pixel with the highest similarity value as a seed for a new cluster.
• Region Growth: Starting from the seed, the algorithm inspects neighbors. If a neighbor's similarity to the current region exceeds a user-defined threshold ($T$), it is added to the cluster.
• Recursive Expansion: This process continues recursively until no further neighbors satisfy the threshold, forming a closed contour delineating a crystallographically distinct domain.
• Refinement: Small clusters (noise-induced fragments) below a minimum size are excluded.

D. Data Reduction and Averaging

• For each identified cluster, all diffraction patterns within that spatial region are averaged to produce a single cluster-averaged diffraction pattern.
• This reduces the data dimensionality from $D(x, y, q_x, q_y)$ to $D(N_{cluster}, q_x, q_y)$, where $N_{cluster}$ is typically $10^{-2}$ to $10^{-3}$ of the original number of probe positions.

3. Key Contributions

• Novel Algorithm: Introduction of a marching-square-based clustering method specifically tailored for 4D-STEM data, using local diffraction-pattern similarity as the metric.
• Integrated Workflow: The method is implemented as a module in the open-source Python package py4DSTEM, making it accessible and reproducible.
• Scalability: The approach is designed to handle both static 4D-STEM and time-resolved 5D-STEM (in situ) data by applying the workflow sequentially to individual frames.
• Parameter Efficiency: The framework requires only a few tunable parameters (similarity threshold, minimum cluster size), making it robust across diverse datasets without extensive user tuning.

4. Results

The method was validated using in situ liquid-cell 4D-STEM data of gold (Au) nanoparticle growth under electron irradiation.

• Signal Enhancement: Cluster-averaged diffraction patterns showed significantly improved SNR compared to raw patterns, revealing weak high-angle diffraction features that were previously obscured by noise.
• Computational Efficiency:
  • The number of diffraction patterns requiring analysis for orientation mapping was reduced by orders of magnitude (from $2.62 \times 10^5$ probe positions to $\sim 10^2$ clusters).
  • This reduction drastically lowered the computational cost of Bragg disk detection and template matching.
• Orientation Mapping Accuracy:
  • Cross-validation using a checkerboard method showed that the mean angular error for orientation determination dropped from 7.32° (raw data) to 5.19° (preprocessed only) and finally to 2.03° (preprocessed + clustered).
  • The clustering successfully delineated grain boundaries and identified substrate-induced orientation biases (e.g., [111] and [110] textures).
• Strain Mapping: The method enabled robust strain mapping by using the stable orientation from clustered data as a reference while extracting Bragg peak positions from non-clustered (raw) patterns to preserve spatial resolution. However, the authors note limitations in regions with overlapping grains where diffraction signals are mixed.

5. Significance

• Data Compression: The workflow provides a scalable route for data compression without losing structural information, essential for managing the massive data volumes generated by modern detectors and in situ experiments.
• Physical Interpretability: By grouping spatially contiguous regions with similar diffraction signatures, the method produces physically meaningful segments that correspond to nanoscale domains, facilitating the analysis of complex microstructures.
• Generalizability: While demonstrated on Au nanoparticles, the framework is applicable to diverse 4D-STEM modalities, including strain mapping, phase identification, and orientation analysis in polycrystalline or beam-sensitive systems.
• Accessibility: By being open-source and integrated into py4DSTEM, it lowers the barrier to entry for advanced 4D-STEM analysis, promoting reproducibility and broader adoption in the electron microscopy community.

In conclusion, this work presents a robust, computationally efficient, and physically grounded solution to the "big data" challenge in electron microscopy, enabling rapid and accurate structural characterization of nanomaterials.