STEM-Bank-26: An Open Dataset of Analytical Scanning Transmission Electron Microscopy Images and Spectra

In the invisible world of materials science, researchers need to see things that are too small for the naked eye, too small for standard microscopes, and even too small for the human mind to easily grasp. To study these tiny structures, scientists use a powerful tool called scanning transmission electron microscopy. Imagine a beam of electrons, which are subatomic particles, being fired at a sample so thin that the electrons can pass through it. As the beam scans across the material, it interacts with the atoms, creating a detailed picture of the sample's internal structure. But seeing the shape is only half the story. To truly understand a material, scientists also need to know what it is made of and how its atoms are behaving. This is where spectroscopy comes in. By analyzing the energy lost by electrons or the X-rays they emit when hitting the sample, researchers can identify the specific elements present and even determine their chemical state. This combination of high-resolution imaging and chemical analysis is essential for developing better batteries, more efficient catalysts, and advanced electronic devices. However, teaching computers to perform these complex tasks has been difficult because the data required to train them has been scattered, incomplete, or locked away in private archives.

A team of researchers at the University of Pennsylvania has now changed the landscape by releasing a massive, open collection of this exact kind of data. They have assembled a dataset called STEM-Bank-26, which contains nearly 38,000 images and thousands of spectroscopic measurements gathered over five years. This collection is not just a random assortment of pictures; it is a structured record of how scientists actually work. The dataset includes 37,921 images taken from 1,095 different material samples across 1,499 separate experimental sessions. For more than two-thirds of these sessions, the researchers also captured spectroscopic data, providing a complete picture of both the physical structure and the chemical composition of the materials. The total size of this collection is 929 gigabytes, a volume of information that makes it the largest openly available dataset of its kind to integrate imaging with both energy-dispersive spectroscopy and electron-energy loss spectroscopy.

The true value of this work lies not just in the sheer number of files, but in how the researchers organized them to reveal the human decision-making process behind the science. Electron microscopy is not a fully automated process; it requires a skilled operator to look at a sample, decide which area is interesting, adjust the microscope settings, and then choose to gather chemical data on that specific spot. This series of decisions forms a "trajectory," a path of actions that leads from a general view to a detailed analysis. Because these decisions are rarely recorded in a single file, the team developed a method to reconstruct these paths by comparing the metadata—the digital tags attached to each file that record time, position, and settings. By analyzing the gaps between images, they could infer when an operator moved the sample to a new location or paused to think, effectively turning a static archive of files into a dynamic record of scientific exploration.

This reconstruction allows the dataset to serve as a training ground for artificial intelligence. In the past, efforts to build "AI scientists" capable of operating these microscopes were stalled by a lack of data showing how humans explore and adapt to new information. Existing datasets often contained only images or only spectra, missing the crucial link between the two. This new collection bridges that gap, offering a realistic view of how a scientist moves from a broad survey of a sample to a focused analysis of a specific feature. The data shows that most experimental sessions involve between 10 and 100 images, with a significant portion including chemical mapping. The samples themselves are diverse, ranging from heterogeneous catalysts used in chemical reactions to lithium-ion battery components, providing a broad foundation for machine learning models to learn from.

The researchers also addressed the logistical challenges that usually prevent such large datasets from being shared. Microscope time is expensive, and the files generated are often too large to publish in full, leading scientists to share only a few representative images. This new work demonstrates that it is possible to share the entire archive without overwhelming the scientific community. The team provided tools and scripts that allow other researchers to easily convert their own archival data into similar trajectory datasets, encouraging a shift in how the microscopy community shares its work. By making this data available under an open license, the authors hope to accelerate the development of autonomous materials research, where intelligent systems can learn from human expertise to explore new materials faster and more efficiently.

While the dataset is a significant step forward, the authors are clear about its limitations. The reconstructed trajectories are snapshots of the microscope's state rather than a continuous video of the operator's actions. They do not capture the moments when a scientist moves the stage to inspect an area but decides not to take a picture, meaning the data might slightly favor areas that were deemed worthy of high-resolution capture. Additionally, some technical details about the microscope's lens settings are missing from the original files, which limits the dataset's use for tasks involving the precise tuning of the instrument itself. Despite these constraints, the collection stands as a robust resource for understanding the complex interplay between imaging and spectroscopy. It offers a rare glimpse into the daily workflow of materials science, providing the raw material needed to teach machines how to see, analyze, and understand the nanoscale world with the same curiosity and judgment as a human expert.