Technical Summary: SimuScan-Agent

Problem Statement

Atomic force microscopy (AFM) is a versatile platform for nanoscale characterization, yet large-area investigations remain inefficient due to the decoupling of data acquisition from real-time analysis. Conventional workflows typically rely on exhaustive raster scanning, which leads to substantial oversampling of uninformative regions or premature termination before statistically representative data is collected. This is particularly problematic for heterogeneous or sparsely populated samples where scientifically relevant features occupy only a small fraction of the surface. Furthermore, realizing autonomous AFM requires coordinating multiple specialized models for perception, control, and decision-making, a process hindered by the difficulty of generating large, accurately labeled training datasets and the need for deep expertise in machine learning (ML) and programming to orchestrate these tools.

Methodology

The authors introduce SimuScan-Agent, a framework that uses a Large Language Model (LLM) as an orchestration layer to translate natural-language objectives into adaptive AFM workflows. The system operates through a modular architecture coordinated via the Model Context Protocol (MCP), delegating execution to specialized, independently verified modules rather than acting as a direct microscope controller.

1. Multimodal Morphological Sources and Synthetic Data Generation
To address the training-data bottleneck, the framework integrates three complementary routes for defining object morphologies, which are then processed through a physics-informed simulation pipeline (SimuScan 2.0):

• Procedural Generation: The LLM translates natural-language descriptions into parameterized Python functions (PyGenerator) to define object geometries.
• Generative Models: General-purpose image models provide visual references for complex morphologies (e.g., DNA structures, nanoparticle clusters), which serve as references rather than direct ground truth.
• Literature Retrieval: A Retrieval-Augmented Generation (RAG) component extracts morphological references from scientific literature.
  Crucially, these morphological inputs are fed into SimuScan, which simulates AFM image formation (including tip-sample convolution, noise, and scanner distortions) to produce precisely labeled synthetic datasets without manual annotation.

2. Machine Learning Models
The synthetic datasets are used to train two complementary models:

• Instance Segmentation: A YOLOv8 model trained for online object detection and segmentation.
• Morphological Representation: A Triplet Network trained to learn continuous latent representations of object shapes, enabling the identification of representative and atypical structures.

3. Adaptive Exploration and Orchestration
The LLM orchestrates the workflow by selecting tools and coordinating actions. During experimental acquisition, the system employs a multiclass probabilistic active exploration strategy. A Bayesian approach updates class-specific spatial occupancy models after each scan. A utility function selects subsequent measurement locations based on predicted class occupancy, uncertainty, remaining sampling deficits, and scanner movement costs. This allows the system to prioritize rare populations and reduce redundant scanning.

Key Results

The framework was validated through both simulations and a physical experiment on a nanostructured sample containing four geometric classes (ellipses, hexagons, squares, and triangles).

• Simulation Efficiency: In simulations comparing five acquisition strategies, the active search strategy reduced the number of measurements and sampled area by approximately 71% relative to standard raster scanning. It required a median of 38 images (13% of the grid) compared to 172.5 images (43% of the grid) for raster scanning.
• Experimental Performance: In a ~10 mm² experimental exploration, SimuScan-Agent sampled only 17.2% of available positions (579 images) to reconstruct class-specific spatial distributions.
  • The system successfully detected abundant classes (ellipses, hexagons, squares) and approached targets for rare classes, though the rarest population (triangles) remained below its target due to time limits.
  • Spatial reconstruction, quantified by the Overlapping Coefficient (OVL) against the nominal design, improved rapidly with coverage (e.g., square overlap increased from ~60% at 2% coverage to 81% at 17.2% coverage).
• Morphological Library: The system constructed an object-level library where learned embeddings successfully separated the four classes. Using a k-nearest neighbor (kNN) metric in the latent space, the framework identified both representative structures and atypical objects exhibiting fabrication defects or acquisition artifacts.

Significance and Claims

The paper claims that SimuScan-Agent bridges the gap between synthetic-data-driven model development and goal-directed experimental exploration. Its primary contributions are:

1. Accessibility: By using natural language to orchestrate complex workflows, the framework lowers barriers for researchers without specialized programming or ML expertise, allowing them to define experimental objectives and coordinate data generation, training, and acquisition.
2. Efficiency and Scalability: The approach demonstrates that decision-driven acquisition can significantly outperform non-adaptive sparse sampling and raster acquisition, enabling the investigation of dispersed populations and long-range spatial heterogeneity while maintaining nanoscale resolution.
3. Integrated Workflow: It connects physics-informed synthetic data generation, model training, and adaptive acquisition into a single, traceable, and reproducible workflow.
4. Beyond Imaging: The resulting object-level morphological library allows for the quantitative characterization and ranking of structures based on learned representations, moving beyond simple imaging to intelligent analysis of sample variability.

The authors note that while the system reduces the need for manual annotation and programming, challenges remain regarding the iterative evaluation of synthetic data diversity and the reliability of spatial reconstruction for extremely sparse populations. They position this work as a foundation for more intelligent AFM systems capable of adapting experimental strategies based on what they learn during measurement.