Technical Summary: AI-Driven Scan-to-Discovery in 20 Minutes

Problem Statement

Synchrotron X-ray micro-computed tomography (micro-CT) generates high-resolution volumetric datasets that reveal detailed 3D microstructures essential for fields ranging from materials science to plant physiology. However, converting raw scans into scientific insights remains computationally demanding. Traditional analysis relies on extensive manual annotation, which can take hundreds of hours per scan, creating a bottleneck for high-throughput experiments. Furthermore, existing foundation models trained on natural images or optical microscopy often struggle to generalize to the unique characteristics of X-ray micro-CT data, such as low contrast and specific structural noise. The challenge is to integrate data acquisition, tomographic reconstruction, and large-scale AI-driven segmentation into a unified pipeline that operates within the strict time constraints of a synchrotron beamline (targeting ~20 minutes) to enable near real-time discovery.

Methodology

The authors present an automated, end-to-end pipeline orchestrated by Prefect, designed to process data from the Advanced Light Source (ALS) Beamline 8.3.2. The workflow is distributed across the National Energy Research Scientific Computing Center (NERSC) and the Argonne Leadership Computing Facility (ALCF).

1. Data Acquisition and Transfer

• Acquisition: Samples are scanned, producing raw projection data stored in HDF5 files (~10 GB for ~1,000 slices).
• Transfer: Upon completion, data are automatically transferred via the Globus Transfer API to HPC resources at NERSC and ALCF.

2. Tomographic Reconstruction

• Processing: Raw projections undergo distributed tomographic reconstruction using large-scale multi-node CPU parallelism.
• Output: The process generates a stack of reconstructed 2D TIFF slices (2560 × 2560 pixels).
• Format Conversion: Reconstructed volumes are converted to the OME-Zarr format to enable scalable storage and efficient parallel access, reducing storage size by approximately 75%.

3. AI-Driven Segmentation

The core innovation involves the distributed inference of two fine-tuned foundation models running in parallel across multi-node GPU clusters (up to 200 GPUs):

• Instance Segmentation (SAM3): The Segment Anything Model 3 (SAM3) is fine-tuned to detect individual cell instances. It utilizes a perception encoder and a DETR-based detector. To handle high-resolution images without losing small details, images are cropped into 640 × 640 patches with 25% overlap, and predictions are merged using a max-confidence strategy.
• Semantic Segmentation (DINOv3-Seg): A DINOv3 backbone (specifically a ViT-L/16 distilled variant) combined with a lightweight mask module (based on the Encoder-only Mask Transformer) is fine-tuned for semantic segmentation. This model captures global context, which is often lost during the cropping required for SAM3.
• Aggregation: The outputs from both models are combined to produce final segmentation masks. SAM3 provides precise polygon boundaries for specific structures (e.g., xylem vessels), while DINOv3-Seg provides global semantic context to filter out-of-region predictions.

4. Training Data Generation

To address the lack of labeled micro-CT data, the authors developed an AI-assisted, human-in-the-loop annotation pipeline:

• Semantic Masks: Domain experts annotate semantic regions (e.g., cortex, phloem, hydrated/dehydrated xylem) using a web-based interface.
• Instance Masks: The Cellpose-SAM model is used to generate initial instance masks (polygons) for cells.
• Label Propagation: Class IDs are automatically assigned to detected cells by checking if the cell polygon falls within a manually annotated semantic region. This process generated a dataset of 1,050 images with over 107,000 annotated cells from 42 grapevine petiole slices.

5. Visualization

Selected scans undergo optional 3D visualization. Segmented volumes are converted to Zarr and visualized using itk-vtk-viewer. Additionally, SurfaceNets algorithms generate 3D surface meshes for interactive exploration of vascular structures.

Key Contributions

1. End-to-End Pipeline: An automated system integrating data acquisition, HPC-based reconstruction, and large-scale AI segmentation, achieving a ~20-minute scan-to-discovery turnaround.
2. Foundation Model Adaptation: Successful fine-tuning of state-of-the-art foundation models (SAM3 and DINOv3-Seg) for the specific domain of synchrotron micro-CT, demonstrating that these models can generalize to scientific imaging modalities when properly adapted.
3. Hybrid Segmentation Strategy: A novel approach combining instance segmentation (for precise boundaries) and semantic segmentation (for global context) to overcome the limitations of cropping high-resolution images.
4. Efficient Annotation Workflow: A human-in-the-loop strategy that leverages AI to generate instance masks, significantly reducing the manual effort required to create training datasets for scientific imaging.
5. Scalable Deployment: Demonstration of the pipeline across two major DOE HPC facilities (NERSC and ALCF), utilizing distributed CPU and GPU resources to handle large-scale inference.

Results

• Performance: The pipeline successfully processed 134 live scans (average size ~9.95 GB) during beamtime.
• Timing:
  • Reconstruction: Completed in under 1 minute using 16 CPU nodes (vs. ~10 minutes on a single node).
  • Segmentation: Completed in under 10 minutes using distributed GPU nodes (32 GPUs for SAM3, 8 for DINOv3-Seg).
  • Total Turnaround: Approximately 20 minutes from scan completion to segmented results.
• Accuracy:
  • SAM3: Fine-tuning improved the Average Precision (AP) from 2.7 (pre-trained) to 54.8 on petiole micro-CT data. The model effectively detected small, dense cells that pre-trained models missed.
  • DINOv3-Seg: Fine-tuning improved Mean Intersection over Union (mIoU) from <7% to ~81% (using ViT-L/16 backbone).
  • Qualitative: The combined model successfully distinguished between hydrated and dehydrated xylem vessels in grapevine petioles, revealing structural changes and embolism formation over time-series scans.
• Resource Usage: The live experiments consumed substantial HPC resources (e.g., ~21,324 GPU hours at NERSC during live processing), highlighting the computational intensity of the task.

Significance and Claims

The paper claims that this work transforms synchrotron imaging from an offline analysis paradigm into an interactive, automated discovery platform. By reducing the time to scientific insight from weeks to minutes, the pipeline enables researchers to:

• Monitor dynamic processes (e.g., plant dehydration and embolism) in near real-time.
• Steer experiments based on immediate structural feedback.
• Scale analysis to datasets that were previously intractable due to manual annotation requirements.

The authors emphasize that the pipeline is generalizable; adapting it to new micro-CT datasets requires only a small set of annotated masks and model selection, without changing the underlying workflow. While the current demonstration focuses on grapevine petioles, the modular design and foundation model approach are presented as a blueprint for broader application in materials science, geoscience, and energy research. The work does not claim to solve all segmentation challenges universally but demonstrates a viable path for integrating AI and HPC into high-throughput scientific facilities.