🤖 machine learning

Unlocking Latent Dimensions: Exploring Representations of Large-Scale X-ray Scattering Data using Variational Autoencoders

This paper introduces a domain-specific Convolutional Variational Autoencoder trained on 1.5 million X-ray scattering images to generate interpretable low-dimensional latent representations for both offline dataset exploration and real-time analysis, outperforming general-purpose vision models and enabling interactive structural exploration via the MLExchange platform.

Original authors: Monika Choudhary, Xiaoya Chong, Runbo Jiang, Wiebke Koepp, Petrus H. Zwart, Damon English, Gregory M. Su, Eric Schaible, Chenhui Zhu, Mostafa Nassr, Noah P. Wamble, Kelvin Kam-Yun Li, Jonathan M. Chan

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Monika Choudhary, Xiaoya Chong, Runbo Jiang, Wiebke Koepp, Petrus H. Zwart, Damon English, Gregory M. Su, Eric Schaible, Chenhui Zhu, Mostafa Nassr, Noah P. Wamble, Kelvin Kam-Yun Li, Jonathan M. Chan, Jose Carlos Diaz, Cameron McKay, Lynn Katz, Benny Freeman, Guillaume Freychet, Yevgen Matviychuk, Eliot Gann, Daniel B. Allan, Benedikt Sochor, Frank Schluenzen, Stephan V. Roth, Ethan Crumlin, Dylan McReynolds, Tanny Chavez, Alexander Hexemer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Too Much Data, Too Fast

Imagine a giant library where books (X-ray images) are being printed and thrown onto a conveyor belt faster than any librarian can read them. This is what happens at modern scientific facilities called "synchrotrons." They use powerful X-rays to take pictures of tiny materials, generating millions of images.

Traditionally, scientists had to stop the conveyor belt, pick up a few books, and read them slowly. But now, the data is coming so fast that this "stop and read" method is impossible. They need a way to understand the story of the data while it is still moving on the belt.

The Solution: A "Smart Translator" (The C-VAE)

The authors built a special computer program called a Convolutional Variational Autoencoder (C-VAE). Think of this program as a highly trained translator or a "smart librarian."

Training: They fed this program 1.5 million historical X-ray images. It wasn't just looking at the pictures; it was learning the "language" of X-ray scattering. It learned to recognize patterns like rings, streaks, and spots that tell scientists what a material is made of.
The Secret Code (Latent Space): Instead of keeping the massive, high-resolution images, the program compresses each one into a tiny, 512-number "ID card." This is called a latent representation.
- Analogy: Imagine taking a 100-page novel and summarizing it into a single sentence that captures the whole plot. If two novels have similar plots, their summary sentences will be very close to each other.
The Map: When the program creates these ID cards for thousands of images, it arranges them on a map.
- Clusters: Images that look similar (like different types of crystals) group together in tight neighborhoods.
- Trajectories: If an experiment changes over time (like a film drying out), the ID cards form a smooth, winding road on the map, showing the step-by-step evolution of the material.

Putting It to the Test: Two Scenarios

The team tested this "Smart Translator" in two ways:

1. The "Time Travel" Test (Offline Analysis)
They took a massive archive of old data and ran it through the program. The result? The program organized the chaos perfectly. It grouped similar experiments together and showed clear paths for how experiments progressed over time. It was like sorting a messy attic and finding that all the winter coats are in one pile and all the summer hats are in another, with a clear path showing how the seasons changed.

2. The "Live Broadcast" Test (On-the-Fly Analysis)
This is the real magic. They turned the program on during live experiments at two different scientific facilities (one in California, one in New York).

No Retraining: They didn't teach the program anything new. They just turned it on.
The Result: As new images streamed in, the program instantly placed them on the map. Scientists could watch a "movie" of the material changing in real-time. They could see exactly when a material shifted from a liquid state to a solid state, just by watching the dot move along the road on the map.

The "Specialist vs. Generalist" Showdown

To prove their point, the authors compared their custom-built program against a famous, general-purpose AI called DINOv3.

DINOv3 is like a brilliant art critic who has seen millions of photos of cats, cars, and landscapes. It's very smart.
The C-VAE is like a specialist who has only studied X-ray patterns for 10 years.

The Verdict: When looking at X-ray data, the specialist (C-VAE) was much better. The generalist (DINOv3) could see that the pictures were different, but it got confused by the specific details of X-ray physics. The specialist organized the data into much clearer groups and smoother paths. This proves that for very specific scientific tasks, a custom-trained tool works better than a "one-size-fits-all" AI.

The "Imagination Engine" (Synthetic Generation)

Because the program understands the "grammar" of X-ray images so well, it can also invent new ones.

The team asked the program: "Show me what an X-ray image would look like if the material was in a state we haven't seen yet."
The program generated brand new, realistic X-ray images that fit perfectly into the gaps on its map. This is like a chef who knows the recipe for every dish in a cuisine and can invent a new dish that tastes exactly right, even if no one has ever ordered it before.

The User Interface: "Latent Space Explorer"

Finally, they put all this power into a web tool called Latent Space Explorer.

For Offline: Scientists can upload old data, click around the map, and explore clusters to find hidden patterns.
For Live: During an experiment, the tool updates in real-time. A scientist can look at the screen, see a dot moving, and immediately know, "Ah, the material is changing structure right now!"

Summary

The paper describes building a specialized AI that acts as a super-organizer for X-ray data. It turns millions of confusing images into a simple, navigable map. This allows scientists to understand complex material changes instantly, whether they are looking at old data or watching a live experiment, and even helps them imagine new possibilities for their research.

Technical Summary: Unlocking Latent Dimensions in Large-Scale X-ray Scattering Data

Problem Statement

Scientific user facilities, particularly synchrotrons, generate X-ray scattering data at rates that exceed the capacity of traditional manual inspection and offline batch processing workflows. The resulting datasets are heterogeneous, high-dimensional, and massive in scale (millions of images). While machine learning has shown promise in analyzing complex X-ray data, a critical gap remains in extracting interpretable structure from these large archives in real time. A specific challenge addressed in this work is determining whether general-purpose vision foundation models (trained on natural images) are sufficient for scattering data analysis, or if the unique structural characteristics of scattering patterns (e.g., diffraction rings, peak symmetry) necessitate domain-specific models.

Methodology

1. Model Architecture: Attention-Based Convolutional Variational Autoencoder (C-VAE)

The authors developed a domain-specific Convolutional Variational Autoencoder (C-VAE) to learn low-dimensional representations of X-ray scattering images.

Hybrid Design: The architecture combines Convolutional Neural Networks (CNNs) for capturing local spatial features with windowed self-attention mechanisms (inspired by Swin Transformers) to model long-range dependencies and hierarchical structures.
Input/Output: The model processes $512 \times 512$ pixel images and maps them to a 512-dimensional latent space.
Probabilistic Framework: As a VAE, it enforces a structured, continuous latent space, facilitating downstream tasks like clustering and interpolation. The encoder outputs the mean and log-variance of a diagonal Gaussian distribution, utilizing the reparameterization trick for stochastic sampling.

2. Training Data and Infrastructure

Dataset: The model was trained on 1.5 million X-ray scattering images collected over a decade at the Advanced Light Source (ALS), specifically from beamline 7.3.3. The data encompasses diverse material systems (crystalline powders, liquid crystals, amorphous materials, thin films) and geometries (transmission and grazing-incidence).
Hardware: Training was conducted on the NERSC Perlmutter supercomputer using 80 NVIDIA A100 GPUs over approximately 48 hours.
Preprocessing: Images were resized from their native resolution ( $1475 \times 1679$ ) to $512 \times 512$ prior to encoding.

3. Synthetic Data Generation Strategies

To generate controlled synthetic scattering images from the latent space, two strategies were implemented:

UMAP-guided PCA Sampling: Cluster-aware PCA models are fitted to core latent vectors. New samples are generated by sampling Gaussian noise in the PCA space of specific clusters, weighted by a temperature-controlled softmax over nearby UMAP centroids.
Conditional Flow Matching (CFM): A velocity network learns a continuous normalizing flow from a standard Gaussian prior to the training latent distribution, conditioned on 2D UMAP coordinates. This method incorporates classifier-free guidance to improve conditioning fidelity.

4. Software Infrastructure

The workflow is integrated into the MLExchange platform via the Latent Space Explorer component. This system supports:

Offline Analysis: Interactive exploration of archived datasets using dimensionality reduction (PCA, UMAP, t-SNE) and clustering (HDBSCAN).
On-the-Fly Analysis: Real-time embedding of streaming detector images with an inference time of approximately 0.05 seconds per sample.
Model Management: Integration with MLflow for versioned model registration and Tiled for data access.

Key Results

1. Latent Space Organization

The C-VAE learned a well-organized latent space from the 1.5 million historical images:

Clustering: HDBSCAN identified distinct clusters corresponding to different scattering regimes (e.g., isotropic rings vs. sharp peaks).
Trajectories: Within clusters, embeddings formed smooth, line-like trajectories that reflected the temporal progression of individual experiments, capturing continuous morphological transitions.
Interpretability: The latent space successfully separated independent experiments while grouping structurally similar patterns, a structure not recoverable by projecting raw pixel intensities directly.

2. On-the-Fly Deployment and Generalization

The pre-trained C-VAE was deployed without retraining for live analysis at two facilities:

ALS Beamline 7.3.3: Successfully tracked the structural evolution of perfluorosulfonic acid (PFSA) ionomer films during blade coating, capturing transitions from solvent-dominated states to semi-crystalline aggregates.
NSLS-II SMI Beamline: The model generalized to a different facility with distinct detector hardware and beamline geometry, maintaining smooth latent trajectories and consistent clustering of structural states.

3. Benchmarking Against General-Purpose Models

The C-VAE was benchmarked against DINOv3 (ViT-7B), a general-purpose vision foundation model trained on natural images:

Performance: While DINOv3 produced coherent trajectories, its cluster assignments were more fragmented, and within-run trajectories were less smooth.
Interpretability: The domain-specific C-VAE demonstrated superior ability to organize scattering-specific features (diffraction ring symmetry, peak sharpness, orientation) compared to the general-purpose model, which relies on natural image features like texture and color contrast.

4. Synthetic Image Generation

Both sampling strategies produced physically realistic synthetic scattering images (concentric rings, diffuse halos, anisotropic distributions) that occupied the same regions of the UMAP embedding as the training data. Conditional Flow Matching achieved superior conditioning fidelity compared to UMAP-guided PCA sampling.

Significance and Claims

The paper claims that training a domain-specific model on large-scale scattering data yields more interpretable latent organization than general-purpose vision models. The key contributions include:

Demonstration of Low-Dimensionality: Showing that dominant structural variations in X-ray scattering data can be captured in a compact 512-dimensional latent space, justifying real-time analysis without significant loss of discriminative information.
Cross-Facility Generalization: Proving that a model trained at one synchrotron (ALS) can effectively organize data from another (NSLS-II) without retraining, suggesting the learned representations capture fundamental physics rather than facility-specific artifacts.
Integrated Workflow: Providing a unified software environment (Latent Space Explorer) that bridges offline archival exploration and live experimental monitoring, enabling interactive structural discovery.
Foundation for Automation: Establishing a structured latent space that can serve as an interface for future autonomous experimental control and language model agents, moving beyond passive monitoring to data-driven decision-making.

The authors note that while the results are promising, the specific scientific findings from the P03 beamline deployment are beyond the scope of this methods-focused study and will be reported separately. The work emphasizes the necessity of domain-specific training for handling the unique textural and structural features of scientific scattering data.