Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

This paper demonstrates that unsupervised domain adaptation, specifically through minimizing maximum mean discrepancy (MMD) between synthetic and unlabeled real-world data, significantly improves the generalization and testing accuracy of machine learning models for radioisotope identification in gamma spectroscopy.

The Gist

Imagine you are trying to teach a dog to identify different types of cars just by looking at them.

The Problem: The "Video Game" vs. The "Real World"
In this paper, the "dog" is a computer program (an AI) designed to identify radioactive materials (like specific isotopes) by looking at gamma-ray spectra (which are like unique fingerprints or barcodes of radiation).

The researchers faced a huge problem: They couldn't get enough real-world examples to train the dog. Real radioactive sources are dangerous, expensive to handle, and hard to find in every possible scenario. So, they trained the AI using simulations (like a high-tech video game). In the game, the AI learned perfectly.

But when they took this "game-trained" AI out into the real world to test it on actual detectors, it struggled. It was like teaching a dog to recognize cars in a cartoon, then bringing it to a real parking lot. The real world has noise, weird angles, and different lighting (or in this case, different detector types and background radiation) that the cartoon never showed. The AI got confused and made mistakes.

The Solution: "Unsupervised Domain Adaptation" (UDA)
The researchers wanted to fix this without needing to label thousands of real-world examples (which would take forever and require experts to tag every single one).

They used a technique called Unsupervised Domain Adaptation. Here is a simple analogy for how it works:

1. The Expert and the Intern: Imagine the AI trained on the simulation is an "Expert" who knows the theory perfectly but has never seen a real car. The real-world data is a pile of "Interns" (unlabeled real spectra) that the Expert hasn't met yet.
2. The Alignment: Instead of teaching the Interns what the cars are (which requires labels), the researchers taught the Expert to change how it sees things so that the real-world cars look more like the cartoon cars to its brain.
3. The Goal: They didn't ask the AI to guess the answer for the real data. Instead, they asked it to make the pattern of the real data match the pattern of the simulation data. Once the patterns align, the AI's existing knowledge (from the simulation) suddenly works on the real data.

The "Magic" Trick: MMD (Maximum Mean Discrepancy)
The paper tested many different ways to do this alignment. Think of it like trying to match two different languages. Some methods tried to translate word-for-word, others tried to match sentence structures.

The researchers found that the best method was called MMD (Maximum Mean Discrepancy).

• The Analogy: Imagine you have a bag of red marbles (simulation data) and a bag of blue marbles (real data). They look different. MMD is like a magic sieve that gently shakes the bags until the distribution of colors inside looks statistically identical, even if the individual marbles are still different. Once the "vibe" of the two bags matches, the AI can use its red-marble knowledge to handle the blue-marble bag.

The Results: From "Okay" to "Great"
The results were impressive:

• Before the fix: The AI got about 75% of the real-world tests right. It was guessing a lot.
• After the fix (using MMD): The AI jumped to 90% accuracy.

It's like taking a student who barely passed a driving test in a simulator and, after a specific kind of "re-calibration" training, having them ace the real road test without ever needing a driving instructor to point out every single mistake.

Why This Matters
This is a big deal for national security and radiation safety.

• Current situation: We often can't afford to collect enough real data to train AI for every possible detector or location.
• Future: We can now build AI on cheap, fast simulations, and then use this "alignment trick" to make it work perfectly on real, messy, noisy equipment in the field. It bridges the gap between the perfect world of computers and the messy world of reality.

In a Nutshell:
The paper shows that you don't need a million labeled real-world examples to train a radiation detector AI. You just need to teach the AI how to "translate" its simulation knowledge into the language of the real world, and it will suddenly become a master at its job.

Technical Summary

1. Problem Statement

Gamma spectroscopy is critical for radiation safety, national security, and nuclear forensics, but training robust machine learning (ML) models for radioisotope identification is hindered by the scarcity of labeled experimental data. Collecting and labeling diverse real-world spectra is expensive and time-consuming. Consequently, researchers rely on synthetic data (simulations) for training.

However, models trained on synthetic data suffer from the "sim-to-real" gap (domain shift), leading to significant performance degradation when deployed in operational environments. This gap arises from three types of domain shifts:

• Covariate Shift: Differences in the distribution of input spectra (e.g., noise levels, shielding).
• Prior Shift: Differences in the distribution of isotope labels (e.g., class imbalance).
• Concept Shift: Fundamental differences in the relationship between labels and spectral shapes due to detector response, geometry, or background.

The core challenge is to adapt a model trained on labeled synthetic data to a new, unlabeled real-world target domain without access to ground-truth labels for the target data.

2. Methodology

The authors propose a framework using Unsupervised Domain Adaptation (UDA) to align feature representations between the source (synthetic) and target (experimental) domains.

A. Model Architectures

The study evaluates three distinct neural network backbones:

1. Multilayer Perceptrons (MLP): Simple dense layers; lacks explicit spatial inductive bias.
2. Convolutional Neural Networks (CNN): Uses 1D convolutions to capture local spectral structure but struggles with long-range correlations.
3. Transformer-Based Neural Networks (TBNN): Leverages attention mechanisms to capture long-range correlations across the spectrum. The authors introduce variations with linear and non-linear embedding layers and global average pooling.

B. Unsupervised Domain Adaptation Techniques

The study compares seven UDA methods, all following a two-stage process: pretraining on labeled source data, followed by alignment using unlabeled target data.

1. ADDA (Adversarial Discriminative Domain Adaptation): Adversarially trains a target feature extractor to match the source distribution using a domain discriminator.
2. DAN (Deep Adaptation Networks): Minimizes the Maximum Mean Discrepancy (MMD) between source and target feature vectors in a Reproducing Kernel Hilbert Space (RKHS).
3. DANN (Domain Adversarial Neural Networks): Uses a gradient reversal layer to train a feature extractor that confuses a domain discriminator, forcing domain-invariant features.
4. DeepCORAL: Aligns second-order statistics (covariance matrices) of source and target distributions.
5. DeepJDOT (Joint Optimal Transport): Aligns joint distributions of features and labels using optimal transport, making it class-aware.
6. Mean Teacher: Enforces predictive consistency between a "student" and an exponential moving average (EMA) "teacher" model under noise augmentation.
7. SimCLR (Simple Contrastive Learning): Uses contrastive loss to learn augmentation-invariant representations by contrasting different views of the same spectrum.

C. Experimental Setup

The study utilizes three scenarios:

• Sim-to-Sim (HPGe): Source and target are both simulated (GADRAS vs. Geant4). This isolates concept shift (detector response differences).
• Sim-to-Real (LaBr3): Source is simulated (GADRAS); target is experimental data from a Lanthanum Bromide detector.
• Sim-to-Real (NaI(Tl)): Source is simulated; target is experimental data from a Sodium Iodide detector.

The datasets involve 32–55 isotopes, with target labels masked during training. Hyperparameters were optimized via Bayesian search (Optuna).

3. Key Contributions

1. Comprehensive UDA Benchmarking: The first systematic comparison of diverse UDA techniques (ADDA, DAN, DANN, etc.) specifically for gamma spectroscopy radioisotope identification.
2. Architecture Analysis: Demonstrates that Transformer-based architectures (TBNN) generally outperform MLPs and CNNs in this domain, particularly when combined with UDA.
3. Identification of Optimal Strategy: Finds that minimizing Maximum Mean Discrepancy (MMD) via DAN and adversarial training via DANN yield the most consistent improvements across different architectures and detectors.
4. Interpretability via SHAP: Uses SHapley Additive exPlanations to show that UDA models learn to ignore spurious detector artifacts (e.g., intrinsic X-ray peaks) and focus on physically relevant isotope lines, whereas source-only models often overfit to artifacts.
5. Diagnostic Validation: Provides a holistic evaluation beyond accuracy, including calibration, uncertainty estimation, and decision boundary smoothness.

4. Results

Performance Metrics

• Sim-to-Real (LaBr3): The best UDA model (DAN with TBNN-LinEmb) achieved a testing accuracy of 0.904 ± 0.022, a significant improvement over the source-only baseline of 0.754 ± 0.014 (a ~15 percentage point gain).
• Sim-to-Real (NaI(Tl): Similar trends were observed, with DAN and DANN providing double-digit accuracy improvements across MLP and TBNN families.
• Sim-to-Sim (HPGe): Improvements were more modest (APE score increase from ~0.65 to ~0.69) but statistically significant. The authors attribute this to the dominance of concept shift in this scenario, which is harder to resolve with unsupervised feature alignment alone.

Statistical Significance

Wilcoxon signed-rank tests confirmed that UDA models (specifically DAN and DANN with Transformers) provided statistically significant improvements over source-only baselines in 30 out of 35 comparisons for the LaBr3 detector and 27 out of 35 for the NaI(Tl) detector.

Diagnostic & Interpretability Findings

• Calibration: UDA models showed significantly lower Expected Calibration Error (ECE) and Brier scores, indicating more reliable probability estimates.
• SHAP Analysis: In a specific case study (40K isotope), the source-only model incorrectly identified the detector's intrinsic 32 keV Ba K-shell X-ray peak as the primary feature. The DAN-corrected model correctly ignored this artifact and focused on the true 1460 keV gamma line, improving classification accuracy for that isotope from 17% to 83%.
• Latent Space: While UMAP visualizations showed improved alignment in Sim-to-Sim scenarios, the Sim-to-Real improvements were driven by high-dimensional feature congruence relevant to the decision boundary rather than obvious 2D cluster alignment.

5. Significance

This paper establishes Unsupervised Domain Adaptation as a practical and essential tool for deploying radioisotope identification systems in the real world. It demonstrates that:

1. Synthetic-to-Real Transfer is Viable: High-performance models can be deployed in operational settings using only unlabeled experimental data for adaptation, bypassing the need for expensive labeled datasets.
2. Transformers are Superior: Transformer architectures are better suited for capturing the complex, long-range correlations in gamma spectra compared to traditional CNNs or MLPs.
3. MMD and Adversarial Methods are Robust: DAN and DANN are the most reliable UDA strategies for this specific physical domain.
4. Safety and Reliability: By correcting model reliance on spurious detector artifacts, UDA not only improves accuracy but also enhances the physical interpretability and safety of automated radiation detection systems.

The work bridges the gap between computer vision/NLP advancements and nuclear physics applications, offering a pathway to scalable, data-efficient radiation monitoring.