Machine learning inference of fission yields from gamma spectroscopy for very low-yield nuclear test verification

This paper demonstrates that machine learning models trained on high-fidelity simulated gamma spectroscopy data can accurately classify and estimate the fission yields of very low-yield nuclear tests, offering a viable technical solution for verifying the Comprehensive Nuclear-Test-Ban Treaty's zero-yield standard.

The Gist

Imagine a world where countries have promised not to build or test nuclear bombs. To keep this promise, they agreed on a "zero-yield" rule: no experiment is allowed to create a self-sustaining nuclear chain reaction, even if it's tiny.

The problem? It's incredibly hard to prove someone didn't do a tiny, secret test. If a country compresses a small amount of plutonium with conventional explosives just enough to make a few atoms split, it might not be loud enough to hear, and the radioactive dust might be too faint to see with standard tools. It's like trying to find a single dropped coin in a dark, noisy room.

This paper proposes a new way to find that "coin" using Machine Learning (AI) and Gamma Spectroscopy (a way of measuring radioactive light).

Here is the simple breakdown of what the researchers did and found:

1. The "Digital Time Machine"

Since we can't actually go around blowing up tiny nuclear devices to test our detectors, the researchers built a massive digital simulation.

• They created a virtual world with 66 million different scenarios.
• They simulated everything: different amounts of plutonium, different sizes of the container holding the test, different times of day the measurement was taken, and different amounts of "noise" in the data.
• Think of this as training a detective by showing them 66 million different crime scenes in a video game, so they learn exactly what a "guilty" scene looks like.

2. The "Fingerprint" of a Test

When a nuclear test happens, it leaves behind a specific mix of radioactive particles (fission products) and leftover plutonium. These particles emit gamma rays (invisible light) that act like a barcode.

• The researchers looked at the ratio between the "barcode" of the fission products and the "barcode" of the leftover plutonium.
• They realized that while many things (like how thick the container walls are) can blur this barcode, the ratio between specific lines of light still holds the secret to how big the explosion was.

3. The AI Detective

The team taught a specific type of AI (called XGBoost, which is like a very sharp, organized decision-maker) to look at these gamma ray barcodes and answer two questions:

1. The "Stop/Go" Question (Classification): Did the test exceed a specific limit (e.g., 1 kilogram of TNT)?
2. The "How Big?" Question (Regression): Exactly how much energy did the test release?

4. The Results: The AI is Surprisingly Good

The AI performed like a champion detective:

• For the "Stop/Go" question: It was incredibly accurate. If the test was just slightly above or below the limit (like 1 kg of TNT), the AI could tell the difference with over 95% accuracy. It's like a security guard who can tell the difference between a 1-pound and a 1.1-pound package almost perfectly.
• For the "How Big?" question: It could estimate the size of the explosion with a very small margin of error (about 12% off on average), even if the measurement was taken a month or a year after the test.

5. Why This Matters for the Future

The paper argues that while the current rules focus on whether a reaction was "self-sustaining" (a physics concept that is hard to measure directly), it might be easier and more effective to enforce a rule based on yield limits (e.g., "No tests bigger than 1 gram of TNT").

The AI shows that we can technically verify these tiny limits. If countries agree on a specific limit, this AI system could be the "truth-teller" that checks if anyone broke the rule, even if the explosion was too small to be seen by traditional methods.

In short: The researchers built a super-smart AI trained on 66 million fake nuclear tests. They found that this AI can look at the radioactive dust left behind and accurately tell if a secret, tiny nuclear test happened and how big it was, offering a new tool to help keep the world's nuclear test ban honest.

Technical Summary

Technical Summary: Machine Learning Inference of Fission Yields from Gamma Spectroscopy for Very Low-Yield Nuclear Test Verification

Problem Statement
The Comprehensive Nuclear-Test-Ban Treaty (CTBT) and existing testing moratoria rely on a "zero-yield standard," which prohibits any explosive experiment producing a self-sustaining fission chain reaction while permitting strictly subcritical experiments. A significant verification challenge arises from "very low-yield" tests (ranging from subcritical to barely supercritical), which may produce yields as low as milligrams of TNT equivalent. While on-site gamma spectroscopy of post-test debris offers potential insights, traditional analytical approaches struggle to infer yield or criticality levels due to confounding factors: the criticality level ($\alpha(t)$) does not produce a direct spectral signature distinct from yield; and measured spectra are heavily influenced by unknown parameters such as the time elapsed since the test, shielding effects, plutonium mass, and pre-test criticality configurations. Consequently, inspectors lack robust, data-driven methods to distinguish compliant subcritical tests from non-compliant supercritical ones or to estimate actual yields with high confidence.

Methodology
The authors propose a machine learning (ML) approach to infer fission yields from simulated gamma spectroscopy data, bypassing the need for explicit analytical inversion of complex physical parameters.

1. Data Generation:

   • Simulation Framework: The authors utilized high-fidelity 3-D Monte Carlo particle transport simulations (OpenMC) coupled with depletion codes (ONIX) and decay modeling (decaypy).
   • Scenarios: They generated 66 million spectral datasets representing a wide range of very low-yield test scenarios.
   • Parameters: The simulations varied key parameters including yield (0.1 mg to 1 ton TNT), time after test (30–365 days), vessel shielding thickness (3–8 cm), plutonium mass (0.15–3 kg), and pre-test assembly configuration (solid sphere vs. shell).
   • Detector Modeling: A High Purity Germanium (HPGe) detector was modeled 20 cm from the vessel, incorporating realistic pulse-height tallies and Gaussian energy broadening. Statistical noise (Poisson fluctuations) was added to simulate real-world counting conditions.

2. Feature Engineering:

   • Instead of using raw spectra, the authors extracted 82 fission-product-to-plutonium-239 peak ratios. This physics-informed approach aims to mitigate the influence of spatial debris distribution and shielding, assuming fission products and plutonium are similarly distributed.
   • Undetectable peaks (below the Currie detection limit) were set to zero, and spectra where the Pu-239 reference line was undetectable were excluded.

3. Machine Learning Models:

   • Tasks: The study addressed two tasks: (1) Binary classification (determining if yield exceeds a specific threshold) and (2) Regression (estimating the actual yield value).
   • Algorithms: Six ML methods were compared: XGBoost (Extreme Gradient Boosting), Random Forest (RF), Decision Tree (DT), Multi-Layer Perceptron (MLP), K-Nearest Neighbors (KNN), and Support Vector Classifier (SVC).
   • Training Strategy: Models were trained using 5-fold stratified cross-validation. Hyperparameters were optimized via randomized search. The authors investigated the impact of training data ranges (narrow vs. wide) and the inclusion of auxiliary parameters (time, shielding, mass) as input features.

Key Contributions

• Dataset Creation: The generation of a massive, high-fidelity dataset of 66 million simulated gamma spectra covering the critical 1 g to 100 kg TNT yield window, which is relevant for verifying the zero-yield standard and potential yield thresholds.
• ML Application in Verification: The first application of ML methods, specifically Gradient Boosting, to infer fission yields from post-test gamma spectroscopy in the context of very low-yield nuclear test verification.
• Feature Importance Analysis: The use of SHAP (SHapley Additive exPlanations) values to identify the most informative gamma-ray peak ratios (e.g., Nb-95, Ru-106, and the 511 keV annihilation line) that drive model predictions, offering physical interpretability.
• Training Strategy Insights: An analysis demonstrating that while adding known parameters (like time or mass) as features can improve performance in specific yield regimes, it introduces dependency on inspector knowledge. The study advocates for conservative training strategies using broad parameter ranges to ensure model robustness when such parameters are unknown.

Results

• Model Performance:
  • Classification: XGBoost outperformed other methods across all tested yield thresholds (1 g to 100 kg TNT). The classifier achieved high accuracy, with F1 scores exceeding 0.99 for thresholds between 100 g and 1 kg TNT. Misclassification rates were highly concentrated near the decision boundary, dropping to near zero for yields significantly above or below the threshold.
  • Regression: The regression model achieved a mean absolute relative error of 12.4% for measurements taken one month to a year after the test. Approximately 80% of predictions fell within a relative error of -16.5% to +19.7%.
• Feature Reduction: Models trained on reduced feature sets (top 20 or top 3 SHAP-ranked ratios) maintained performance comparable to the full 82-feature model for lower yield thresholds, though performance degraded slightly at higher thresholds when using only the top 3 features.
• Parameter Sensitivity:
  • Time: For low yields, longer wait times reduced detectability; for high yields, longer wait times improved detectability as plutonium lines emerged from the fission product background.
  • Shielding: Increased shielding degraded performance for low yields but had negligible impact on high yields.
  • Plutonium Mass: The model showed non-monotonic performance relative to mass due to the implicit encoding of mass in the peak ratios.
• Training Ranges: Training on wider parameter ranges generally prevented performance degradation when testing on unseen regimes, whereas narrow training ranges led to poor extrapolation.

Significance and Claims
The paper claims that using machine learning to infer the yield of past very low-yield nuclear tests from gamma spectroscopy data is feasible and accurate. Specifically:

• The proposed method can reliably classify whether a test exceeded a chosen yield threshold (e.g., 1 kg TNT) with high confidence, even months after the event.
• While the current zero-yield standard focuses on criticality (which is difficult to infer directly from spectra), the ability to accurately estimate yield provides a pathway to narrow the estimate of criticality levels, thereby supporting compliance verification.
• The authors suggest that a yield-threshold-based verification regime (e.g., banning tests above 1 g TNT) might be technically more verifiable than the current zero-yield standard, potentially facilitating the entry into force of the CTBT.
• The study emphasizes that this approach is not a standalone solution but should be integrated with other verification methods (e.g., satellite imagery, transparency measures) and that the ML models rely on simulated data, with future work needed to incorporate experimental data via transfer learning.

The authors maintain a modest tone, noting that while the technical capability exists, the political and geopolitical context of verification remains a separate, critical challenge. They do not claim to solve the ambiguity of the zero-yield standard itself but rather provide a technical tool that could inform discussions on its interpretation and the establishment of a robust verification protocol.