New Deep Learning Data Analysis Method for PROSPECT using GAPE: Genetic Algorithm Powered Evolution

This paper introduces GAPE, a genetic algorithm-powered evolution method that optimizes deep learning models for the PROSPECT experiment, achieving a nearly 2.8-fold improvement in signal-to-background ratio for reactor antineutrino identification while addressing and mitigating time-dependent training biases.

The Gist

Imagine you are trying to find a specific, rare type of bird (a neutrino) in a massive, noisy forest (a nuclear reactor). The forest is filled with thousands of other birds, rustling leaves, and wind (background noise) that sound almost exactly like the bird you are looking for. Your job is to spot the rare bird, figure out exactly where it landed, and measure how big it is, all while the forest itself is slowly changing shape and color over time.

This paper describes a new, high-tech way to do this using a method called GAPE (Genetic Algorithm Powered Evolution). Here is how it works, broken down into simple concepts:

1. The Problem: Finding a Needle in a Haystack

The PROSPECT experiment is a giant detector sitting next to a nuclear reactor. It's designed to catch "antineutrinos" (ghostly particles) that fly out of the reactor.

• The Challenge: The detector is huge and complex. When a neutrino hits it, it creates a tiny flash of light. But the detector also gets hit by cosmic rays and reactor noise constantly.
• The Old Way: Scientists used traditional math rules (like a rigid checklist) to decide if a flash of light was a real neutrino or just noise. They also used standard math to guess where the flash happened and how much energy it had.
• The Issue: These old rules are a bit clumsy. They miss some real neutrinos and sometimes mistake noise for neutrinos. Also, the detector gets "tired" over time (its sensors degrade), making the old rules less accurate as the experiment goes on.

2. The Solution: Evolution in a Computer (GAPE)

Instead of writing the rules by hand, the authors let a computer evolve its own rules. They used a technique inspired by Darwin's Theory of Evolution.

• The "Genes": Imagine a computer program that can build a "brain" (a Deep Learning model). This brain has many parts: how many layers it has, how it learns, and which data it looks at. Each part is a "gene."
• The "Survival of the Fittest":
  1. The computer creates 1,000 different "brains" with random settings.
  2. It tests them all to see which one is best at finding neutrinos.
  3. The "losers" (bad brains) are deleted.
  4. The "winners" (good brains) are mated together to create a new generation of brains.
  5. Sometimes, a random "mutation" happens (a tiny change) to see if it makes the brain smarter.
• The Result: After many generations, the computer evolves a "Super Brain" that is perfectly tuned to the PROSPECT data. It figures out the best way to look at the data without a human needing to guess the settings.

3. What Did the Super Brain Do?

The GAPE method created three specialized tools:

• The Map Maker (Position Estimator): It learned to pinpoint exactly which tiny tube in the detector the neutrino hit.
  • The Win: It was slightly better than the old method, especially in crowded areas of the detector where the old method got confused.
• The Scale (Energy Estimator): It learned to calculate the true energy of the neutrino.
  • The Win: It was more accurate at guessing the energy, especially for higher-energy neutrinos, reducing the "fuzziness" of the measurement.
• The Bouncer (IBD Classifier): This is the most important tool. It decides, "Is this a real neutrino interaction or just background noise?"
  • The Win: This is where the magic happened. The new AI classifier improved the Signal-to-Background Ratio by nearly 3 times.
  • Analogy: Imagine the old method let 100 noise events through for every 100 real neutrinos. The new AI lets through only about 35 noise events for every 100 real neutrinos. It's like upgrading from a sieve with big holes to a fine mesh that catches almost all the dust.

4. The "Aging" Problem and the Fix

There was a catch. When they first tested the "Super Brain" on real data, it was biased. It was too picky and started rejecting real neutrinos because the detector had changed slightly over time (like a camera lens getting a bit dusty). The AI thought the dusty lens meant the bird wasn't there.

• The Fix: The scientists realized they needed to train the AI on data from a specific "season" of the experiment, rather than mixing data from the whole year.
• The Result: By training the AI on a specific time period, they fixed the bias. The AI learned to ignore the "dust" and focus on the bird, making it much fairer and more accurate for future data.

Summary

This paper is about teaching a computer to evolve its own best way to analyze particle physics data.

• Old Way: Humans write rigid rules.
• New Way (GAPE): Humans set up a competition, and the computer evolves the best possible rules automatically.

The result is a system that is much better at finding rare particles, measuring them accurately, and ignoring the noise, even as the equipment changes over time. It's a powerful new tool that could help scientists understand the universe better, not just in this experiment, but in many other fields of science.

Technical Summary

1. Problem Statement

The PROSPECT (Precision Reactor Oscillation and SPECTrum) experiment aims to measure the antineutrino energy spectrum from the High Flux Isotope Reactor (HFIR) at Oak Ridge National Laboratory and search for sterile neutrinos. The experiment faces two primary analytical challenges:

• Signal vs. Background Separation: Distinguishing true Inverse Beta Decay (IBD) interactions (signal) from various backgrounds, including accidental coincidences (reactor gammas + cosmic neutrons) and correlated cosmic backgrounds.
• Reconstruction Accuracy: Precisely estimating the energy and the Segment of Interaction (SOI) for detected events.
• Detector Instability: The detector exhibits time-dependent response variations (e.g., scintillation light decay, PMT malfunctions) that complicate the training of machine learning models, often leading to biases when models trained on Monte Carlo (MC) simulations are applied to real data.

Traditional analysis methods (P2X) rely on maximum-likelihood fits and fixed selection cuts. While effective, these methods may not fully exploit the complex, non-linear relationships within the high-dimensional data space.

2. Methodology: GAPE

The authors propose GAPE (Genetic Algorithm Powered Evolution), a novel framework that uses Genetic Algorithms (GAs) to automate the design and optimization of Deep Learning (DL) models.

• Genetic Algorithm Core:
  • Population: A population of "genes" (chromosomes) encodes the architecture and hyperparameters of neural networks.
  • Evolution: Models compete in a "survival of the fittest" cycle. The top-performing models (based on a fitness function) mate via "uniform crossing" to create offspring.
  • Mutation: Random mutations (7% rate) introduce diversity to prevent local optima.
  • Fitness Functions:
    • Regression (Energy): $R^2$ score minus standard error.
    • Classification (SOI): Accuracy.
    • Classification (IBD): F1-score (optimized for balancing Precision and Recall).
• Two-Stage Feature Selection:
  1. Stage 1: The GA evolves both the network architecture and the feature set. It systematically weeds out less useful features (dimensionality reduction).
  2. Stage 2: The best feature set from Stage 1 is fixed, and the GA optimizes only the network architecture and hyperparameters.
• Input Features: The models utilize 8 potential features per segment, including reconstructed longitudinal position ($z$), hit times ($t$), photon counts (PEL, PER), visible energy ($E_{vis}$), and Pulse Shape Discrimination (PSD). The GA consistently selected $t$, PEL, PER, and PSD as the most critical inputs.
• Model Architectures: The GA evolved specific architectures for three tasks:
  • Energy Estimator: 6-layer feed-forward network (Input: 616 $\to$ Hidden: 900, 700, 350, 250 $\to$ Output).
  • SOI Classifier: 4-layer network (Input: 616 $\to$ Hidden: 475, 450 $\to$ Output: 154 segments).
  • IBD Classifier: 6-layer network with sigmoid output.

3. Key Contributions

• Automated Architecture Search: Demonstrated that GAPE can autonomously design deep neural networks that outperform traditional statistical methods and manually tuned models for reactor neutrino analysis.
• Bias Mitigation Strategy: Identified that time-dependent detector response differences between MC simulations and real data caused significant classification bias. The authors introduced a data-period-specific training regimen (training on a specific, shorter time window where detector response was stable) to mitigate this bias.
• Real-Background Training: Developed a "mixed bag" training strategy using real reactor-off cosmogenic backgrounds and reactor-on accidental backgrounds, combined with simulated IBD signals, to create a realistic training environment.

4. Results

A. Segment of Interaction (SOI) Classification

• Performance: The ML model achieved 98.7% accuracy (with low energy and dead-segment cuts), compared to 97.9% for the traditional P2X method.
• Observation: The ML model showed the most significant gains in segments surrounded by active neighbors, whereas traditional methods performed better in isolated segments.

B. Energy Estimation

• Performance: The ML model achieved an $R^2$ score of 0.892 (with cuts), outperforming the traditional model's 0.873.
• Resolution: The ML model provided superior energy resolution, particularly above 3 MeV, with reduced bias and standard deviation compared to the traditional method.

C. IBD Classification (Signal-to-Background Ratio)

• Classifier 1 (Initial): Trained on a broad dataset. It improved the Signal-to-Background Ratio (SBR) from 0.77 to 2.8 (a ~3.6x improvement). However, it exhibited a 32% bias in the selected fraction when applied to real data versus the validation set.
• Classifier 2 (Bias-Corrected): Trained on a specific, shorter data period ("Period 2") to align MC and real data responses.
  • Bias Reduction: The bias in the selected fraction dropped significantly (from 32% to ~6%), indicating the time-dependent response was the primary source of the initial bias.
  • SBR Improvement: Achieved an SBR of 2.4 (improving from a baseline of 0.87), representing a ~2.8x improvement over conventional selection.
  • Efficiency: While precision/recall metrics were slightly lower than Classifier 1, the reduction in systematic bias makes Classifier 2 more robust for physics analysis.

5. Significance

• Superior Performance: The GAPE-selected models demonstrate that automated deep learning can outperform traditional, manually tuned statistical methods in reconstructing neutrino energy and position, and in background rejection.
• Systematic Uncertainty Reduction: By improving the Signal-to-Background Ratio by nearly 2.8 times, the method significantly suppresses systematic uncertainties related to background modeling. This allows for potential signal deviation testing without explicit background subtraction.
• Generalizability: The GAPE framework is not limited to PROSPECT; it offers a generalizable approach for optimizing machine learning models in particle physics, particularly for problems involving complex detector responses and high-dimensional feature spaces.
• Addressing the "Black Box": The paper highlights the importance of understanding data drift (time-dependent response) in ML applications. By using period-specific training, the authors successfully mitigated the "black box" bias where models learn artifacts of the simulation rather than physical reality.

In conclusion, this work establishes GAPE as a powerful tool for next-generation neutrino experiments, providing a pathway to unbiased, high-precision data analysis that adapts to the evolving nature of complex detector systems.