Weakly supervised machine learning for model-agnostic searches of new phenomena in the $\gamma$-ray sky

Technical Summary: Weakly Supervised Machine Learning for Model-Agnostic Searches in the $\gamma$-Ray Sky

Problem Statement
The $\gamma$-ray sky, particularly as observed by the Fermi Large Area Telescope (Fermi-LAT), contains a significant population of unassociated sources that may represent new astrophysical populations or exotic physics (e.g., dark matter subhalos, axion-photon oscillations). While machine learning (ML) has become central to source classification and new physics searches, existing approaches rely heavily on fully supervised training. This requires explicit, labeled signal models for every hypothesis tested. A critical limitation arises when the true exotic signal deviates from the assumed training model, leading to a significant reduction in classifier sensitivity. Furthermore, fully supervised methods are ill-suited for scenarios where event-level labels are unavailable or the signal model is only partially specified.

Methodology
This work explores weakly supervised classification as a less model-dependent alternative. The core strategy employs a background-versus-mixture (BvM) setup:

• Training Data: Instead of labeled signal and background events, classifiers are trained on two samples with different proportions of the signal component but identical background distributions.
  • Sample $B$ (Background): Pure astrophysical background $p_A(x)$.
  • Sample $M$ (Mixture): A mixture $f p_E(x) + (1-f) p_A(x)$, where $f$ is the signal fraction and $p_E(x)$ is the exotic signal distribution.
• Theoretical Basis: Under the assumption that the background component is distributed identically in both samples, a classifier trained to distinguish $B$ from $M$ learns a score monotone in the optimal likelihood ratio $p_E(x)/p_A(x)$. This allows the identification of signal-like subsets without requiring individual event labels or prior knowledge of the signal fraction.
• Classifier Architecture: The authors utilize Boosted Decision Trees (BDTs) (specifically GradientBoostingClassifier from sklearn) as the default classifier. BDTs were chosen over neural networks for their stability, robustness to noisy/correlated inputs, and ease of optimization. The input features are low-level spectral fluxes in standard energy bands (7 bins for pulsar/AGN/DM, 50 bins for ALP searches) to maintain a model-agnostic baseline.
• Data Augmentation: To handle class imbalance and probe small signal fractions, the authors employ normalizing flows to generate synthetic background spectra (specifically for AGN) that match the statistical properties of observed sources.

Key Contributions and Applications
The paper validates this approach across three distinct physical scenarios, comparing weakly supervised results against fully supervised benchmarks:

1. Pulsars vs. Active Galactic Nuclei (AGN) – Controlled Benchmark:

   • Setup: Using the 4FGL catalog, pulsars are treated as the signal and AGN as the background.
   • Result: The weakly supervised approach successfully identifies pulsars. As the signal fraction in the mixture increases, the True Positive Rate (TPR) rises and approaches the performance of the fully supervised benchmark. The False Positive Rate (FPR) remains stable. This demonstrates that weak supervision can recover near-optimal performance when the signal and background are spectrally distinct.

2. Dark Matter Subhalos – Model-Dependent Challenge:

   • Setup: Simulated subhalo spectra (WIMP annihilation into $b\bar{b}$) are injected into a background of identified astrophysical sources.
   • Result: The problem is more challenging due to the spectral overlap between subhalos and astrophysical sources. While weak supervision can identify subhalo-like spectra without explicit signal labels, a performance gap remains compared to the fully supervised baseline. The TPR is lower, and the FPR is slightly higher, reflecting the conservative decision boundary inherent to the BvM setup at low signal fractions. However, the method avoids the need to retrain for every specific dark matter mass or annihilation channel hypothesis.

3. Axion-Like Particle (ALP) Oscillations – Spectral Irregularities:

   • Setup: The search targets subtle, oscillatory spectral deformations ("wiggles") in pulsar spectra induced by ALP-photon mixing in Galactic magnetic fields, rather than a new source population.
   • Result: The study simulates modulated spectra using the GammaALPs tool and the Jansson-Farrar Galactic magnetic field model. The authors compare supervised and weakly supervised strategies for detecting these modulations. The results highlight the difficulty of detecting subtle spectral features with high class imbalance (10% signal), where TPR remains low even in the supervised case. The weakly supervised approach is shown to be a viable strategy for ranking candidates based on spectral irregularities without scanning the full ALP parameter space ($m_a, g_{a\gamma}$).

Key Results

• Performance: In the "clean" benchmark (Pulsar vs. AGN), weak supervision approaches the performance of fully supervised classifiers. In more complex, overlapping scenarios (Dark Matter), it provides a robust, albeit slightly less sensitive, alternative that reduces reliance on specific signal templates.
• Signal Fraction Dependence: Classification performance (specifically TPR) generally improves as the signal fraction in the mixture increases, as the difference between the pure background and mixed samples becomes more pronounced.
• Robustness: The method successfully identifies anomalous subsets of data while reducing the reliance on detailed signal modeling during the training phase.

Significance and Claims
The authors position this work not as a replacement for dedicated likelihood-based searches or a direct tool for maximizing discovery significance in counting experiments. Instead, they claim that weakly supervised learning serves as a complementary candidate-selection and anomaly-ranking strategy.

• Model Agnosticism: It allows for the analysis of $\gamma$-ray data where the signal model is uncertain, only partially specified, or where event-level labels are unavailable.
• Prioritization: The method can prioritize interesting targets in large $\gamma$-ray catalogs (such as unassociated sources) for subsequent study via dedicated model-dependent analyses, multiwavelength observations, or follow-up measurements.
• Validation: By progressing from a well-controlled benchmark to increasingly subtle scenarios, the paper establishes the achievable behavior of weak supervision under controlled conditions, laying the groundwork for future studies addressing realistic selection effects and domain shifts.

The paper concludes that weakly supervised learning provides a flexible addition to the toolkit for $\gamma$-ray data analysis, bridging developments in machine learning and particle physics with astrophysical applications.