Application of a Mixture of Experts-based Foundation Model to the GlueX DIRC Detector

This paper presents a Mixture-of-Experts-based foundation model that unifies fast simulation, particle identification, and noise filtering for the GlueX DIRC detector by leveraging a single shared transformer backbone to outperform or match established task-specific methods across the full kinematic phase space.

The Gist

Imagine you are trying to understand a complex orchestra playing a piece of music. In the world of particle physics, the "music" is the pattern of light created when a charged particle zips through a special detector called the GlueX DIRC. This detector is like a giant, mirrored hallway made of glass bars. When a particle (like a pion or a kaon) runs through it, it leaves behind a trail of light flashes, or "hits," on a grid of sensors.

For a long time, scientists have used three different, separate "tools" to make sense of this light:

1. The Map Reader: A geometric method that tries to calculate the particle's identity based on strict rules of physics (like a GPS calculating a route). It works well at low speeds but gets confused when particles move very fast.
2. The Specialist Musicians: Different AI models trained to do just one job: one to identify the particle, another to simulate how the light should look, and a third to filter out static noise.
3. The Slow Simulator: A super-accurate but incredibly slow computer program (Geant4) that simulates every single photon. It's like watching a movie in slow motion to get the details right, but it takes too long to run for real experiments.

The New "Universal Translator"

This paper introduces a new approach: a Mixture-of-Experts-based Foundation Model. Think of this as a single, super-smart "Universal Translator" that can do all three jobs at once, using one shared brain (a Transformer backbone) instead of three separate specialists.

Here is how it works, using simple analogies:

1. The "Mixture of Experts" (The Team of Specialists)

Imagine a classroom where the teacher (the main AI) has a team of four teaching assistants (the "Experts").

• Two assistants are experts at understanding Pions.
• Two assistants are experts at understanding Kaons.
• When a new student (a particle) walks in, the teacher quickly decides which assistants should handle the lesson. This allows the model to be highly specialized for each particle type while still sharing the same general knowledge base.

2. Learning the "Language" of Light

The model doesn't just look at the picture; it learns the "language" of the detector.

• Tokenization: It breaks the light hits down into words. It treats the where (spatial location) and the when (time of arrival) as two separate sentences that it reads simultaneously.
• Autoregressive Generation: It predicts the next "word" (hit) in the sequence, just like a text-prediction app guesses the next word you will type. By doing this, it learns the natural rhythm and pattern of how light behaves in the detector.

3. The Three Superpowers

Because this model learned the fundamental "language" of the detector, it can perform three distinct tasks without needing to be rebuilt:

• Particle Identification (Who is it?):
  The model looks at the pattern of light and guesses if it was a pion or a kaon.

  • The Result: It got a score of 0.952 out of 1.0, beating the old geometric map reader (0.871) and other AI models. It is especially good at telling fast-moving particles apart, a job where the old methods usually fail.

• Fast Simulation (What would it look like?):
  Instead of running the slow, heavy "movie" simulation, this model can instantly "imagine" what the light pattern should look like for a specific particle.

  • The Result: It creates images that look almost identical to the real, slow simulations. Interestingly, it also learned to guess how many photons would appear (the "yield") automatically, without needing a separate calculator. It's like an artist who can paint a scene and naturally get the lighting intensity right without measuring it first.

• Noise Filtering (What is static?):
  Detectors often have "static" or random noise (like the hiss on an old radio). This model can look at every single light hit and decide, "This is a real signal" or "This is just noise."

  • The Result: It is incredibly accurate, filtering out noise while keeping the real signal almost perfectly intact (AUC of 0.971).

The One Catch: The Library Size

The paper notes one limitation. The model is like a student who has read a very large library of books, but the library isn't infinite.

• Because the training data (the "books") wasn't large enough to cover every single possible scenario, the model is slightly less accurate when dealing with the most complex, high-speed particles.
• The authors suggest that if they feed the model more data (a bigger library), it will get even better. The current performance is a "lower bound," meaning it will likely improve as more data becomes available.

The Bottom Line

This paper shows that instead of building a different tool for every job in particle physics, we can build one powerful foundation model that learns the deep rules of the detector. Once it learns those rules, it can identify particles, simulate experiments, and clean up noise all at once, doing it faster and more accurately than the old, fragmented methods. It turns a collection of specialized tools into a single, versatile Swiss Army knife for physics.

Technical Summary

1. Problem Statement

The GlueX experiment at Jefferson Lab utilizes a Detection of Internally Reflected Cherenkov (DIRC) detector for charged hadron identification (specifically distinguishing pions from kaons). The analysis of DIRC data faces three primary challenges that traditional methods struggle to address simultaneously:

• Fragmentation: Current workflows rely on separate, specialized models for different tasks: geometrical reconstruction (Look-Up Tables) for Particle Identification (PID), Geant4 simulations for high-fidelity but computationally expensive simulation, and separate noise filtering algorithms. This creates high training overhead and deployment complexity.
• Performance Degradation: Traditional geometrical reconstruction methods degrade significantly at high momenta ($>3$ GeV/c) because the Cherenkov angles of pions and kaons converge, making discrimination difficult.
• Simulation Cost: Full Geant4 tracking of Cherenkov photons is computationally prohibitive for large-scale analyses, necessitating "fast simulation" surrogates that often lack fidelity or require auxiliary models to reproduce photon yields.

2. Methodology

The authors propose a Mixture-of-Experts (MoE) based Foundation Model (FM) that unifies fast simulation, PID, and noise filtering into a single architecture. The model is adapted from a previous framework developed for the EIC hpDIRC [10] and applied directly to GlueX without architectural changes.

• Architecture & Tokenization:

  • Input: The model operates on low-level detector inputs: 3D hit coordinates $(x, y, t)$ on the Photomultiplier Tube (PMT) plane.
  • Tokenization: Spatial coordinates are mapped to discrete pixel indices (vocabulary size ~5670). Time is discretized into bins (0.06 ns width) to create a temporal vocabulary.
  • Conditioning: Kinematic parameters (momentum $|p|$, polar angle $\theta$, azimuthal angle $\phi$) are projected and prepended as contextual tokens to both spatial and temporal sequences.
  • Core Mechanism: The model uses a Causal Multi-Head Cross-Attention (CMHCA) block to fuse spatio-temporal information. It queries spatial locations based on time embeddings, encoding the physical intuition that photon arrival times depend on geometric path lengths.
  • Mixture of Experts (MoE): To handle class-conditional generation (pions vs. kaons), the model employs 4 experts (2 per particle type) with fixed routing. An auxiliary loss ensures balanced load balancing.

• Task-Specific Heads:

  • Generation: Autoregressive next-token prediction over spatial and temporal vocabularies (Cross-Entropy loss).
  • Particle Identification (PID): A CLS token is added; the final latent representation is passed through a linear classifier (Binary Cross-Entropy loss).
  • Noise Filtering: Per-token latent representations are classified as signal or noise (Focal loss to handle class imbalance).

• Training Strategy:

  • Pre-training: The model is first trained autoregressively on Geant4 data to learn the underlying detector response distribution.
  • Fine-tuning: For PID, the pre-trained weights are fine-tuned. For noise filtering, the model is trained from scratch (random initialization) as fine-tuning provided no benefit in previous studies.
  • Data Augmentation: To prevent overfitting on the limited dataset (~700k samples per class), the authors applied spatial jittering (within PMT boundaries) and time smearing ($\pm 1$ ns).

3. Key Contributions

• Unified Framework: Demonstrates that a single transformer backbone can effectively replace the suite of task-specific models currently used for GlueX DIRC analysis.
• Implicit Photon Yield Learning: Unlike previous fast simulation methods that require auxiliary networks to reproduce photon yields, this foundation model learns the yield implicitly as part of the autoregressive generation process.
• Transferability: Proves that a foundation model trained on one detector type (EIC hpDIRC) can be successfully transferred to a different detector (GlueX DIRC) with minimal modification, maintaining high performance across all tasks.
• MoE for Physics: Successfully applies the MoE architecture to physics data, allowing for class-conditional specialization (pion vs. kaon) while sharing a common latent representation.

4. Results

• Particle Identification (PID):

  • The foundation model achieved an AUC of 0.952, outperforming the Swin Transformer (0.932), the NF-DLL method (0.933), and the standard geometrical reconstruction baseline (0.871).
  • The performance gap was most significant at high momenta ($>3$ GeV/c), where traditional methods fail due to Cherenkov angle convergence.
  • Fine-tuning from the pre-trained generative model provided a consistent ~2% AUC improvement over training from scratch.

• Fast Simulation (Generative Fidelity):

  • Visual inspection confirmed the model faithfully reproduces spatial hit patterns and the characteristic double-peaked Cherenkov timing structure.
  • Photon Yield: The generated photon yield closely matched Geant4 ground truth across all 48 bars for both pions and kaons without auxiliary modeling.
  • Fidelity Validation: When a classifier was trained on the fast-simulated data and tested on Geant4 data, it achieved an AUC of 0.904 (vs. 0.935 for Geant4-trained). The ~3% degradation was attributed to data sparsity in high-momentum regions rather than architectural flaws, suggesting the current results are a lower bound on achievable fidelity.

• Noise Filtering:

  • The model achieved near-ideal discrimination between signal and noise at the hit level, with an AUC of 0.971 and an Average Precision (AP) of ~0.869.
  • Performance was robust across both particle species and integrated over the full phase space.

5. Significance and Conclusion

This work establishes the Foundation Model as a practical, scalable, and high-fidelity alternative to traditional analysis pipelines in experimental nuclear physics.

• Efficiency: It eliminates the fragmentation of task-specific pipelines, reducing deployment complexity.
• Scalability: The results suggest that as pre-training datasets grow larger and more diverse, the generative fidelity will improve, potentially closing the gap between fast simulation and full Geant4 simulations.
• Paradigm Shift: The study reinforces the emerging paradigm that a single, well-trained model can serve as a general-purpose representation of detector data, capable of supporting diverse downstream tasks through fine-tuning. This offers a path toward more maintainable and high-performance workflows for the GlueX experiment and future detector systems.