⚛️ high-energy experiments

Accelerating Optical Photon Simulation in DUNE with Opticks

This paper presents the first successful implementation and validation of GPU-accelerated optical photon simulation using Opticks for the 10-kiloton DUNE far-detector, demonstrating a speedup of up to 313 times over CPU-based GEANT4 while maintaining full Monte Carlo fidelity to enable high-statistics studies and machine learning dataset generation.

Original authors: Ilker Parmaksiz, Aaron Higuera, Laura Paulucci, Viktor Pec, Estanislao Forino

Published 2026-08-28

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Ilker Parmaksiz, Aaron Higuera, Laura Paulucci, Viktor Pec, Estanislao Forino

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Deep beneath the surface of the Earth, a massive experiment is being built to listen for the faintest whispers of the universe. The Deep Underground Neutrino Experiment, known as DUNE, aims to study neutrinos, ghostly particles that pass through almost everything without interacting. To catch these elusive particles, scientists are constructing enormous tanks filled with liquid argon, a substance kept at temperatures colder than outer space. When a neutrino strikes an atom inside this liquid, it creates a flash of light and a trail of electric charge. By tracking these signals with extreme precision, researchers hope to understand the fundamental nature of matter and the history of the cosmos. However, simulating how these flashes of light travel through the complex, multi-story tanks of the detector is a task so heavy that it threatens to overwhelm the world's most powerful computers.

The core difficulty lies in the sheer number of particles involved. When a neutrino interacts within the liquid argon, it does not just produce a single spark; it triggers a cascade that generates millions of tiny particles of light, called optical photons. In a real detector, each of these photons must be tracked individually as it bounces off walls, scatters through the liquid, and eventually hits a sensor. For a detector as large as the one planned for DUNE, which holds ten thousand tons of liquid argon, tracking every single photon one by one using standard computer methods takes an impractical amount of time. It is a computational bottleneck that has forced scientists to use shortcuts, which often sacrifice the detailed, step-by-step truth of how the light actually moves.

A team of researchers has now found a way to break this bottleneck by harnessing the power of graphics processing units, the specialized chips found in high-end computers that are designed to render complex images. They developed a new system called Opticks, which acts as a bridge between the standard simulation software used by physicists and the parallel processing power of these graphics chips. In a recent study, the team applied this system to the full-scale geometry of DUNE's horizontal drift detector. They demonstrated that by moving the simulation of light particles onto a graphics card, they could track the same millions of photons that a standard computer processor could, but in a fraction of the time.

The results of this work are striking. When the researchers compared their new GPU-accelerated method against the traditional single-threaded computer simulation, they found that the new system was more than three hundred times faster. Even when compared to a standard computer running four processing threads simultaneously, the new method remained more than eighty times faster. This speedup was not achieved by simplifying the physics or guessing where the light would go. Instead, the system followed every single photon through the detector, accounting for how it scatters, reflects, and changes color, just as the slower, traditional methods do. The researchers verified this by running thousands of simulations and checking that the patterns of light hitting the sensors matched the trusted, slower models perfectly.

Beyond raw speed, this advancement changes what is possible for the experiment. Because the simulation is now fast enough to run on a single graphics card, scientists can generate vast amounts of realistic data that includes the complete history of every photon. This level of detail is crucial for training artificial intelligence systems to recognize patterns in the detector, a task that previously lacked sufficient high-quality training data. The team successfully integrated this new tool into the main software framework used by the DUNE experiment, meaning that future simulations can now include the full, detailed journey of light without waiting months for results. This opens the door to more precise measurements of neutrino properties and a deeper understanding of the universe, all by teaching the computer to calculate light the way a graphics card renders a movie.

Technical Summary: Accelerating Optical Photon Simulation in DUNE with Opticks

Problem Statement
In high-energy physics experiments like the Deep Underground Neutrino Experiment (DUNE), detector simulation is a primary computational bottleneck, often consuming 40% to 80% of total CPU resources. Within this workflow, optical photon simulation is the most demanding task. DUNE's far-detector modules (10 kt scale) involve cryostats of approximately 14 × 12 × 58 m³, where a single neutrino interaction can generate $\sim 10^7$ optical photons. Transporting these billions of photons step-by-step through complex geometries using single-threaded GEANT4 on CPUs is prohibitively expensive.

Current alternatives, such as lookup-tables ("photon libraries") or semi-analytical models, bypass the underlying particle transport to achieve speed. However, these methods rely on spatially averaged probabilities and cannot provide "truth-level" information regarding the stochastic fluctuations of Rayleigh scattering, bulk absorption, and boundary reflections. Consequently, they fail to reproduce localized optical effects, smear correlated timing structures, and obscure the exact trajectory and micro-interaction history of individual photons, which is essential for low-level reconstruction, calibration, and machine learning (ML) training.

Methodology
The authors present the implementation and evaluation of Opticks, an open-source API that integrates GEANT4 with the NVIDIA OptiX ray-tracing engine, to accelerate optical photon transport on GPUs. The work focuses on the DUNE far-detector horizontal-drift (FD-HD) geometry.

Key methodological components include:

RiceOpticks: A fork of Opticks developed specifically for this work to handle large-scale simulations. It features accelerated geometry import, global photon-sensor identifier (SID) mapping to the GPU, a custom scintillation collection function independent of standard GEANT4 stepping actions, and the propagation of parent-particle identifiers to preserve truth-level associations.
Dual Workflows:
1. Stand-alone Simulation: A framework comparing GEANT4 and Opticks directly using GDML geometry. A custom primary photon generator ensures identical kinematics for both CPU and GPU to enable direct photon-by-photon validation.
2. LArSoft Integration: The development of the PDFullSimOpticks module, which interfaces Opticks with the LArSoft software stack. This module captures scintillation data from the IonAndScint module and transfers it to the GPU for propagation, returning optical hits as OpDetBacktrackerRecord objects.
Physics Implementation: The simulation includes liquid argon absorption lengths, Rayleigh scattering, and metal reflections. While the current study focuses on photon propagation to the X-ARAPUCA sensor surface, the framework is designed to eventually incorporate wavelength-shifting (WLS) physics, with preliminary validation already performed in simplified geometries.

Key Contributions

First 10 kt-Scale Implementation: This is the first demonstration of GPU-accelerated, full-fidelity optical photon transport at the scale of a 10 kt detector module.
Full Fidelity at Low Cost: The approach maintains complete Monte Carlo fidelity, propagating every photon individually with GEANT4-equivalent physics, unlike fast approximations.
LArSoft Integration: The successful integration of Opticks into the DUNE software stack via PDFullSimOpticks, making GPU-accelerated transport available to any experiment using LArSoft.
Truth-Level Preservation: The system retains the ability to backtrack every detected photon to its specific energy deposit and parent particle, a capability lost in surrogate or parameterized models.

Results

Performance: On the full FD-HD geometry, Opticks achieved a speedup of 313 ± 3 compared to single-threaded GEANT4 and 83 ± 1 compared to four-threaded GEANT4.
Validation: Using a reduced FD-HD geometry, the authors compared visibility distributions, detected photon counts per sensor, propagation times, and 2D spatial hit profiles. Opticks results showed close agreement with the reference GEANT4 simulation across all metrics, confirming high-fidelity reproduction of physics and geometry.
Resource Efficiency: GPU runs utilized 5.8 GB of VRAM (18% of total) and reached 100% compute utilization. The remaining VRAM allows for concurrent execution of multiple events.
Comparison to Semi-Analytical Methods: In tests with representative physics interactions (e.g., CC $\nu_e$ , $\nu_\mu$ , supernova neutrinos, proton decay), the full Opticks simulation achieved processing times comparable to the current semi-analytical default in DUNE. As photon counts increase, Opticks becomes more efficient than the semi-analytical approach.

Significance
The paper claims that this work establishes a new paradigm for optical simulation in kiloton-scale detectors. By combining full fidelity with tractable computational costs, it makes previously infeasible studies possible, including:

High-statistics optimization of photon-detector layouts.
Data-driven tuning of optical models.
Systematic uncertainty quantification for light-based calorimetry.
Trigger studies for low-energy signals (e.g., supernova bursts).

Crucially, the ability to generate large volumes of high-fidelity, fully labeled optical data at GPU speed addresses a major bottleneck for machine learning in this domain. It enables the creation of training datasets of the required scale for supervised learning and simulation-based inference, which were previously unattainable due to the lack of truth information in fast methods or the prohibitive cost of full CPU simulations.

More like this