GPU-Accelerated Analytic Simulation of Sparse Signals in Pixelated Time Projection Detector

This paper introduces TRED, a sustainable and extensible GPU-accelerated simulation package for pixelated time projection detectors that utilizes Gaussian quadrature for effective-charge calculation and a sparse block-binned tensor representation to enable efficient, low-memory FFT-based signal computation for sparse detector volumes.

The Gist

Imagine you are trying to take a photograph of a massive, dark warehouse (a particle detector) using a camera with millions of tiny sensors (pixels). Inside this warehouse, particles are zipping around, leaving behind faint trails of glowing dust (ionization). Your goal is to figure out exactly where those particles went and how much energy they had, just by looking at the patterns of light hitting the sensors.

The problem? The warehouse is huge, the sensors are incredibly numerous, and the glowing dust is incredibly sparse. Most of the time, the sensors see nothing but darkness. Only a few tiny spots light up.

If you tried to process this data using a standard computer (a CPU), it would be like trying to count every single grain of sand on a beach, even though you only care about the few grains that are glowing. It would take forever and run out of memory.

This paper introduces TRED, a new, super-fast software tool designed to run on powerful graphics cards (GPUs) to solve this problem. Here is how it works, explained through simple analogies:

1. The "Smart Sketch" vs. The "High-Res Photo" (Effective Charge)

Usually, to simulate how a particle moves, computers try to take a "high-resolution photo" of every single millimeter of space. This creates a massive amount of data.

The authors came up with a clever trick called Effective Charge. Imagine you are an artist trying to paint a cloud. Instead of painting every single water droplet (which is impossible), you use a "smart sketch." You calculate the average weight and shape of the cloud in a specific area using a mathematical shortcut (Gaussian quadrature).

• The Analogy: Instead of counting every single raindrop falling on a roof to know how much water hit a specific tile, you just measure the "splash zone" and calculate the total water based on a few smart sampling points.
• The Result: The computer doesn't need to store millions of data points. It just stores a few "smart summaries" that are mathematically perfect enough to recreate the exact signal later.

2. The "Empty Warehouse" Strategy (Sparse Data)

In a typical computer simulation, the software creates a giant 3D grid representing the whole detector, filling every single cell with data, even if 99% of it is empty. This is like filling a massive warehouse with boxes, even if only 10 boxes actually contain anything. It wastes space and time.

TRED uses a Block-Sparse Binned Tensor.

• The Analogy: Imagine you are a librarian. Instead of walking down every single aisle of a library to check if a book is there, you only check the specific shelves where people actually dropped off books. You ignore the empty aisles entirely.
• How it works: TRED only looks at the "blocks" of the detector where activity is happening. If a section of the detector is silent, TRED ignores it completely. This saves a massive amount of computer memory and makes the simulation incredibly fast.

3. The "Magic Echo" (FFT and Convolution)

When a particle moves, it creates a signal that ripples out to the sensors, kind of like a stone thrown in a pond creating waves. To figure out what the sensors see, you have to calculate how these waves overlap. Doing this for millions of sensors is usually a math nightmare.

The authors use a technique called FFT (Fast Fourier Transform) combined with a clever trick called Mirror-Pair Complex Packing.

• The Analogy: Imagine you are in a canyon shouting. To hear your echo, you usually have to wait for the sound to bounce back. But if you know the shape of the canyon perfectly, you can use a "magic echo machine" (the FFT) to predict the echo instantly without waiting.
• The Trick: Because the detector is symmetrical (it looks the same from the left as it does from the right), the software can pack two different calculations into one "complex number" (like putting two different songs into one stereo track). This cuts the work in half, making the simulation twice as fast.

4. The "Smart Traffic Manager" (GPU Management)

GPUs are like a fleet of thousands of tiny workers. If you give them all the same amount of work, they are efficient. But in a particle detector, some areas are busy (traffic jams) and some are empty. If you try to give everyone the same amount of work, the workers in the empty areas just sit idle.

TRED uses Hierarchical Batching.

• The Analogy: Imagine a bus driver who doesn't just drive a fixed route. Instead, they look at the passengers. If a bus stop has 50 people, they send a big bus. If a stop has 2 people, they send a small van. They dynamically adjust the size of the "bus" (the data chunk) based on how many people (particles) are actually there. This ensures no worker is ever sitting idle, and no memory is wasted.

Why Does This Matter?

The DUNE experiment (Deep Underground Neutrino Experiment) is building a massive detector to study neutrinos (ghostly particles that pass through everything). This detector will be so big and produce so much data that old computers simply can't keep up.

• The Impact: TRED allows scientists to simulate these massive detectors in a fraction of the time it used to take. It's like upgrading from a bicycle to a supersonic jet for data processing.
• The Future: Because this software is built on modern, open tools (like PyTorch, which is also used for AI), it's easy to update and can be used for other types of detectors or even for training AI models to recognize particle patterns.

In short: The authors built a super-efficient, "smart" simulation tool that ignores the empty space, uses mathematical shortcuts to avoid counting every single grain of dust, and runs on powerful graphics cards to solve the "needle in a haystack" problem of particle physics.

Technical Summary

1. Problem Statement

The Deep Underground Neutrino Experiment (DUNE) Near Detector (ND-LAr) utilizes a Liquid Argon Time Projection Chamber (LArTPC) with a pixelated charge readout. This design features approximately $5 \times 10^5$ readout channels per module.

• Scalability Challenge: Traditional CPU-based simulation frameworks struggle to scale to the massive channel counts and high interaction rates ($O(10^2)$ interactions per beam spill) of the ND-LAr.
• Sparsity vs. Density: While the detector volume is large, the ionization signals from neutrino interactions are highly sparse and localized in space and time. Standard GPU architectures are optimized for dense, regular workloads, making them inefficient for processing sparse data without excessive memory usage or computational waste.
• Memory Bottleneck: Simulating induced signals on every pixel for every time step creates a prohibitive memory footprint, particularly when accounting for electron diffusion and the convolution of charge distributions with detector response functions.

2. Methodology

The authors developed TRED, a GPU-native simulation package built on the PyTorch ecosystem to leverage community-driven software sustainability and high-performance GPU acceleration. The methodology relies on three core technical pillars:

A. Analytic Effective-Charge Formulation

Instead of dense sampling, the paper introduces an analytic method to calculate the "effective charge" ($Q_{eff}$) on a grid:

• Gaussian Quadrature: The continuous induced current integral is discretized using Gauss-Legendre quadrature rules. This allows the simulation to capture sub-grid charge structures without requiring a dense field-response grid.
• Line vs. Point Source: The framework handles ionization tracks as line segments (convolved with a 3D Gaussian diffusion kernel) or point sources, switching formulations based on segment length relative to diffusion width to ensure numerical stability.
• Trilinear Interpolation: The field response function (Green's function) is evaluated on a coarse grid, and values at quadrature nodes are obtained via trilinear interpolation from neighboring grid points.

B. Block-Sparse Binned Tensor Representation

To handle the sparsity of the data efficiently:

• Block-Sparse Structure: The detector volume is partitioned into $N$-dimensional "blocks" (binned tensors). Only blocks containing active charge are retained, drastically reducing memory usage.
• Locality Preservation: This structure preserves spatial locality, allowing for efficient batched execution on GPUs.
• Dynamic Batching: The system employs "hierarchical batching" and "sub-dimensional chunking." It dynamically adjusts batch sizes based on realized sparsity and partitions tensors along orthogonal axes (e.g., drift direction) to bound peak memory consumption without sacrificing throughput.

C. Interlaced Convolution via FFT

The induced signal is modeled as a convolution of the effective charge distribution with the detector's field response function.

• Interlaced Convolution: The authors reformulate the signal calculation as a cross-correlation in the pixel plane and a convolution in time.
• Mirror-Pair Complex Packing (MPCP): To accelerate the convolution, they exploit the reflection symmetry of the field response. By packing pairs of channels into complex arrays, the number of required Fast Fourier Transforms (FFTs) is halved.
• FFT-Based Computation: The convolution is evaluated in the frequency domain, avoiding the construction of a global dense grid.

3. Key Contributions

1. TRED Package: A GPU-accelerated, PyTorch-based simulation framework tailored for pixelated LArTPCs, ensuring long-term maintainability and extensibility.
2. Effective-Charge Calculation: A novel numerical integration scheme using Gaussian quadrature that captures sub-grid physics without dense sampling.
3. Block-Sparse Binned Tensor: A generic data structure that enables efficient FFT-based processing of sparse signals, applicable to large-scale detectors and sparse sub-manifold convolution networks in machine learning.
4. MPCP Optimization: A specific algorithmic optimization that reduces FFT operations by exploiting symmetry in the field response function.

4. Results and Performance

The framework was benchmarked using DUNE ND-LAr simulation data (GENIE + edep-sim) on an NVIDIA GeForce RTX 4090 GPU.

• Accuracy: The effective-charge approach (using 2-point quadrature) showed deviations in accumulated charge well below the typical front-end electronic noise level ($O(500 e^-)$) compared to a 4-point quadrature reference.
• Memory Efficiency:
  • Hierarchical batching and chunking strategies successfully managed memory spikes.
  • Chunking reduced peak GPU memory usage by a factor of 2–5 compared to unchunked processing.
  • Memory usage scales linearly with activity for small batches but converges to an asymptotic limit determined by the output waveform size for large batches.
• Runtime:
  • Runtime scales linearly with input size once GPU resources are fully utilized.
  • Convolution is the primary performance bottleneck, followed by block-sparse binning.
  • Bin Size Optimization: An optimal bin size (approx. 50 time ticks) balances the cost of sparse binning against downstream convolution costs. Bins larger than the signal extent degrade performance due to processing zero-filled regions.
• Differentiable Simulation: The framework successfully demonstrated end-to-end differentiable simulation, automatically tuning the electron lifetime parameter to match ground truth, proving its utility for calibration workflows.

5. Significance

• Scalability for DUNE: TRED provides a viable path to simulate the massive channel counts of the DUNE Near Detector, which is currently intractable with traditional CPU methods.
• Generalizability: The underlying concepts (block-sparse tensors and analytic effective charge) are not limited to DUNE. They are applicable to the DUNE Far Detector (which requires long readout windows for supernova bursts) and other large-volume detectors with sparse activity.
• Computational Paradigm Shift: By integrating machine learning infrastructure (PyTorch) with high-energy physics simulation, the paper demonstrates a sustainable, extensible approach to detector simulation. It bridges the gap between sparse data processing in physics and efficient tensor operations in AI, offering a blueprint for future large-scale scientific computing.
• Calibration Potential: The differentiable nature of the framework opens new avenues for automated, simultaneous parameter optimization in detector calibration, moving beyond traditional sequential tuning methods.