⚛️ high-energy experiments

Cross-Geometry Transfer Learning in Fast Electromagnetic Shower Simulation

This paper introduces a transfer learning methodology for fast electromagnetic shower simulation that leverages point cloud representations and pre-training on the International Large Detector to efficiently adapt generative models to new detector geometries with minimal target-domain data, significantly outperforming training from scratch while updating only a fraction of the model parameters.

Original authors: Frank Gaede, Gregor Kasieczka, Lorenzo Valente

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Frank Gaede, Gregor Kasieczka, Lorenzo Valente

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

High-energy physics is the science of smashing particles together at nearly the speed of light to see what they are made of and how they behave. When these particles collide, they do not just bounce off; they explode into cascades of new particles, creating showers that ripple through massive detectors. To understand these collisions, scientists must simulate exactly how these showers develop. For decades, they have relied on a rigorous method called Monte Carlo simulation, which tracks every single particle interaction step-by-step as it moves through layers of metal and silicon. While this method is incredibly accurate, it is also agonizingly slow. A single simulated event can take minutes of computer time, and with the next generation of particle colliders producing data at unprecedented volumes, the current computing power simply cannot keep up. The bottleneck is not a lack of faster computers, but the fundamental complexity of the calculations themselves.

To solve this, researchers have turned to artificial intelligence, training computer models to predict the final result of a particle shower without simulating every intermediate step. These "fast simulation" models act as surrogates, learning the patterns of particle behavior so they can generate results in a fraction of a second. However, a major hurdle has emerged: these AI models are usually tied to the specific shape and layout of the detector they were trained on. If a scientist changes the detector's geometry or builds a new one, the old model becomes useless, and the expensive training process must start all over again. This limitation threatens to stall progress, as the field moves toward more complex and varied detector designs.

In a recent study published in the Journal of Instrumentation, a team of physicists led by Frank Gaede, Gregor Kasieczka, and Lorenzo Valente tackled this problem by teaching an AI model to adapt to new detector shapes without starting from zero. They focused on electromagnetic showers, the cascades of particles created when high-energy photons or electrons hit a detector. The researchers used a technique called transfer learning, which is akin to a student who has mastered a subject in one context and then applies that deep understanding to a slightly different context with very little new study. They took a model that had been extensively trained on data from the International Large Detector, a specific design with flat, rectangular layers, and attempted to adapt it to a completely different geometry: a cylindrical detector with curved, radial layers, similar to those used in the CaloChallenge benchmark.

The challenge was significant. The two detectors differed not just in shape, but in the number of layers, the size of the cells, and the range of particle energies they could measure. The original model was trained on photons with energies between 10 and 90 billion electron volts, while the new target involved electrons ranging from 1 to 1,000 billion electron volts. Furthermore, the new detector had 45 layers compared to the original 30. Instead of retraining the entire model from scratch, which would require massive amounts of new data and computing power, the team tried to "fine-tune" the existing model. They tested whether the physics knowledge the model had already learned about how particles spread and lose energy could be transferred to this new, curved environment.

The results were striking, particularly when data was scarce. In scenarios where the team had only 100 examples of the new detector's particle showers to work with, the adapted model performed 51 percent better than a model trained from scratch on those same 100 examples. This improvement was measured by how closely the simulated particle showers matched the real physical data generated by the slow, traditional simulation methods. The team found that the model successfully learned to adjust its internal understanding of particle behavior to fit the new cylindrical shape, preserving the fundamental laws of physics while accommodating the new geometry.

To make this adaptation even more efficient, the researchers explored methods that update only a small fraction of the model's internal settings, rather than changing everything. One such method, known as BitFit, adjusted only the bias terms—essentially the baseline thresholds that determine how neurons in the network respond—while leaving the rest of the model frozen. This approach changed only 17 percent of the model's parameters but achieved performance nearly identical to fully retraining the entire network. Another method, which updated only the final layers of the model, also proved highly effective. In contrast, a popular technique called Low-Rank Adaptation, which is often successful in other fields like language processing, struggled here. The researchers discovered that the complex transformations required to simulate particle showers are too intricate to be compressed into the simple, low-dimensional adjustments that this technique relies on.

The study also addressed a practical hurdle: the mismatch in the number of layers between the old and new detectors. The team developed a clever workaround where the model used its pre-trained knowledge for the original 30 layers and learned to generate the additional 15 layers on the fly, using a small amount of new data to guide the process. This allowed the system to handle detectors of arbitrary sizes without needing a complete architectural overhaul. The entire process demonstrated that a single, well-trained model could serve as a flexible foundation for many different detector designs, provided the right adaptation strategies were used.

By proving that transfer learning works for particle showers, the researchers have opened a path toward a more sustainable future for high-energy physics simulations. Instead of training a new model for every new detector design, scientists can now start with a pre-trained model and adapt it quickly and cheaply. This is especially crucial for the next decade of experiments, where detector designs are evolving rapidly and computing resources are at a premium. The work suggests that the key to faster simulation lies not just in building bigger models, but in teaching them to be flexible, allowing them to carry their knowledge of particle physics across the changing landscapes of future experiments.

Technical Summary: Cross-Geometry Transfer Learning in Fast Electromagnetic Shower Simulation

Problem Statement
Accurate particle shower simulation remains a critical computational bottleneck for high-energy physics (HEP), particularly for future experiments like the High-Luminosity Large Hadron Collider (HL-LHC). Traditional Monte Carlo methods (e.g., Geant4) are computationally prohibitive, often requiring minutes of CPU time per event. While machine learning (ML) surrogates have emerged as fast alternatives, existing generative models are typically tied to specific detector geometries. When detector designs change or new configurations are proposed, these models require complete retraining with new simulation datasets, negating the efficiency gains of transfer learning. Furthermore, while point cloud representations offer geometric flexibility, it remains unclear whether representations learned on a single detector can generalize to fundamentally different geometries and particle types without re-voxelizing showers for each design.

Methodology
The authors propose a transfer learning framework for generative calorimeter simulation models that adapts pre-trained weights to new detector geometries with high data efficiency. The approach utilizes the CaloClouds architecture, which consists of two complementary models:

ShowerFlow: A normalizing flow model that predicts the number of points (occupancy) per calorimeter layer based on incident particle energy.
PointWise Net: A diffusion model (following the EDM framework) that generates the spatial coordinates ( $x, y, z$ ) and energy depositions of shower hits as continuous point clouds.

The methodology involves pre-training on a large dataset of photon-induced showers in the International Large Detector (ILD) geometry (planar, 30 layers, 10–90 GeV). This pre-trained model is then adapted to a downstream task involving electron showers in the CaloChallenge Dataset 3 geometry (cylindrical, 45 layers, 1–1000 GeV).

Key technical adaptations include:

Handling Dimensional Mismatch: Since the target geometry has 45 layers compared to the pre-trained 30, the ShowerFlow model is extended. The original 30 layers utilize pre-trained weights, while the additional 15 layers are initialized from log-normal distributions estimated from a small target-domain sample and then learned during fine-tuning.
Preprocessing: The cylindrical CaloChallenge data is preprocessed via "cylindrical smearing" to convert voxelized deposits into continuous point clouds, aligning with the point cloud representation used in pre-training.
Training Strategies: The study compares training from scratch (random initialization) against full fine-tuning (updating all parameters) and Parameter-Efficient Fine-Tuning (PEFT) strategies, specifically BitFit (updating only bias terms), Top2 (updating the final two layers and time-step layer), and LoRA (Low-Rank Adaptation).

Key Contributions

Single-Detector Pre-Training for Cross-Geometry Transfer: The paper demonstrates that pre-training on a single, well-characterized detector (ILD) is sufficient to enable effective adaptation to a significantly different detector geometry (cylindrical CaloChallenge) and particle type (photons to electrons), provided the underlying physics (electromagnetic cascades) remains similar.
Data Efficiency: The study quantifies the benefits of transfer learning in low-data regimes. Using only 100 target-domain samples, the transfer learning approach achieves a 51% improvement in the geometric mean of the Wasserstein distance compared to training from scratch.
Evaluation of PEFT in Physics Simulation: This work presents the first application of PEFT methods (BitFit, Top2, LoRA) to pre-trained models for fast particle shower simulations. It identifies that BitFit (updating only 17% of parameters) achieves performance within 5% of full fine-tuning, offering a highly efficient adaptation pipeline.
Analysis of LoRA Limitations: Through post-hoc singular value decomposition (SVD) of weight updates, the authors provide theoretical insight into why LoRA underperforms in this domain. They find that the intrinsic dimensionality of particle shower transformations is significantly higher than in typical NLP tasks, requiring ranks (e.g., >100) that exceed standard LoRA configurations and negating its compression benefits.

Results

Performance Gains: Full fine-tuning consistently outperforms training from scratch, particularly when training data is limited ( $<10^3$ samples). At $10^2$ samples, the geometric mean Wasserstein distance drops from $0.132$ (from scratch) to $0.064$ (fine-tuned).
PEFT Effectiveness: BitFit emerges as the most efficient strategy, updating only bias terms (17% of parameters) while maintaining performance comparable to full fine-tuning. Top2 fine-tuning (44% of parameters) also performs competitively.
LoRA Struggles: Despite utilizing 52% of parameters, LoRA (even at rank 106) shows degraded performance, particularly in the voxel energy spectrum. SVD analysis reveals that weight updates for shower physics require high ranks (up to 106 for internal layers) to capture 95% of the energy, suggesting the low-rank assumption is ill-suited for the high-dimensional nature of these physics transformations.
Computational Efficiency: BitFit reduces per-step training time by 21% and peak memory by 23% compared to full fine-tuning. While full fine-tuning converges faster (100k steps) than training from scratch (500k steps), the combination of BitFit's lower per-step cost and reasonable convergence steps offers a highly efficient total training time.

Significance and Claims
The paper claims that this work establishes a baseline for future progress in point cloud approaches for calorimeter simulation. It argues that representation flexibility (point clouds) combined with model transferability (pre-training) is a viable path for generalizable simulation models, even when only a single detector dataset is available for pre-training.

The authors modestly note that while their results are promising, they do not claim superiority over multi-detector pre-training approaches (which were not directly compared). They emphasize that the success of BitFit and selective layer fine-tuning suggests that adaptation in this domain primarily involves targeted recalibration (adjusting activation thresholds and refining high-level features) rather than fundamental changes to the learned physics representations. The work highlights that standard PEFT techniques like LoRA may require physics-aware modifications due to the high intrinsic dimensionality of particle shower data, motivating the development of specialized adaptation strategies for scientific applications.

More like this