🔬 physics

Cross-geometry transfer and model collapse in point cloud calorimeter shower generation

This paper demonstrates that the CaloClouds II generative model can effectively transfer knowledge across different detector geometries with minimal fine-tuning to accelerate particle shower simulation, while also investigating the risks of model collapse when training on its own generated data.

Original authors: Thorsten Buss, Frank Gaede, Gregor Kasieczka, Lorenzo Valente, Duncan Weber

Published 2026-09-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Thorsten Buss, Frank Gaede, Gregor Kasieczka, Lorenzo Valente, Duncan Weber

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

High-energy physics is a field dedicated to understanding the fundamental building blocks of the universe by smashing particles together at incredible speeds. To make sense of these collisions, scientists rely on massive detectors that act like three-dimensional cameras, capturing the spray of debris that flies out when particles collide. One of the most critical components of these detectors is the calorimeter, a device designed to stop particles and measure their energy by watching how they break apart into cascades of smaller particles, known as showers. Simulating how these showers develop inside a detector is essential for interpreting real data, but doing so with the most accurate methods currently available is a monumental computational task. It can take minutes of computer time to simulate a single event, a delay that becomes impossible to manage as future experiments generate vastly larger volumes of data.

To solve this bottleneck, researchers have turned to machine learning, training artificial intelligence to predict the final result of a particle shower without simulating every single step of the process. These digital surrogates can be thousands of times faster than traditional methods. However, a significant hurdle remains: most of these AI models are rigid. They are trained on the specific shape and layout of one particular detector, and if the detector's design changes, the model becomes useless and must be retrained from scratch. This inefficiency poses a problem for the future of physics, where detector designs are constantly evolving. The question facing the community is whether a model trained on one type of detector can learn the underlying physics well enough to adapt to a completely different shape without needing a massive amount of new data.

A team of researchers at the University of Hamburg and the DESY laboratory in Germany set out to answer this question using a specific type of artificial intelligence called CaloClouds II. Unlike older models that view a particle shower as a grid of boxes, this model represents the shower as a collection of points in space, a format that is naturally flexible enough to fit into any detector shape. The researchers began by training this model on a vast dataset of photon showers generated for the International Large Detector, a design for a future linear collider. This detector has a flat, layered structure. Once the model had learned the physics of how photons create showers in this flat environment, the team attempted to transfer that knowledge to a completely different scenario: electron showers in a cylindrical detector, which is the shape used in the CaloChallenge Dataset 3. This new environment was not just a different shape; it had a different number of layers, different dimensions, and involved a different type of particle entirely.

The team tested whether they could simply take the pre-trained model and adjust it slightly to work in this new cylindrical world, a process known as fine-tuning. They compared this approach to training a new model from the ground up using only the new data. The results showed that the pre-trained model was remarkably efficient. When the researchers gave the model only one hundred examples of the new electron showers to learn from, the pre-trained version produced results that were significantly closer to the accurate physical simulations than a model trained from scratch. Specifically, the pre-trained model reduced the error in its predictions by about half compared to starting over. This advantage was most pronounced when data was scarce, a common situation in physics where generating new simulation data is expensive and time-consuming.

The study also explored how to make this adaptation even more efficient by updating only a small fraction of the model's internal settings. They tested a method that changed only the bias terms, which are essentially the baseline settings of the network, leaving the rest of the model untouched. This approach, which updated only seventeen percent of the trainable parameters, performed almost as well as updating the entire model. It suggests that the heavy lifting of learning the physics had already been done during the initial training, and the new task only required a slight recalibration of the existing knowledge. This finding is crucial because it means that powerful, adaptable simulation tools could be developed without the massive computational cost of retraining every single part of the network for every new detector design.

However, the researchers also investigated a potential danger in using these AI models: the risk of the model degrading over time if it is repeatedly trained on its own output. In a scenario where a model generates fake data, and that fake data is then used to train the next version of the model, the system can begin to lose touch with reality. The team simulated this process by having the model generate showers and then retraining it on those generated showers, repeating the cycle over several generations. They observed that the quality of the generated data slowly but steadily declined. The distributions of energy and particle counts began to drift away from the true physical behavior, a phenomenon known as model collapse. This indicates that while these AI surrogates are powerful tools for speeding up simulations, they cannot simply be used to generate infinite training data for themselves without eventually losing accuracy.

The work demonstrates that a single detector's geometry is sufficient to teach a model the fundamental physics needed to adapt to a completely different shape. By training on one design and fine-tuning on another, the researchers achieved a level of data efficiency that makes rapid development of new simulation tools feasible. While the approach is not a replacement for the most accurate traditional methods, it offers a practical path forward for handling the massive data volumes expected in the next generation of particle physics experiments. The findings suggest that the future of detector simulation may rely less on building a new model for every new machine and more on creating adaptable systems that can learn once and apply that knowledge everywhere.

Technical Summary: Cross-geometry Transfer and Model Collapse in Point Cloud Calorimeter Shower Generation

Problem Statement
Particle shower simulation is a primary computational bottleneck in high-energy physics, particularly for future facilities like the High-Luminosity Large Hadron Collider (HL-LHC). While Monte Carlo methods like Geant4 provide accurate detector response simulations, they are computationally expensive, often requiring minutes of CPU time per event. Machine learning (ML) surrogates offer speed-ups of several orders of magnitude but face a critical limitation: most are tied to specific detector geometries. Consequently, any design change or transition to a new detector requires retraining models from scratch, which is inefficient. Furthermore, while point cloud representations offer geometric flexibility by describing showers as sets of space points rather than fixed voxels, it remains unproven whether a model pre-trained on a single detector geometry can effectively transfer its learned physics to a fundamentally different geometry. Additionally, the risk of "model collapse"—where generative models degrade when iteratively retrained on their own synthetic output—has not been systematically studied in this context.

Methodology
The authors investigate cross-geometry transfer learning using CaloClouds II, a generative model that produces point clouds rather than voxels. The model architecture consists of two components:

Diffusion Network (PointWise Net): Generates point positions and energies conditioned on incident particle energy.
Normalizing Flow (ShowerFlow): Models per-layer point counts.

The study employs a two-stage approach:

Pre-training: The model is pre-trained on 524,000 photon showers simulated in the planar electromagnetic calorimeter (ECAL) of the International Large Detector (ILD). The ILD features 30 layers of tungsten absorbers and silicon sensors.
Transfer/Adaptation: The pre-trained model is adapted to electron showers in the CaloChallenge Dataset 3, which simulates a cylindrical calorimeter (Par04) with 45 layers of concentric tungsten and silicon cylinders. This transfer involves bridging significant differences: planar vs. cylindrical geometry, 30 vs. 45 layers, photon vs. electron incident particles, and a shift in energy spectrum from uniform (10–90 GeV) to log-uniform (1 GeV–1 TeV).

The authors compare full fine-tuning against three Parameter-Efficient Fine-Tuning (PEFT) strategies:

BitFit: Updates only bias terms.
Top2: Updates the last two layers and time-step embeddings.
LoRA: Low-Rank Adaptation with rank 106.

Adaptation is tested across target dataset sizes ranging from $10^2$ to $10^5$ showers. The study also examines model collapse by iteratively retraining the diffusion and flow components on their own generated data across successive generations.

Key Results

Data Efficiency in Transfer: Pre-training significantly improves performance in low-data regimes. With as few as 102 target showers, full fine-tuning reduces the geometric mean Wasserstein distance to Geant4 by approximately 51% compared to training from scratch (0.064 vs. 0.132). As the target dataset size increases, the relative advantage of pre-training diminishes.
Parameter-Efficient Fine-Tuning: Among PEFT strategies, BitFit (bias-only) is the most effective. It achieves performance within 5% of full fine-tuning while updating only 17% of the diffusion network's parameters. Top2 (44% parameters) is competitive, while LoRA (52% parameters) underperforms, attributed to the high intrinsic dimensionality of the required weight updates.
Model Collapse: Iterative retraining on synthetic data leads to progressive degradation of the generated distributions relative to the Geant4 reference. This "model collapse" is observed in both the normalizing flow and diffusion components, with degradation accelerating in later generations for certain observables.

Significance and Claims
The paper claims that single-detector pre-training on point clouds is a practical and data-efficient route for cross-geometry transfer in calorimeter simulation. The authors emphasize that meaningful transfer is achievable even when adapting from a single geometry, provided the model architecture (point clouds) allows for projection onto arbitrary readouts.

The study provides the first systematic comparison of parameter-efficient strategies for fast calorimeter simulation, demonstrating that bias-only fine-tuning (BitFit) offers a highly efficient alternative to full retraining. However, the authors modestly note that without a direct comparison to multi-detector pre-training (as seen in foundation model approaches like CaloDiT-2), they cannot benchmark their single-detector approach against those strategies. They position single-detector pre-training as an adequate starting point when multi-detector datasets are unavailable or computational resources are limited. Finally, the identification of model collapse as a genuine failure mode for point cloud surrogates highlights a critical constraint for future applications involving self-consuming training loops.

Technical Summary: Cross-geometry Transfer and Model Collapse in Point Cloud Calorimeter Shower Generation

More like this