⚛️ high-energy experiments

Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

This paper introduces a data-driven, augmentation-free self-supervised pre-training method for the Large Hadron Collider that pairs distinct events based on their Energy Mover's Distance similarity to learn invariant representations, achieving downstream performance comparable to or better than traditional augmentation-based baselines while preserving event fidelity.

Original authors: Ho Fung Tsoi, Dylan Rankin

Published 2026-09-17

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Ho Fung Tsoi, Dylan Rankin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the heart of Europe, the Large Hadron Collider smashes protons together at nearly the speed of light, creating a chaotic shower of new particles that physicists must sort through to understand the fundamental laws of nature. To make sense of this deluge of data, researchers rely on powerful computer models, often called foundation models, which are trained to recognize patterns and distinguish between different types of particle collisions. A major challenge in teaching these models is that they need to learn what stays the same when a particle collision is viewed from slightly different angles or with minor variations in how the detector records it. Traditionally, scientists have tried to teach this by artificially distorting the data, creating slightly altered copies of each event to show the model that the underlying physics remains unchanged despite the noise. However, in the high-stakes world of particle physics, these artificial distortions are risky; if a computer simulation changes a particle's path in a way that violates the laws of physics, the model learns the wrong lessons, becoming unreliable when faced with real data.

A team of researchers at the University of Pennsylvania has proposed a different way to teach these models, one that avoids artificial changes entirely. Instead of inventing fake variations of a particle collision, they simply look for two real, distinct collisions that happen to look very similar to one another. They use a mathematical tool called the energy mover's distance, which calculates exactly how much work it would take to rearrange the energy in one collision to match the layout of another. If the work required is small, the two events are considered close neighbors in the world of physics. By pairing these naturally similar events together, the researchers can train their model to recognize that these two different real-world occurrences share the same core identity, without ever having to break or bend the physical laws governing the data.

The researchers tested this idea using a massive dataset of simulated particle jets, which are sprays of particles created when quarks or gluons are knocked loose during a collision. They focused on two common types of jets, known as light quark jets and gluon jets, and used their new pairing method to train a computer model. Instead of taking a single jet and twisting it into a fake version, the system took two separate jets that were naturally close in their energy arrangement and asked the model to treat them as if they were the same thing. The model learned to ignore the tiny, random differences between these real events and focus on the deeper structure that defines them. When the researchers checked the results, the model had developed a clear mental map where different types of jets grouped together neatly, even for types of jets it had never seen during training.

To see if this new method actually worked for real-world problems, the team asked the model to act as a detective, looking for rare, unusual particles hidden among the common background noise. This is a critical task at the collider, where finding a new particle often means spotting a single odd event in a sea of millions of ordinary ones. The model trained with the natural pairing method proved to be just as good, and in some cases slightly better, at spotting these rare signals than models trained with the traditional method of artificial distortion. The results showed that by relying on the natural similarities found in real data rather than handcrafted tricks, the model learned a more robust and accurate understanding of particle physics. This approach suggests that the future of training these powerful tools may lie not in making up new data, but in finding the hidden connections that already exist within the vast amounts of real data we have collected.

Technical Summary: Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

Problem Statement
Self-supervised learning (SSL) has become a cornerstone for training foundation models in high-energy physics (HEP), particularly for Large Hadron Collider (LHC) data. Standard SSL workflows rely on data augmentations to create multiple views of an event, encouraging the model to learn invariance to specific distortions or symmetries. However, in HEP, unlike computer vision, applying arbitrary augmentations is constrained by strict physical laws and detector limitations. Handcrafted augmentations risk breaking physical symmetries or violating event fidelity, while generating additional simulated event variants is computationally expensive. Consequently, there is a significant challenge in selecting a robust set of augmentations that preserves the underlying physics while providing sufficient variation for learning.

Methodology
The authors propose a data-driven alternative to handcrafted augmentations: similarity pairing via Energy Mover's Distance (EMD). Instead of distorting a single event to create augmented views, the method pairs two distinct, real collision events that are geometrically similar in terms of their energy distributions.

EMD Pairing: The Energy Mover's Distance, an optimal transport metric, quantifies the "work" required to transform the energy distribution of one event into another. A small EMD indicates that two distinct events are geometrically similar, effectively representing "the same event up to fluctuations." By pairing events with low EMD values, the model learns invariance to genuine physical fluctuations without altering the physics content of the events or introducing artificial distortions.
Pre-Training Framework: The authors integrate this pairing strategy into the jBOT framework, a self-distillation method based on the iBOT architecture. In this setup:
- Two distinct jets are paired based on their EMD similarity.
- A teacher-student encoder architecture is employed. The student processes masked versions of the jets, while the teacher processes the full jets.
- The objective is to minimize the cross-entropy between the teacher and student output distributions at both the particle and jet levels.
- The pairing replaces the augmentation step, making the method agnostic to the specific SSL algorithm as long as it requires view pairs.

Experimental Setup
The method was evaluated using the JetNet dataset, containing simulated proton-proton collision jets at 13 TeV. The dataset includes five classes: light quarks ( $q$ ), gluons ( $g$ ), $W$ bosons, $Z$ bosons, and top quarks ( $t$ ).

Pre-training: The model was pre-trained exclusively on QCD jets ( $q$ and $g$ ) using a Transformer encoder.
Pairing Schemes: Three pairing modes were explored:
1. Same-class: $q$ paired with $q$ , $g$ with $g$ .
2. Different-class: $q$ paired with $g$ , and vice versa.
3. Random: No class constraints, pairing based solely on EMD similarity.
Downstream Task: The pre-trained embeddings were probed for anomaly detection (identifying $W$ , $Z$ , and $t$ jets against the QCD background) without fine-tuning, using $k$ -nearest neighbors (kNN), cosine similarity, and Mahalanobis distance.

Key Results

Embedding Structure: t-SNE visualizations revealed that the EMD-pairing method produces embeddings with clear clustering structures. Notably, the model successfully separated jet classes not used during pre-training ( $W$ , $Z$ , $t$ ), demonstrating that the learned representations capture high-level semantics. The QCD jets ( $q$ and $g$ ) formed denser, more localized clusters compared to a baseline augmentation-based approach.
Anomaly Detection Performance:
- Among the pairing schemes, the random pairing mode (free of class constraints) yielded the highest performance (Combined Signal AUC: $0.8188 \pm 0.0076$ ), which was statistically compatible with the same-class mode ( $0.8142 \pm 0.0090$ ). This suggests that removing class constraints broadens the space of invariance learning.
- Comparison to Baseline: When compared to an augmentation-based jBOT baseline, the EMD-pairing method achieved comparable or superior performance. Specifically, using a cosine similarity score, the EMD-pairing model achieved a higher AUC for the combined signal ($0.8309$ vs. $0.8269$) and for the $W$ signal class, while the $Z$ and $t$ classes showed comparable or slightly lower performance depending on the metric.

Significance and Claims
The paper claims that similarity pairing via EMD is a viable, data-driven substitute for handcrafted augmentations in self-supervised pre-training for HEP. The primary significance lies in the ability to:

Preserve Physics: Maintain the integrity of physical laws and detector constraints by avoiding artificial distortions.
Reduce Computational Cost: Eliminate the need for generating additional simulated event variants.
Improve Representation: Generate semantic jet embeddings that are insensitive to fluctuations between physically similar events, leading to downstream discrimination power that matches or exceeds traditional augmentation-based methods.

The authors conclude that this technique is valid for training foundation models and suggest future applications to other data types, such as image-based detector data, though they do not elaborate on specific implementations beyond this scope.

Technical Summary: Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

More like this