🤖 machine learning

Contrastive Learning for Interpretable Anomaly Detection at Collider Experiments

The paper introduces ORCA, a two-stage framework combining supervised contrastive learning with autoencoders to achieve both improved sensitivity and interpretability in collider anomaly detection by mapping events into an embedding space where anomalous samples can be quantitatively attributed to known physics processes.

Original authors: Haoyi Jia, Sagar Addepalli, Julia Gonski

Published 2026-08-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoyi Jia, Sagar Addepalli, Julia Gonski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Large Hadron Collider (LHC) as the world's most powerful particle smasher. It fires protons at each other at nearly the speed of light, creating a chaotic explosion of new particles. Physicists are hunting for "new physics"—mysterious particles or forces that don't fit into our current rulebook, known as the Standard Model. The problem is that these new particles are like needles in a haystack, hidden among billions of boring, ordinary collisions. To find them, scientists use "Anomaly Detection," a bit like a security guard at a party who ignores the usual crowd and only stops anyone acting strangely.

However, traditional security guards have two big flaws. First, they often get confused by how loud the party is (the energy of the collision) or how many people are dancing (the number of particles), flagging normal events as suspicious just because they are energetic. Second, when the guard does stop someone, they can't explain why. They just say, "This person is weird," without telling you if they are a spy, a lost tourist, or just wearing a funny hat. This paper introduces a new, smarter security system called ORCA that not only spots the weirdos better but also explains exactly what kind of weirdness they are.

The ORCA Method: A Two-Step Detective Story

The authors, Haoyi Jia, Sagar Addepalli, and Julia Gonski, propose a new framework called ORCA (Organized Representation via Contrastive learning for Anomaly detection). Think of ORCA as a two-step process designed to clean up the mess at the particle party before the security guard even starts looking.

Step 1: The "Sorter" (Contrastive Learning)
Imagine you have a giant pile of mixed-up LEGO bricks from different sets: a castle, a spaceship, and a race car. If you just throw them all in a box, it's hard to tell which brick belongs to which set. Traditional methods try to rebuild the whole pile, but they often get confused by the red bricks that look similar across all sets.

ORCA's first step uses a technique called Contrastive Learning. It's like a super-smart teacher who takes the bricks and sorts them into distinct piles based on their type before you even try to build anything. The model is trained on 48 different Standard Model processes (the "known" physics), grouping them into 14 categories like "Single Vector Boson" or "Top Pair." It learns to push bricks from different sets far apart in a virtual space, while keeping bricks from the same set close together.

The result is a highly organized "embedding space." Instead of a messy pile of raw data, the model now has a neat map where every known type of physics event has its own distinct neighborhood. This solves the first problem: the model stops getting confused by the sheer energy or number of particles because it has already organized the data by what the particles are.

Step 2: The "Security Guard" (Autoencoder)
Once the data is sorted into these neat neighborhoods, the second step kicks in. This is a standard Autoencoder, which acts like a security guard. The guard is trained only on the "background" events (the boring, normal stuff). It learns to perfectly reconstruct the layout of the normal neighborhoods.

When a new event comes in, the guard tries to rebuild it. If the event is normal, the guard can rebuild it easily. But if the event is an anomaly (a new physics signal), it doesn't fit the pattern of the known neighborhoods. The guard struggles to rebuild it, and the "reconstruction error" becomes huge. This error score is the anomaly score. Because the data was already sorted by the first step, this score is much more accurate and less likely to be fooled by high energy or crowded events.

What They Found: Better Detection and a Clearer Explanation

The researchers tested ORCA on a simulated dataset of 750 million proton-proton collisions, mimicking the conditions of the High-Luminosity LHC. They compared ORCA to a standard autoencoder that looked directly at the raw data without the sorting step.

1. Smarter Detection
The results showed that ORCA is significantly better at spotting new physics. Across almost all the different types of signals they tested, ORCA found more "true" anomalies while ignoring more "false" alarms. In the language of the paper, it improved both the "breadth" (finding more types of signals) and the "depth" (finding them even when the background noise was huge). The only slight exception was a specific low-mass resonance, which the model still treated as background-like, but for almost everything else, the new method was a clear winner.

2. The "Why" Behind the "Weird"
The most exciting part of the paper is the interpretability. In the past, if a detector flagged an event as "weird," scientists were left guessing. Was it a new particle? Was it a glitch?

With ORCA, because the events are sorted into distinct neighborhoods, scientists can now look at the "weird" event and ask: "Which known neighborhood does this look most like?" They use a statistical technique called a template fit. Imagine you have a mystery guest at the party. Instead of just saying "they are weird," you can say, "This guest looks 80% like a lost tourist from the 'Top Pair' neighborhood and 20% like a spy from the 'Di-Higgs' neighborhood."

The paper demonstrates this by "injecting" fake new signals into the data. When they did this, ORCA could accurately count how many fake signals were there and correctly identify which known physics process they resembled most. For example, if they injected a "Di-Higgs" signal but didn't tell the model it existed, the model correctly categorized the events as looking like "Top Pair" events, revealing that the model had learned the underlying physics structure (like the number of particles in the final state) even without being explicitly told.

The Bottom Line

This paper suggests that by adding a "sorting" step before the "searching" step, we can make anomaly detection at particle colliders both more powerful and more understandable. It's not just about finding the needle in the haystack; it's about being able to describe the needle's shape, color, and origin once you find it.

The authors emphasize that this is a simulation-based result. They tested it on a dataset of 750 million simulated collisions, not on real data from the LHC yet. However, the framework is designed to be modular. The "sorter" could be swapped for a faster version to run on the actual hardware triggers, and the "guard" could be any other anomaly detector. This opens the door for future real-world experiments to not just find new physics, but to immediately start understanding what it might be.

Technical Summary: Organized Representation via Contrastive learning for Anomaly detection (ORCA)

Problem Statement
Generic event-level anomaly detection (AD) in collider physics faces two persistent challenges:

Interpretability: Anomaly scores are often difficult to interpret physically. Understanding the behavior of these scores is typically limited to examining correlations with raw kinematic inputs, and signal extraction often requires narrowing the analysis to a specific, narrow phase space (e.g., a specific invariant mass window), which reduces the generality of the search.
Correlation with Kinematics: Traditional AD scores often correlate strongly with energy scales and object multiplicities rather than genuine deviations from the Standard Model (SM). Compression-based techniques, such as standard autoencoders trained on low-level inputs, can be sensitive to training population characteristics, leading to poor reconstruction of normal events or good reconstruction of rare anomalies due to dataset topology or interpolation biases.

Methodology
The authors propose ORCA (Organized Representation via Contrastive learning for Anomaly detection), a two-stage framework designed to decouple the learning of a physics-informed representation from the anomaly scoring mechanism.

Stage 1: Supervised Contrastive Embedding
- Architecture: The model utilizes an encoder network inspired by transformer architectures. It processes input events (composed of up to 10 jets, 4 electrons, 4 muons, 4 photons, and MET) by mapping each object type to a common embedding space via dedicated networks. These object tokens are augmented with positional embeddings and processed through transformer encoder blocks, followed by a multi-head attention pooling layer to aggregate them into a single event representation vector $\vec{r}$ .
- Training Objective: The embedder is trained using a supervised contrastive loss (InfoNCE-like) over 48 distinct SM processes, which are merged into 14 physics process groups. The loss encourages events from the same process group to cluster together (alignment) while spreading representations across the available space (uniformity). A variance regularizer is added to prevent representational collapse.
- Output: The model produces a 128-dimensional $\ell_2$ -normalized embedding vector. A projection head is used only during training to compute the contrastive loss; downstream tasks operate on the embedding $\vec{r}$ .
Stage 2: Unsupervised Anomaly Detection
- Architecture: A standard autoencoder is trained exclusively on the embeddings of background events (minimum-bias data).
- Training Objective: The autoencoder minimizes the mean squared reconstruction error between the input embedding and its reconstruction.
- Anomaly Score: The anomaly score for an event is defined as the mean squared error (MSE) between its standardized embedding and the autoencoder's reconstruction. Since the autoencoder learns to reconstruct only the background distribution, events from processes dissimilar to the background yield high reconstruction errors.
Interpretation Framework
- The structured nature of the embedding space allows for a maximum-likelihood template fit. The embedding dimensions are reduced via Principal Component Analysis (PCA) and Independent Component Analysis (ICA).
- Probability Mass Functions (PMFs) are constructed for each known process group in the reduced dimensions.
- A composite likelihood fit is performed on the distribution of events in an anomalous sample to attribute them to specific template processes, yielding signal and background counts with quantified uncertainties (using a sandwich estimator for composite likelihoods).

Key Contributions and Results
The method was evaluated on the COLLIDE-2V dataset, simulating High-Luminosity LHC conditions ( $\sqrt{s} = 13.6$ TeV, $\mu=200$ ) with over 50 distinct SM processes.

Improved Sensitivity:
- Compared to a baseline autoencoder trained directly on raw 110-dimensional input features, ORCA demonstrated significant gains in both the Area Under the Curve (AUC) and True Positive Rate (TPR) at a fixed False Positive Rate (FPR) of $10^{-4}$ .
- The contrastive embedding successfully separated different signal classes in the latent space (visualized via PCA), whereas the baseline autoencoder latent space showed broad, overlapping distributions.
Interpretability and Signal Attribution:
- Template Fitting: The authors demonstrated that the embedding geometry allows for the decomposition of an anomalous sample into known physics processes.
- Holdout Tests: In tests where a signal group was excluded from the training of the embedder, the blind fit (without the signal template) correctly categorized the injected events into the most similar known processes (e.g., Di-Higgs events were categorized as Top Pair events in the absence of a Di-Higgs template).
- Signal Recovery: In signal injection tests, the multi-dimensional contrastive embedding fit achieved near-perfect recovery of injected yields. This performance significantly outperformed:
  - Fits performed on the one-dimensional anomaly score (highlighting the value of multi-dimensional information).
  - Fits performed on the latent space of a standard unsupervised autoencoder (highlighting the value of the contrastive objective).

Significance and Claims
The paper claims that ORCA establishes a route to interpretable anomaly detection at colliders. By structuring the latent space via contrastive learning, the framework:

Enhances the sensitivity of AD searches to new physics signals across a broad range of processes.
Transforms the anomaly detection output from a single, uninterpretable score into a high-dimensional representation that carries physics information.
Enables the use of standard statistical techniques (template fitting) to characterize anomalous samples, attributing them to known processes or identifying them as resembling specific known processes even if the true signal is absent from the template library.
Offers a modular architecture where the embedder and the anomaly detector can be independently optimized or replaced (e.g., for low-latency trigger environments) without losing the interpretability of the final results.

The authors emphasize that this approach does not require narrowing the analysis to a dedicated measurement phase space, thereby preserving the generality of the anomaly detection search while providing a mechanism for downstream statistical analysis and model re-interpretation.

The ORCA Method: A Two-Step Detective Story

What They Found: Better Detection and a Clearer Explanation

The Bottom Line

Technical Summary: Organized Representation via Contrastive learning for Anomaly detection (ORCA)

More like this