Signal-Aware Contrastive Latent Spaces for Anomaly Detection

This paper proposes a signal-aware latent space construction using supervised contrastive learning to overcome the curse of dimensionality in particle physics, enabling high-fidelity weakly supervised anomaly detection that significantly improves discovery sensitivity for both trained and unseen Beyond the Standard Model signals.

The Gist

Imagine you are a detective trying to find a single, tiny counterfeit coin hidden inside a massive, chaotic warehouse filled with billions of genuine coins. The warehouse is so huge and the coins are so similar that looking at them one by one is impossible. This is the daily challenge for physicists at the Large Hadron Collider (LHC), where they smash particles together to find "new physics" (like supersymmetry or extra dimensions) hidden among the billions of ordinary collisions.

The problem is that the "coins" (particle collisions) have hundreds of different features (speed, angle, energy, etc.). When you try to look at all these features at once, your brain (or a computer) gets overwhelmed. This is called the "curse of dimensionality."

This paper proposes a clever new way to solve this problem using a technique called Signal-Aware Contrastive Latent Spaces. Here is how it works, broken down into simple concepts:

1. The Old Way: Trying to Memorize the Warehouse

Previous methods tried to build a perfect map of what "normal" coins look like (Standard Model physics). Then, they would look for anything that didn't fit that map.

• The Problem: If the warehouse is too big and complex (high-dimensional), the map becomes blurry and inaccurate. You might miss the counterfeit coin, or worse, you might mistake a weirdly shaped real coin for a fake one (a "false alarm").

2. The New Idea: A "Smart Sorting Machine"

The authors built a new kind of sorting machine (an AI encoder) that doesn't just memorize the "normal" coins. Instead, it learns to group things based on what they are.

Think of it like a librarian who has been trained not just on "normal books," but also on a huge library of hypothetical books (different types of alien languages, sci-fi genres, etc.).

• Contrastive Learning: The AI is told: "Take all the 'Standard Model' events and put them in one pile. Take all the 'Supersymmetry' events and put them in another pile. Take the 'Heavy Resonance' events and put them in a third pile."
• The Magic: Even though the AI is trained on simulated fake data (because we don't have real alien coins yet), it learns the shape of the differences. It creates a compressed, low-dimensional "map" where similar things are close together and different things are far apart.

3. The "Signal-Aware" Twist

Here is the genius part of this paper. Most previous AI models were trained only on "normal" data to learn what to ignore. This model was trained on both the "normal" data AND a wide variety of "what-if" signal data.

• The Analogy: Imagine a security guard who has only seen photos of regular people. If a person in a clown costume walks by, the guard might get confused. But if the guard has also studied photos of clowns, acrobats, and magicians, they instantly recognize that the person in the costume is different from the crowd, even if they've never seen that specific clown before.
• By training on many different "what-if" scenarios (signals), the AI learns a better, more sensitive map. It knows exactly where to look for the weird stuff.

4. The Two-Step Process

The paper uses a two-step pipeline to find the anomaly:

1. Compression (The Encoder): The AI squashes the complex, high-dimensional data into a simple, clean 6-dimensional "shadow" (the latent space). Because it was trained with the "what-if" scenarios, this shadow preserves the differences between normal and weird events.
2. The Search (CATHODE): Once the data is squashed into this clean shadow, it's much easier to build a perfect map of the "normal" crowd. The system then looks for anything in the real data that doesn't fit the "normal" map.

5. The Results: Finding the Invisible

The authors tested this on a specific type of particle collision (diphoton events).

• Interpolation: If they trained the AI on "clowns" with red noses and blue noses, but tested it on a "clown" with a green nose, the AI still found it easily. It learned the concept of a clown, not just the specific colors.
• Extrapolation: Even more impressively, if they trained the AI on "clowns" and "acrobats," but tested it on a completely new "magician" it had never seen, the AI still did a much better job than the old methods. It could generalize the idea of "weirdness."

Why This Matters

In the past, finding new physics required guessing exactly what the new particle looked like before you could look for it. If your guess was wrong, you missed it.
This new method is like having a universal metal detector. It doesn't need to know exactly what the counterfeit coin looks like. It just knows what the real coins look like, and it's been trained to be hyper-aware that something different might be hiding in the pile.

In a nutshell: The authors created a smart, compressed "map" of particle collisions that is trained to recognize the shape of new physics, even if it hasn't seen that specific new physics before. This allows them to find hidden signals in the noise much faster and more accurately than before, potentially leading to the discovery of new laws of the universe.

Technical Summary

1. Problem Statement

High-dimensional feature spaces in particle physics events pose a fundamental challenge for weakly supervised anomaly detection (WSAD) methods based on density estimation. As the number of dimensions increases, the fidelity of density estimation degrades rapidly (the "curse of dimensionality"), making it difficult to distinguish rare Beyond the Standard Model (BSM) signals from Standard Model (SM) backgrounds.

Existing approaches, such as CATHODE (Classifying Anomalies THrough Outer Density Estimation), struggle in high-dimensional spaces. Previous attempts to compress data using unsupervised or symmetry-based contrastive learning often fail to produce latent representations expressive enough to capture subtle differences between complex physics models, or they lack sensitivity to specific signal topologies.

2. Methodology

The authors propose a two-stage framework that bridges supervised contrastive learning with weakly supervised anomaly detection.

A. Signal-Aware Contrastive Embedding (Stage 1)

The core innovation is the construction of a low-dimensional, regularized latent space using Supervised Contrastive Learning.

• Architecture: A Particle Transformer encoder processes event data. The input consists of 11 reconstructed physics objects (4 small-radius jets, 2 large-radius jets, 2 electrons, 2 muons, and $E_T^{miss}$), each described by 12 features (four-momentum, b-tagging, n-subjettiness, and one-hot encoding).
• Training Strategy: Unlike previous methods that only used SM backgrounds, this model is trained on a diverse set of SM backgrounds and hypothesized BSM signals (e.g., SUSY, extended Higgs sectors, heavy resonances).
• Loss Function: The encoder minimizes a combined loss:
  1. Supervised Contrastive Loss ($L_{con}$): Pulls events of the same physics process (same label) together in the latent space while pushing events of different processes apart.
  2. Kullback-Leibler (KL) Regularization: Forces the latent distribution of each process toward a unit Gaussian prior ($\mathcal{N}(0,1)$). This ensures the latent space is "modelable" by downstream generative models.
• Output: A 6-dimensional latent space that is both signal-sensitive (due to contrastive training) and regularized (due to KL divergence), facilitating high-fidelity density estimation.
• Decorrelation: Photon four-momenta are explicitly excluded from the transformer input to prevent the latent space from being directly correlated with the diphoton invariant mass ($m_{\gamma\gamma}$), which simplifies the subsequent density estimation step and prevents "background sculpting."

B. Weakly Supervised Anomaly Detection (Stage 2)

The CATHODE pipeline is applied within the learned latent space:

1. Region Splitting: Events are split into a Signal Region (SR) and Sideband Region (SB) based on $m_{\gamma\gamma}$.
2. Density Estimation: A Normalizing Flow (NF) is trained on SB events in the latent space to learn the background density.
3. Synthetic Background Generation: The NF interpolates the learned density to the SR to generate a synthetic background sample.
4. Classification: A classifier (Boosted Decision Tree, BDT) trained via Classification Without Labels (CWoLa) distinguishes between the real data (SR) and the synthetic background. Events classified as "data-like" are anomaly candidates.

3. Key Contributions

• Signal-Aware Contrastive Learning: The novelty lies in including a diverse set of BSM signal processes in the contrastive training. The authors hypothesize and demonstrate that this improves sensitivity not only to trained signals but also generalizes to unseen signals via interpolation and extrapolation.
• High-Dimensional Handling: By compressing high-dimensional event data into a 6D regularized latent space, the method overcomes the limitations of density estimation in high dimensions, allowing for the use of more features (11 objects $\times$ 12 features) than previous works (which were limited to $\sim$10 features).
• Generalization Capabilities: The paper defines and tests three embedding configurations:
  • In-Dataset (ID): All signals included in training (best performance).
  • Interpolation (IP): Specific mass points of a known topology are withheld (tests parameter space generalization).
  • Extrapolation (EP): Entire signal topologies are withheld (tests generalization to new physics classes).
• Robustness: The method avoids background sculpting (artificial bumps in the $m_{\gamma\gamma}$ spectrum), a critical failure mode in anomaly detection.

4. Results

The method was tested on simulated proton-proton collision data ($\sqrt{s}=13$ TeV, $L=137$ fb$^{-1}$) in the diphoton ($H \to \gamma\gamma$) final state, covering 8 BSM signal groups.

• Latent Space Structure: The 6D latent space showed smooth, Gaussian-like background distributions and clear separation between SM and BSM processes. t-SNE visualizations confirmed that held-out signals (IP/EP) occupied distinct regions adjacent to their known topologies rather than collapsing onto the background.
• Sculpting Check: The method produced no significant sculpting in the $m_{\gamma\gamma}$ spectrum (AUC $\approx$ 0.504 for distinguishing generated vs. true background), confirming the validity of the density estimation.
• Sensitivity Improvements:
  • ID Configuration: Compared to a previous VAE-based approach (Ref. [22]), the new method improved the Significance Improvement Characteristic (SIC) by approximately 40% for tested signals. At a tighter working point ($\epsilon_B = 0.1\%$), SIC reached values as high as 17.8 for extended Higgs sector signals.
  • Interpolation (IP): Performance nearly matched the ID configuration for unseen mass points of known topologies, demonstrating robust parameter space generalization.
  • Extrapolation (EP): For entirely unseen signal topologies, the EP configuration significantly outperformed the "background-only" baseline (which had no BSM training data). This proves that training on diverse BSM signals enhances the latent space's ability to detect new physics classes.
• Information Preservation: A multi-class classification test showed that the contrastive encoder retained $\sim$95% of the information relevant for distinguishing processes compared to a fully supervised transformer, with minimal loss compared to raw features.

5. Significance

This work offers a viable path toward high-dimensional anomaly detection at the Large Hadron Collider (LHC) and future colliders. By combining supervised contrastive learning with weakly supervised density estimation, the authors successfully:

1. Overcame the dimensionality constraints that previously limited the feature space size.
2. Demonstrated that training on a diverse set of hypothesized signals creates a latent space that is robust and generalizable to unseen physics scenarios.
3. Provided a strategy that bridges the gap between model-specific supervised learning and model-agnostic anomaly detection, potentially elevating discovery sensitivity from inaccessible levels to the discovery regime.

The approach is particularly significant because it retains sensitivity to BSM models not present during training, making it a powerful tool for "model-independent" searches in a high-dimensional feature space.