⚛️ high-energy experiments

Higher Order Component Attribution via Learned Surrogates in High Energy Physics

This paper proposes using Kolmogorov-Arnold Networks (KANs) as learned surrogates to attribute and interpret the effectiveness of the Particle Transformer (ParT) architecture in High Energy Physics by directly linking its components, such as Multi-Head Attention, to physical jet descriptors.

Original author: Lauri Laatu, Aritra Bal, Benedikt Maier, Markus Klute, Michael Spannowsky

Published 2026-11-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original author: Lauri Laatu, Aritra Bal, Benedikt Maier, Markus Klute, Michael Spannowsky

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast, silent halls of particle physics, scientists do not look for objects that sit still. Instead, they watch for the fleeting, chaotic aftermath of collisions, where the most fundamental building blocks of the universe smash together and instantly dissolve into showers of new particles. When a heavy particle like a top quark or a Z boson decays, it does not simply vanish; it sprays a narrow, cone-shaped stream of debris known as a "jet." These jets are not uniform clouds but possess a complex internal architecture, much like a tree with distinct branches. A jet from a simple particle might look like a single, thick trunk, while a jet from a more complex, heavy particle might split into two or three distinct prongs. For decades, physicists have relied on hand-crafted mathematical tools to measure these patterns, counting how energy is shared between pairs or groups of particles to identify what created the jet. Today, however, powerful artificial intelligence systems have taken over this task, learning to recognize these subtle shapes with incredible speed and accuracy. Yet, a lingering question remains: do these massive computer models actually understand the physics they are mimicking, or are they simply finding clever shortcuts to get the right answer?

A team of researchers set out to answer this by peering inside the "black box" of a modern machine learning model used to identify these jets. They focused on a specific type of artificial intelligence called a transformer, which has become the gold standard for analyzing particle data. These models, particularly one known as the Particle Transformer, are designed to pay attention to how particles relate to one another, much like a human observer scanning a crowd to see who is standing near whom. The researchers wanted to know if the model was truly learning the physical laws that govern how energy flows within a jet, or if it was just memorizing patterns without understanding the underlying cause. To do this, they did not just look at the final answer the model gave; they built a new, simpler tool to act as a translator, mapping the model's complex internal decisions back to the specific physical relationships between particles.

The researchers trained a sophisticated computer model to distinguish between jets created by light particles, which form a single-pronged structure, and jets from heavier particles that form two or three prongs. Once the model was fully trained, they introduced tiny, controlled changes to the input data—shifting the energy or position of individual particles slightly—to see how the model's confidence changed. They then used a specialized mathematical framework, known as a Kolmogorov-Arnold Network, to act as a surrogate or a stand-in. This stand-in model was simple enough to be analyzed in detail, allowing the researchers to calculate exactly how much the model relied on specific pairs of particles or groups of three particles to make its decision. They compared these internal "attention scores" against the traditional physical measurements that physicists have used for years to describe jet structures.

The results revealed a clear and encouraging alignment between the artificial intelligence and the laws of physics, but with a distinct limit. For jets that split into two prongs, the researchers found that the Particle Transformer model paid the most attention to the exact same pairs of particles that traditional physics formulas identify as the most important. The model was not just guessing; it had learned to focus on the physical correlations that define a two-pronged jet. However, when the researchers looked at jets with three prongs, the connection became weaker. While the model still performed well, its internal focus on groups of three particles did not align as perfectly with the physical rules as it did for the two-pronged case. This suggests that while the model has successfully learned the physics of simpler structures, it may be relying on different, perhaps less intuitive, strategies to handle the more complex, three-pronged patterns.

Crucially, the study also tested whether making the model deeper and more complex would fix this issue. They added more layers of processing blocks to the network, hoping that a larger, more powerful system would naturally learn the deeper physical connections. Instead, they found that adding more layers did not improve the model's ability to identify the correct physical patterns. In fact, for some standard models, adding depth made the alignment with physical laws worse. This indicates that simply throwing more computing power at the problem is not the solution. The model does not automatically learn the deeper, irreducible physics just because it is bigger.

The study concludes that while modern artificial intelligence is remarkably effective at identifying particle jets, its understanding of the underlying physics is not uniform. It excels at recognizing the clear, two-part structures that dominate certain particle decays, aligning its internal logic with established physical laws. Yet, for more complex, three-part structures, the model's reasoning diverges from the traditional physical descriptions. This does not mean the model is failing; it simply means that its path to the correct answer is different from the one a human physicist would take. The researchers suggest that to truly master the complex, multi-layered patterns of the subatomic world, future models may need to be built with specific physical insights baked into their design from the start, rather than hoping they will discover these laws on their own through sheer size and complexity.

Technical Summary: Higher Order Component Attribution via Learned Surrogates in High Energy Physics

Problem Statement
Experimental High Energy Physics (HEP) has increasingly adopted Transformer architectures, specifically the Particle Transformer (ParT), for jet identification. These models achieve state-of-the-art accuracy by explicitly injecting particle correlation information into attention mechanisms. However, a significant gap remains in interpretability: it is unclear whether these high-capacity models learn the underlying physical descriptors of jet substructure (such as energy flow patterns) or rely on algorithmic shortcuts. Specifically, there is a need to determine if models like ParT can learn higher-order, irreducible interactions (e.g., three-body correlations) that define complex jet topologies, or if they merely approximate these through pairwise interactions.

Methodology
The authors propose a framework for Higher-Order Component Attribution using Kolmogorov-Arnold Networks (KANs) as learned surrogates. The methodology proceeds as follows:

Target Models: The study utilizes two architectures trained on the JETCLASS dataset for multiclassification of jet flavors (up to ten classes): a standard Multi-Head Attention (MHA) transformer and the Particle Transformer (ParT).
Surrogate Construction: Starting from a fully trained target model, the authors apply localized Gaussian perturbations to the input particle kinematic vectors. They then train a KAN with cubic b-spline activations to act as a local surrogate. This surrogate maps the input perturbations to the shifts in the target model's output logits. The b-spline formulation ensures the surrogate is twice continuously differentiable and avoids global oscillatory artifacts (e.g., Runge's phenomenon).
Higher-Order Extraction: The authors extract interaction coefficients directly from the analytical gradients of the surrogate KAN evaluated at the origin (the intact jet configuration).
- The second-order derivative tensor (Hessian, $I^{(2)}$ ) quantifies 2-body interactions.
- The third-order derivative tensor ( $I^{(3)}$ ) quantifies 3-body interactions.
- These are interpreted as Taylor expansion coefficients of the surrogate.
Physical Alignment: The extracted interaction scores are compared against physical observables known as Energy Correlation Functions (ECFs), specifically $e^{(1)}_2$ (2-point) and $e^{(1)}_3$ (3-point). These ECFs measure how energy is shared among particle pairs and triplets, weighted by their transverse momentum fractions and angular separations.
Metric: The alignment between the model's learned importance (via the KAN surrogate) and the physical descriptors is quantified using the Spearman rank correlation coefficient ( $R_{CORR}^{(r)}$ ).

Key Contributions

Novel Attribution Framework: The paper introduces a method to attribute model decisions to higher-order physical components (pairs and triplets) using KAN surrogates, moving beyond standard first-order or pairwise attribution methods.
Physics-Informed Interpretability: It provides a direct mechanism to test whether deep learning models learn the same irreducible patterns defined by Standard Model physics (specifically ECFs) or if they rely on different, potentially non-physical, features.
Architecture Comparison: The study systematically compares standard MHA blocks against the specialized Particle Transformer (ParT) architecture regarding their ability to capture higher-order correlations.

Results

Two-Prong Jets (2-body interactions): For jets with two-pronged substructures (e.g., $Z \to q\bar{q}$ ), the Particle Transformer (ParT) exhibits a clear positive correlation ( $R_{CORR}^{(2)} \approx 0.3$ ) between its extracted 2-body interaction scores and the physical $e^{(1)}_2$ contributions. In contrast, the standard MHA baseline shows no significant alignment ( $\approx 0$ ). This suggests ParT's architecture inherently biases the model toward physics-informed pairwise interactions.
Three-Prong Jets (3-body interactions): For three-pronged jets (e.g., top quark decays), the correlation for both architectures is weaker. However, ParT still demonstrates a stronger correlation than MHA, with error bars excluding null correlation. The authors note that ParT does not receive explicit physics-informed inductive bias for these irreducible 3-body terms, yet it identifies the dominant triplets more effectively than standard MHA.
Effect of Model Depth: Increasing the number of attention blocks (depth) does not improve the alignment between the model's learned interactions and the physical ECFs. For MHA, adding depth actually reduces alignment. For ParT, the alignment remains consistent but does not significantly improve with additional blocks.
Surrogate Fidelity: The KAN surrogates achieve high fidelity ( $R^2 \approx 0.99$ median for ParT), accurately capturing the variance in the target model's output under small perturbations.

Significance and Claims
The paper claims that higher-order component attribution via learned KAN surrogates allows for a direct mapping between physical descriptors of jet radiation patterns and the decision boundaries of transformer-based networks. The primary finding is that while state-of-the-art architectures like ParT show a correlation with physical 2-body and 3-body observables, this alignment is not automatically guaranteed by increasing model capacity (depth). The results suggest that specialized architectures (like ParT) possess a superior ability to identify physically relevant particle groupings compared to standard MHA, but they do not necessarily learn higher-order irreducible structures more effectively simply by adding more layers. The authors conclude that further work is needed to extend this methodology to larger datasets, more exotic substructures, and interaction orders beyond three.

Technical Summary: Higher Order Component Attribution via Learned Surrogates in High Energy Physics

More like this