⚛️ phenomenology

The proton structure one mode at a time: a Neural Tangent Kernel analysis of a DIS fit

This paper applies Neural Tangent Kernel (NTK) analysis to deep inelastic scattering data fitting, revealing that the training dynamics of neural networks for parton distribution functions are governed by the incremental learning of NTK modes, where leading modes reconstruct the target physics while sub-leading modes capture data noise, thereby providing a geometric and analytical framework to understand regularization, initialization, and the interplay between data and methodology.

Original authors: Amedeo Chiefa, Luigi Del Debbio, Richard Kenway

Published 2026-10-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amedeo Chiefa, Luigi Del Debbio, Richard Kenway

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Inside the heart of a proton, a storm of invisible particles known as quarks and gluons swirls in a chaotic dance. To understand how these particles behave, physicists rely on mathematical maps called parton distribution functions. These maps tell us the probability of finding a specific particle at a certain speed and location within the proton. However, these maps cannot be measured directly. Instead, scientists must reconstruct them by smashing protons together in giant accelerators and analyzing the debris. This process is like trying to figure out the shape of a hidden object by listening to the echoes of sound bouncing off it; the data is incomplete, and the path from the echo back to the object is fraught with ambiguity. To solve this, researchers use sophisticated computer models, often powered by artificial intelligence, to fill in the gaps. But because the problem is so difficult, the way the computer is set up can sometimes influence the final picture as much as the data itself.

A team of researchers at the University of Edinburgh has taken a deep dive into how these artificial intelligence models learn to draw these maps. They focused on a specific type of neural network, a computer program inspired by the human brain, which is currently the standard tool for determining the structure of the proton. The scientists wanted to understand the exact mechanics of the learning process: how does the computer decide what to trust in the data, and how does it avoid being misled by noise? To do this, they developed a new way of watching the computer think, not by looking at the final result, but by analyzing the internal "muscles" and "nerves" of the network as it trains. They treated the learning process like a physical system that flows over time, allowing them to predict exactly how the computer would behave without having to run the simulation thousands of times.

The researchers discovered that the computer does not learn the entire proton structure all at once. Instead, it learns in layers, starting with the most obvious and important features before moving on to the finer, more difficult details. They found that the network first captures the broad, smooth shapes of the particle distributions, which are strongly supported by the experimental data. Only later does it begin to pick up on the smaller, fuzzier details. Crucially, they observed that these later details are often just noise—random fluctuations in the data that do not represent real physics. If the computer is allowed to train for too long, it starts to memorize this noise, which makes the final map less accurate. This finding confirms that stopping the training process early is not just a practical trick, but a necessary step to separate the true signal of the proton's structure from the static of the experiment.

Another major discovery concerns how the computer is started. The researchers tested what happens when the computer's initial settings are very small versus when they are larger. They found that if the starting values are too small, the computer gets stuck in a simple, linear mode where it behaves like a basic calculator. In this state, it cannot learn the complex, curved shapes of the proton's interior. It sits idle for a long time, unable to make progress, until it finally gathers enough momentum to break out of this simple state and begin learning the true complexity. This suggests that the initial setup of the computer is critical; a setup that is too conservative can actually slow down the discovery of new physics.

The team also showed that the computer's ability to learn is limited by the number of independent directions it can explore. Even though the neural network has hundreds of adjustable settings, the data only allows it to learn effectively along a few dozen specific paths. The rest of the settings are essentially invisible to the data and do not contribute to the final answer. This means the computer is far more efficient than it appears, using only a tiny fraction of its potential to reconstruct the proton. The researchers demonstrated that by understanding these specific paths, they could predict the computer's behavior with high precision, effectively creating a mathematical blueprint for how the machine learns.

This work provides a clear window into the "black box" of machine learning in physics. It shows that the reliability of the proton maps depends heavily on knowing when to stop the computer from learning and how to set it up in the first place. By proving that the learning process follows a predictable, step-by-step pattern, the researchers have given physicists a new tool to validate their results. They can now check if their models are learning the real physics or just memorizing errors. This level of control is essential as experiments become more precise, ensuring that the next generation of discoveries about the fundamental building blocks of the universe are built on a solid, understood foundation.

Technical Summary: The Proton Structure One Mode at a Time

Problem Statement
The determination of Parton Distribution Functions (PDFs) from Deep Inelastic Scattering (DIS) data constitutes a classic ill-posed inverse problem. The finite amount of experimental data cannot uniquely pin down the continuous functional form of PDFs, necessitating implicit and explicit assumptions within the methodology to regularize the solution. While Machine Learning (ML), specifically Neural Networks (NNs), has become a standard tool for PDF parametrization (e.g., in the NNPDF4.0 framework), the training dynamics and the specific role of the methodology in shaping the final results remain difficult to disentangle from the data-driven signal. Previous analyses using Neural Tangent Kernels (NTK) were limited to single-flavour toy models. This work addresses the gap by extending the NTK analysis to a realistic, multi-flavour PDF fit to understand how gradient-based optimization and parametrization interact to regularize the inverse problem.

Methodological Framework
The authors employ a "closure test" approach, generating synthetic DIS data from a known underlying PDF set ( $f^{(in)}$ ) to validate the methodology against a ground truth. The analysis utilizes the NNPDF4.0 methodology, featuring:

Dataset: 2,968 DIS data points (neutral and charged current) restricted to observables linear in PDFs.
Parametrization: A feed-forward NN with two hidden layers (25 and 20 nodes) parametrizing 8 independent flavour combinations ( $\Sigma, g, V, V_3, V_8, T_3, T_8, T_{15}$ ) at $Q_0 = 1.65$ GeV. Preprocessing factors ( $x^{1-\delta}(1-x)^\epsilon$ ) are included.
Training: Vanilla gradient descent (GD) without stochasticity, momentum, or adaptive learning rates to align with analytical derivations.
Theoretical Tool: The Neural Tangent Kernel (NTK), $\Theta_t$ , and a metric operator $M = (FK)^T C_Y^{-1} (FK)$ , where $FK$ represents the factorization theorem tables and $C_Y$ the experimental covariance.

The core theoretical contribution is the generalization of the analytical solution of the gradient flow to the multi-flavour case. By assuming a "frozen" regime where the NTK is approximately constant (frozen) after an initial transient, the authors derive an exact analytical expression for the trained PDFs $f_t$ as a function of the initial condition $f_0$ and the data $Y$ .

Key Contributions and Results

Analytical Solution and Mode Decomposition:
The training dynamics are shown to be governed by the interplay between the NTK (encoding the architecture's expressivity) and the operator $M$ (encoding the data's constraining power). The solution is decomposed into modes $q^{(i)}$ of a "training Hamiltonian" $H_\perp$ . Each mode is learned on a timescale $\tau^{(i)} \propto 1/h^{(i)}$ , where $h^{(i)}$ is the corresponding eigenvalue.
- Result: Only the leading $\sim 30$ modes of the NTK spectrum are sufficient to reconstruct the underlying law. Sub-leading modes, learned at later stages, are poorly constrained by the data and primarily carry noise. This provides a geometric interpretation of early stopping as a bias-variance trade-off: stopping before sub-leading modes are fully resolved prevents noise inflation.
Statistical and Geometric Interpretations:
- Fisher Information: The operator $M$ is identified as the Fisher Information Matrix (FIM) of the PDFs.
- Shapley Value: For linear observables, the Shapley value (used to attribute data importance to specific flavours/kinematic regions) is derived directly from the quadratic form defined by $M$ .
- Geometry: Training is visualized as a flow on a functional manifold. The frozen NTK corresponds to a tangent plane; the analytical solution traces a trajectory on the affine plane defined by the initial condition and the image of the NTK.
Impact of Initialisation:
The study investigates the dependence of training dynamics on the width of the parameter initialisation distribution, controlled by a scaling exponent $\gamma$ applied to the variance of the Glorot normal distribution.
- Linear Regime ( $\gamma \gtrsim 3$ ): Small initialisations force the network into a linear regime (Deep Linear Network approximation) where the NTK is constant from the start. The loss function stalls on a plateau until the hidden layers saturate, triggering a "saddle-to-saddle" like escape. In this regime, the NTK at initialisation is a poor descriptor of the full training because the initial image of the NTK is too low-dimensional (limited to the span of $\{1, x, \log x\}$ ) to capture the underlying law.
- Frozen Regime ( $\gamma \lesssim 1$ ): Larger initialisations allow the network to operate in a non-linear regime immediately. The NTK evolves rapidly at the start (rich phase) and then stabilizes. Here, the analytical solution with the NTK frozen at initialisation provides a good approximation of the training dynamics.
- Optimality: The authors find that $\gamma \approx 0.6$ yields the fastest convergence and lowest validation loss, while also maximizing analytical control (agreement between analytical and trained solutions).
Wide Networks vs. Small Initialisation:
The paper clarifies that increasing the network width (number of nodes) is not equivalent to decreasing the initialisation width ( $\gamma$ ). While both can lead to similar initial weight distributions, they scale the NTK differently. A wide network maintains an $O(1)$ NTK spectrum size, whereas a small initialisation shrinks the spectrum geometrically, leading to distinct training dynamics.

Significance and Claims
The paper claims to provide a quantitative, analytical framework for understanding the training dynamics of NNs in PDF fits. Its primary significance lies in:

Demystifying Regularization: Demonstrating that the implicit regularization of gradient descent is the incremental inclusion of NTK modes, where the "frozen" regime allows for an exact analytical description of how data and initial conditions contribute to the final solution.
Separating Signal and Noise: Showing that the central value of the PDFs is driven by the data, while the initial condition contributes primarily to the uncertainty, particularly in extrapolation regions.
Diagnostic Utility: Establishing the NTK spectrum and the "escape epoch" (transition from linear to non-linear dynamics) as diagnostic tools to assess the efficiency and stability of training.
Methodological Rigor: Providing a statistical foundation (via the FIM and Shapley value) for interpreting PDF fits and offering a framework applicable to other parametrisations and physics ML applications.

The authors conclude that their framework sharpens the understanding of training dynamics and offers a pathway for stress-testing methodologies against inconsistent data, without claiming that the specific findings immediately solve all challenges in PDF determination or that the results generalize automatically to optimizers other than vanilla GD (e.g., Adam) without further investigation.

Technical Summary: The Proton Structure One Mode at a Time

More like this