⚛️ phenomenology

Faithful Uncertainties in ML-Based Inverse Problems: GPs, NNs and NTKs for the Triplet PDF

This paper demonstrates that in machine learning-based inverse problems for reconstructing parton distribution functions, methods with similar central predictions can yield vastly different uncertainty estimates, highlighting the critical importance of incorporating full experimental covariance, appropriate functional priors, and rigorous closure tests to ensure faithful uncertainty calibration.

Original authors: Nina Elmer, Sven Krippendorf, Maria Ubiali

Published 2026-10-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nina Elmer, Sven Krippendorf, Maria Ubiali

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the high-stakes world of particle physics, scientists do not simply observe the building blocks of the universe; they must reconstruct them from the shadows they cast. When protons smash together at immense speeds, they produce a torrent of debris, but the fundamental particles inside the proton—the quarks and gluons—remain hidden. To understand the proton's structure, researchers must work backward from the messy, noisy data of these collisions to deduce the invisible map of how momentum is shared among the proton's constituents. This is known as an inverse problem, a mathematical puzzle where the answer is not unique and the data is imperfect. The challenge is not just to find the most likely map, but to know how much trust to place in it. If the map is wrong, predictions for future experiments at facilities like the Large Hadron Collider could be misleading. For years, physicists have relied on complex statistical methods to draw these maps, but as machine learning enters the field, a new question has emerged: can these powerful algorithms tell us not just what the map looks like, but how uncertain they are about it?

A team of researchers at the University of Cambridge has tackled this question by putting three different machine learning approaches to the test in a controlled environment. They focused on a specific, simplified version of the proton puzzle: reconstructing the distribution of a particular combination of quarks, known as the triplet distribution, using data from a historic experiment called BCDMS. To ensure they could judge the results fairly, they did not use real experimental data, which has no known "true" answer. Instead, they created a simulation where they knew the exact underlying truth beforehand. They generated fake data points that mimicked real measurements, complete with realistic errors and correlations, and then asked three different machine learning systems to recover the original map. The systems they compared were Gaussian processes, which are statistical models known for their ability to quantify uncertainty; neural networks, the deep learning engines behind many modern AI breakthroughs; and a newer method called the neural tangent kernel, which attempts to describe how neural networks learn.

The researchers discovered that while all three methods could produce a central map that looked very similar to the truth, their confidence in that map varied wildly. It turned out that getting the average answer right was not enough; the crucial test was whether the estimated uncertainty matched the actual errors. The study revealed that the way a machine learning model handles the known errors in the data is the single most important factor in producing a trustworthy uncertainty estimate. Models that ignored the complex correlations between different data points, or those that tried to learn the errors from scratch without being told how they were structured, produced confidence bands that were either far too narrow or misleadingly wide. Only the models that explicitly accounted for the full structure of experimental errors, including how one measurement's error might be linked to another, managed to produce uncertainty estimates that were statistically honest.

Perhaps the most surprising finding concerned the behavior of these models in regions where no data existed. In the real world, physicists must often guess the behavior of particles at energy levels or distances they have never directly measured. The study showed that in these empty zones, the machine learning models did not simply extrapolate based on the data they had seen. Instead, their guesses were heavily dictated by the mathematical assumptions built into them before training even began. These assumptions, known as priors, acted like a hidden hand guiding the model's predictions. When the researchers changed these initial assumptions, the models' predictions in the unmeasured regions changed drastically, even though their performance in the measured regions remained identical. This means that the uncertainty reported by a machine learning model in an unexplored region is not a pure reflection of the data, but a conditional statement that depends entirely on the specific mathematical rules the model was given.

The team also tested a newer technique involving neural tangent kernels, which are designed to bridge the gap between neural networks and traditional statistical models. They found that the timing of the analysis mattered immensely. If they evaluated the model's uncertainty at the very beginning of its training, before it had learned anything from the data, the resulting uncertainty was dangerously small and completely untrustworthy. It was only after the network had been trained for a significant number of steps that the uncertainty estimates became reliable. This suggests that for these advanced methods to be useful in physics, they cannot be treated as static tools; their internal state must be monitored and evaluated at the correct stage of learning.

Ultimately, the study serves as a vital warning and a guide for the future of physics research. It demonstrates that a machine learning model can look perfect on the surface, reconstructing the known data with high precision, yet still fail to provide a faithful measure of its own ignorance. The researchers concluded that to trust these tools, scientists must move beyond simply checking if the central prediction is correct. They must rigorously test whether the model's uncertainty bands actually cover the truth, using methods that account for the complex correlations in real-world data. Without these careful checks, the powerful new tools of artificial intelligence could provide answers that are precise but dangerously misleading, especially when they are asked to venture into the unknown. The path forward requires a marriage of machine learning power with the rigorous statistical discipline of traditional physics, ensuring that when we look at the shadows of the subatomic world, we know exactly how much of the picture is real and how much is a guess.

Technical Summary: Faithful Uncertainties in ML-Based Inverse Problems

Problem Statement
This paper addresses the critical challenge of uncertainty quantification (UQ) in machine-learning (ML) based inverse problems, specifically within the context of Parton Distribution Function (PDF) determinations in high-energy physics. The authors focus on the reconstruction of the non-singlet triplet PDF, $T_3(x, Q_0^2)$ , from deep-inelastic scattering (DIS) structure functions measured by the BCDMS experiment. This serves as a controlled testbed for global PDF fits, where the goal is to reconstruct a continuous function from a finite, noisy set of indirect measurements. The central problem is that while many ML methods can produce accurate central values, their associated uncertainty estimates often lack "faithfulness"—meaning the reported uncertainties do not statistically align with the actual deviation from the true underlying function. This is particularly critical in the extrapolation regions (e.g., small- $x$ ) where data is absent, and in regions with correlated experimental uncertainties.

Methodology
The study employs a closure-test framework, where pseudo-data is generated from a known "truth" (the central value of the NNPDF4.0 global analysis) with realistic, correlated experimental uncertainties. This allows for a direct comparison of reconstructed functions and their uncertainties against the ground truth. The authors compare three distinct methodological approaches:

Gaussian Processes (GPs): Bayesian non-parametric models using different kernel choices (Radial Basis Function, Gibbs, and Matérn kernels). The authors test both "raw" GPs and those incorporating endpoint preprocessing ( $x^\alpha(1-x)^\beta$ ) to encode physical priors on small- and large- $x$ behavior.
Neural Networks (NNs): Ensembles of Multi-Layer Perceptrons (MLPs) trained with different loss functions:
- Standard Mean Squared Error (MSE).
- Heteroscedastic MSE (learning pointwise variances).
- $\chi^2$ loss (incorporating the full experimental covariance matrix $C_{exp}$ ).
  The NNs are trained on three levels of closure-test data: Level-0 (noise-free), Level-1 (single noisy realization), and Level-2 (Monte Carlo replicas of noisy data).
Neural Tangent Kernels (NTKs): Empirical approximations of the training dynamics of the NNs. The authors evaluate the NTK at three stages: random initialization, after 100 epochs, and after full training (5000 epochs). These are treated as the covariance of a GP prior conditioned on the data.

Key Contributions and Diagnostics
The paper introduces and applies specific diagnostics to assess the "faithfulness" of uncertainties:

Normalized Bias ( $R_b$ ): A diagnostic metric evaluated in the interpolation (data) region. It measures whether the deviation of the ensemble mean from the truth is statistically compatible with the ensemble covariance. The authors emphasize that $R_b$ depends not only on the pointwise width of the uncertainty bands but crucially on the correlation structure of the predictive covariance.
Extrapolation Propagation: The reconstructed $T_3$ functions are propagated to HERA neutral-current reduced cross-sections (datasets with $x$ values outside the BCDMS training range) to test the reliability of uncertainties in the extrapolation region.
Decomposition of Uncertainty: The study distinguishes between "data uncertainty" (aleatoric, from $C_{exp}$ ) and "model uncertainty" (epistemic, from ensemble spread), analyzing how different training levels (L0, L1, L2) and loss functions propagate these components.

Results
The authors find that methods yielding comparable central reconstructions can produce markedly different, and often unfaithful, uncertainty estimates:

Loss Function Impact: In the data region, NNs trained with MSE, heteroscedastic MSE, and $\chi^2$ losses produce similar central predictions. However, their uncertainties differ significantly. The $\chi^2$ loss (using full $C_{exp}$ ) yields faithful uncertainties in the interpolation region. The heteroscedastic loss, while capturing pointwise variance, fails to account for correlations, leading to uncalibrated bands. The standard MSE, ignoring data uncertainties entirely, produces overly narrow bands that underestimate the true uncertainty.
Role of Priors and Preprocessing: The choice of functional prior is dominant in the extrapolation region.
- For GPs, the choice of kernel alone is insufficient; without endpoint preprocessing, central predictions fail to reconstruct the peak. With preprocessing, reconstruction improves, but the resulting uncertainties vanish at the endpoints ( $x \to 0, 1$ ) by construction, failing to cover the truth if the assumed exponent ( $\alpha$ ) is incorrect.
- For NNs, the endpoint prefactor stabilizes the extrapolation but makes the uncertainty conditional on the assumed exponents. Fits with different small- $x$ exponents ( $\alpha=0.5$ vs. $\alpha=1.0$ ) agree in the data region but diverge significantly in the extrapolation region.
NTK Performance: The NTK evaluated at random initialization is a poor surrogate, significantly underestimating the peak and providing unfaithful uncertainties ( $R_b \approx 9.5$ ). Evaluating the NTK after training (100 or 5000 epochs) improves the central prediction and uncertainty calibration, though the trained NTK still yields slightly conservative uncertainties compared to the full NN ensemble.
Extrapolation Tests: When propagated to HERA data, all models except the NTK at initialization pass the consistency check. However, the behavior of the uncertainty bands varies: the $\chi^2$ -trained NN ensemble (L2) shows growing uncertainty in the small- $x$ region that covers the truth, whereas preprocessed GPs show vanishing uncertainty that does not cover the truth if the prior is mismatched.

Significance and Claims
The paper concludes that closure tests combined with covariance-aware diagnostics are essential for validating uncertainties in ML-based inverse problems. The authors identify three specific ingredients that control the faithfulness of uncertainties:

Likelihood Treatment: The inclusion of the full experimental covariance matrix in the likelihood (or propagation via replicas) is necessary to capture correlated uncertainties.
Functional Priors: The uncertainty in extrapolation regions is largely determined by the functional prior (encoded in kernels or endpoint preprocessing) rather than the data. Consequently, extrapolation uncertainties are conditional on these modeling assumptions.
Training Stage for NTKs: For NTK approximations, the kernel must be evaluated at trained parameters; the initial NTK is not a faithful surrogate for the trained network.

The authors modestly claim that their results illustrate why central-value agreement and visual inspection of uncertainty bands are insufficient for validation. They argue that the procedure used here—closure tests with known truth, comparison of L0/L1/L2 levels, and covariance-aware diagnostics—provides a necessary framework for validating ML approaches in physics inverse problems, though they note that a systematic study over many independent realisations is a necessary next step to fully characterize the distribution of the bias diagnostic.

More like this