⚛️ high-energy experiments

Goodness-of-fit for multi-distribution neutrino cross-section measurements with shared events

This paper introduces the range-projected $\chi^{2}$ test statistic to resolve unphysical goodness-of-fit results in multi-distribution neutrino cross-section measurements caused by rank-deficient covariance matrices arising from shared events, by restricting the analysis to the subspace of independent statistical information.

Original authors: Francisco Martinez Lopez, London Cooper-Troendle, Steven Gardiner, Patrick Green, Tanaz Mohayai

Published 2026-09-22

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Francisco Martinez Lopez, London Cooper-Troendle, Steven Gardiner, Patrick Green, Tanaz Mohayai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the subatomic world, scientists study how neutrinos—ghostly particles that pass through almost everything—interact with matter. To understand these interactions, researchers fire beams of neutrinos at targets and record the debris that flies out. From these collisions, they extract detailed maps called cross-sections, which describe the probability of specific outcomes at different energies and angles. Modern experiments have become so sophisticated that they can now pull multiple different maps from the exact same collection of collision events. Instead of just looking at the speed of a particle, they might simultaneously chart its speed, its angle, and how those two properties relate to one another, all from a single batch of data. This approach is powerful because it uses every available piece of information, but it creates a subtle mathematical trap. Because the same events are used to build every single map, the numbers in these different charts are not independent; they are locked together by the very fact that they share the same source.

This interdependence creates a hidden problem when scientists try to test their theories against the data. To see if a model of particle physics matches reality, researchers typically compare their predictions to the measurements using a statistical score that sums up the differences. However, when the data comes from these shared-event maps, the standard way of calculating this score breaks down. The shared events impose strict rules on the numbers: if you add up the counts in one map, they must mathematically equal the counts in another, because they are counting the same physical objects. These rules mean that some directions in the data carry no new information at all; they are fixed by the geometry of the measurement rather than by the physics of the collision. When scientists ignore this and run their standard tests, the math treats these fixed, unchanging directions as if they were free to vary. This causes the statistical score to explode with artificial, nonsensical values, leading researchers to falsely reject correct theories or to draw the wrong conclusions about how particles behave.

A team of physicists has now developed a way to fix this flaw, ensuring that these multi-dimensional measurements can be tested correctly. They realized that the problem is not a mistake in the data, but a predictable feature of how the data is organized. By analyzing the structure of the bins—the boxes used to sort the events—they identified exactly which combinations of numbers are locked together and which are free to vary. They then created a new testing method that simply ignores the locked directions and focuses only on the parts of the data that actually contain independent information. This approach, which they call the range-projected test, strips away the artificial noise that was inflating the scores.

To prove their method works, the researchers first tested it on a simplified, computer-generated model of a neutrino experiment. In this simulation, they compared a "true" model of particle behavior against a "fake" model that was slightly different. When they used the old, uncorrected method, the test falsely rejected the correct model about one-third of the time, simply because the math was confused by the shared events. When they applied their new projection method, the test behaved perfectly, accepting the correct model and rejecting the wrong one with the expected reliability. They then took the method to a more realistic scenario, simulating a complex experiment involving argon targets and real-world detector effects. Here, the uncorrected test again failed, rejecting both the correct and incorrect models as if neither made sense. The new method, however, correctly identified the true model while rejecting the false one, restoring the experiment's ability to distinguish between competing theories.

The researchers also showed that they can predict exactly how many of these locked directions exist before even looking at the data, simply by knowing how the experiment is set up and how the bins are arranged. This means the fix is not a guess; it is a precise calculation based on the design of the measurement itself. By removing the directions that carry no statistical power, the new test statistic provides a clear, honest answer about whether a theory fits the data. This work ensures that as experiments become more complex and extract more information from the same events, scientists can trust their conclusions. It turns a potential source of error into a solved problem, allowing the community to move forward with confidence in their understanding of the neutrino's elusive nature.

Technical Summary: Goodness-of-fit for multi-distribution neutrino cross-section measurements with shared events

Problem Statement
Modern neutrino cross-section analyses increasingly extract multiple differential distributions from a single, common event sample (e.g., MicroBooNE measurements of final states with and without protons, or multi-differential QE-like cross sections). In such analyses, the same physical events contribute simultaneously to multiple reported distributions. This "event-sharing" structure imposes exact linear constraints among the bin counts across different distributions. For instance, if every selected event populates exactly one bin in each distribution, the total event counts across distributions must be identical.

These constraints reduce the effective dimensionality of the measurement below the total number of bins. Consequently, the statistical covariance matrix ( $C_{stat}$ ) exhibits a rank deficiency, possessing a null space where the variance is identically zero. While limited numerical precision often prevents the covariance matrix from being exactly singular (allowing for inversion), and systematic uncertainties may "lift" these null directions to small non-zero variances, standard global goodness-of-fit tests (such as $\chi^2$ ) fail. When a forward-folded prediction is compared to data, small residuals along these constrained directions are divided by near-zero variances. This results in arbitrarily large, unphysical contributions to the $\chi^2$ statistic, inflating the total value and leading to the false rejection of correct models. Furthermore, ignoring the rank deficiency results in an incorrect number of degrees of freedom.

Methodology
The authors propose a "range-projected $\chi^2$ " test statistic designed to restrict the goodness-of-fit test to the subspace carrying independent statistical information. The methodology proceeds as follows:

Identification of Constraints: The paper formalizes the constraints using a "combination matrix" ( $\Omega$ ). Each row of $\Omega$ represents a unique indicator vector describing which bins an event populates across all distributions. The rank of $\Omega$ determines the number of independent directions in the measurement space. The number of null directions is defined as $N_{null} = N_{bins} - \text{rank}(\Omega)$ .
Decomposition of Null Space: The authors decompose $N_{null}$ $N_{n u l l}$ into two components:
- Structural Null Space ( $N_{null}^{struct}$ ): Determined a priori by the geometry of the binning scheme (bin edges) and the topology of the blocks, independent of the specific event sample.
- Kinematic Null Space ( $N_{null}^{kinem}$ ): Arising from phase-space occupancy (e.g., regions of phase space that are geometrically allowed but physically empty due to cross-section suppression or detector acceptance). This component is determined from the Monte Carlo event-to-bin mapping.
Construction of the Projector: The authors construct an orthogonal projector ( $P_{reco}$ ) onto the subspace orthogonal to the null space (the range of the covariance). This is achieved via the eigendecomposition of $\Omega^T\Omega$ , separating non-zero eigenvalues (physical directions) from zero eigenvalues (constrained directions).
Test Statistic Definition: The range-projected $\chi^2$ is defined in reconstructed space as:
$\chi^2_{proj} = \delta^T (P_{reco} C_{total} P_{reco})^+ \delta$
where $\delta$ is the residual vector, $C_{total}$ is the total covariance matrix, and $(\cdot)^+$ denotes the Moore–Penrose pseudoinverse. Practically, this is computed by reducing the problem to the $r = N_{bins} - N_{null}$ dimensions spanned by the non-null eigenvectors, avoiding the need for a pseudoinverse on the full matrix.

Key Contributions
The paper makes two primary contributions:

Theoretical Insight: It demonstrates that the rank deficiency in the covariance matrix is not an incidental numerical artifact but a predictable, structural consequence of the shared-event measurement design. The constrained directions correspond to linear combinations of bins that are fixed by event conservation rather than measured, meaning the data carries no statistical power along these directions.
Prescriptive Algorithm: It provides a method to compute the number of constrained directions ( $N_{null}$ ) a priori. This includes an algorithm to determine the structural contribution directly from bin edges (Appendix A) and a procedure to identify kinematic contributions from the event sample. This allows for the exact determination of degrees of freedom without relying on arbitrary eigenvalue thresholds applied to the systematics-contaminated covariance matrix.

Results
The method was validated through two distinct studies:

Analytical Toy Model: A simulation of a muon neutrino charged-current quasi-elastic-like interaction with a three-block binning scheme (22 bins total) revealed 5 null directions.
- The unprojected $\chi^2$ (using 22 degrees of freedom) yielded a mean value of 30.6, significantly higher than the expected 22, leading to a false rejection rate of 32.4% at the 5% significance level.
- The range-projected $\chi^2$ (using 17 degrees of freedom) yielded a mean of 17.23, consistent with the expected $\chi^2(17)$ distribution, and a false rejection rate of 5.7%.
- Parameter estimation using the projected statistic recovered the true parameter values with correct frequentist coverage, whereas the unprojected statistic produced biased results and severe undercoverage.
Event Generator Study: A realistic simulation of neutrino-argon cross-section measurements using GENIE and NuWro generators, including unfolding (Wiener-SVD) and multi-source systematic uncertainties.
- The study identified 5 null directions (4 structural, 1 kinematic).
- The unprojected statistic rejected both the correct (NuWro) and incorrect (GENIE) models, rendering the test useless for discrimination.
- The projected statistic correctly accepted the true model ( $\chi^2 = 15.7/16$ ) and strongly rejected the incorrect model ( $\chi^2 = 154.4/16$ ), restoring both calibration and discriminating power.
- The study confirmed that the projected statistic yields identical numerical values whether computed in reconstructed or unfolded space, provided the projection is applied before unfolding.

Significance and Scope
The paper argues that as multi-distribution measurements with shared events become standard, a correctly calibrated global goodness-of-fit test is essential for model tuning and generator comparison. The range-projected $\chi^2$ offers a lightweight, a priori solution that requires only the published covariance, binning, and response matrices, making it applicable to already-published measurements.

The authors note that the method relies on the assumption of exact event sharing. If distributions use different phase-space limits, background subtractions, or signal definitions, the constraints become approximate. In such cases, the null directions acquire small but genuine variances. The paper provides diagnostics (comparing structural vs. empirical ranks and checking spectral gaps) to detect imperfect sharing. If sharing is not exact, the authors suggest that the structural projection may over-count constraints, and alternative approaches like binned likelihood fits (which absorb constraints automatically) may be more appropriate. However, for measurements designed with exact sharing, the projection method is presented as the necessary correction to avoid unphysical inflation of goodness-of-fit metrics.

More like this