⚛️ high-energy experiments

Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations: Methodology and Application to the 7-8 TeV ATLAS-CMS Top Quark Mass Combination

This paper derives relations connecting the BLUE and nuisance-parameter methods, generalizes the latter to include the Gamma Variance Model for handling errors-on-errors in systematic uncertainties, and demonstrates its utility in identifying sensitive uncertainties and assessing data consistency through the 7-8 TeV ATLAS-CMS top quark mass combination.

Original authors: Enzo Canonero, Glen Cowan

Published 2026-10-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Enzo Canonero, Glen Cowan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the high-stakes world of particle physics, scientists at the Large Hadron Collider are constantly trying to weigh the heaviest known elementary particle: the top quark. To do this, they do not place the particle on a scale; instead, they smash protons together at nearly the speed of light, creating a shower of new particles that detectors record. From these chaotic collisions, physicists extract a value for the mass, but this number is never perfect. Every measurement comes with a margin of error, a range of uncertainty that acknowledges the limits of their instruments and methods. These margins are split into two types: random errors, which are like static on a radio signal that averages out over time, and systematic errors, which are more like a ruler that is slightly bent. If a ruler is bent, every measurement taken with it will be off by the same amount, and no amount of repeating the experiment will fix it.

For decades, physicists have treated these systematic errors as fixed, known quantities. They assume that if a measurement says the error is 1.0 unit, it is exactly 1.0 unit. However, in reality, the process of determining that error is itself an experiment with its own limitations. The data used to estimate the error might be small, or the method used to define the error might be somewhat arbitrary. This creates a situation where the error bar itself has an error bar. Until now, standard methods for combining results from different experiments have largely ignored this second layer of uncertainty. This oversight becomes critical when scientists try to merge data from different sources to get a single, more precise answer. If the underlying assumptions about the errors are shaky, the final combined result might be misleadingly precise or biased.

Two researchers, Enzo Canonero and Glen Cowan, have developed a new statistical framework to handle this problem, applying it to the most recent combined measurement of the top quark mass from the ATLAS and CMS experiments at the Large Hadron Collider. Their work introduces a model that acknowledges that the size of a systematic error is not a fixed fact, but a value that can fluctuate. They tested this model by taking the existing data, which includes measurements from collisions at 7 and 8 trillion electron volts, and asking a simple question: what happens to the final answer if we admit that our estimates of the systematic errors might be slightly wrong? They found that for the current top quark measurements, the final answer is remarkably stable. Even when they allowed the error estimates to vary significantly, the central value of the top quark mass barely moved, staying within a tiny fraction of a gigaelectronvolt of the original result. This suggests that the current combined measurement is robust and that the standard methods used to produce it are reliable, at least for this specific dataset.

However, the researchers also explored a more dramatic scenario to see how their model behaves under stress. They artificially added a fake measurement to the mix, one that was significantly different from all the others, acting as an outlier. In a traditional analysis, such an outlier would pull the final average toward it, potentially skewing the result. But when the new model was applied, the behavior changed. As the researchers increased the uncertainty regarding the error of that fake measurement, the model automatically gave it less weight. The final result shifted back toward the original average, effectively rejecting the outlier's influence. Furthermore, the model responded to the tension between the data points by widening the confidence interval. Instead of pretending the data agreed perfectly, the model admitted that the disagreement itself was a source of uncertainty, making the final range of possible values larger. This behavior highlights a key strength of the new approach: it is sensitive to the internal consistency of the data. If the measurements agree, the result is precise; if they disagree, the model widens the error bars to reflect that reality, rather than forcing a false precision.

The study also provided a practical toolkit for other scientists who might want to use this method. The researchers derived a set of formulas that allow for these complex calculations to be done without needing to run massive, time-consuming computer simulations every time. This makes the model much easier to apply to future combinations of data. They tested their formulas against numerical simulations and found them to be highly accurate, even in difficult cases. The work serves as a bridge between two different ways of thinking about data combination, showing that they are mathematically equivalent when handled correctly. By treating the uncertainty of the error as a variable rather than a constant, the researchers have created a more honest and flexible way to combine scientific measurements. This ensures that when the next generation of top quark measurements arrives, or when other complex particles are weighed, the final numbers will reflect not just the data, but also the true confidence scientists have in the methods used to gather it.

Technical Summary: Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations

Problem Statement
In high-energy physics analyses, particularly those at the Large Hadron Collider (LHC), results are increasingly dominated by systematic uncertainties. A critical, often overlooked source of uncertainty is the imprecision in the assignment of these systematic errors themselves, termed "errors-on-errors." These arise from limited sample sizes in control datasets, ambiguities in defining a $1\sigma$ error (e.g., in theoretical uncertainties), or "two-point" systematics derived from differences between methods. Standard combination techniques, such as the Best Linear Unbiased Estimator (BLUE) or nuisance-parameter (NP) likelihood methods, typically treat systematic uncertainties as fixed, known quantities. This paper addresses the need for a statistical framework that incorporates the uncertainty of these systematic assignments, specifically when systematic uncertainties induce non-trivial correlations (correlation coefficients $\rho \neq 0, \pm 1$ ) between measurements.

Methodology
The authors extend the Gamma Variance Model (GVM), originally proposed in [1, 2], to handle complex correlation structures and provide analytical tools for practical implementation.

Equivalence of Methods: The paper first establishes a general relation between the BLUE method and the nuisance-parameter method under non-trivial correlations. While standard NP approaches induce only trivial correlations ( $\pm 1$ or $0$), the authors demonstrate that by introducing distinct nuisance parameters $\theta_i^s$ and auxiliary measurements $u_i^s$ for each measurement $i$ and systematic source $s$ , and correlating the auxiliary measurements, the NP likelihood becomes mathematically equivalent to the BLUE likelihood with a general covariance matrix.
Extension to Errors-on-Errors: The GVM is generalized to include errors-on-errors by treating the variance of the systematic uncertainty, $\sigma_{us}^2$ , as an adjustable parameter rather than a fixed constant. The estimate of this variance, $v_s$ , is modeled as a gamma-distributed random variable. This introduces a logarithmic constraint term into the log-likelihood, replacing the standard quadratic constraint.
Analytical Formulas: To avoid the computational burden of numerical maximization, the authors derive perturbative analytical formulas (expansions in the error-on-error parameter $\epsilon_s^2$ $ϵ_{s}^{2}$ ) for:
- Profiled Nuisance Parameters: Solving the coupled cubic score equations for the nuisance parameters.
- Confidence Intervals: Applying the Bartlett correction to the profile likelihood ratio to correct for deviations from the $\chi^2$ distribution caused by the non-quadratic terms in the GVM likelihood.
- Goodness-of-Fit (GOF): Deriving a corrected GOF statistic and its expectation value to assess model fit quality.

Key Contributions

Generalized Equivalence: A rigorous derivation showing that the nuisance-parameter approach can replicate the BLUE method for any correlation structure, provided the auxiliary measurements are appropriately correlated.
Non-Trivial Correlation GVM: The first extension of the GVM to handle non-trivial correlations, allowing for the simultaneous treatment of complex systematic dependencies and errors-on-errors.
Perturbative Solutions: The provision of closed-form analytical approximations for profile likelihoods, confidence intervals, and GOF statistics, facilitating the application of the GVM in complex analyses without relying solely on numerical optimization.
Robustness Analysis: A demonstration that the GVM naturally down-weights outliers when errors-on-errors are considered, while simultaneously inflating confidence intervals to reflect internal data inconsistencies.

Results: Application to ATLAS-CMS Top Quark Mass
The framework was applied to the combination of 7–8 TeV top quark mass measurements from ATLAS and CMS.

Baseline Consistency: The authors reproduced the standard BLUE combination result ( $m_t = 172.51 \pm 0.33$ GeV) using their nuisance-parameter formalism.
Sensitivity to Systematic Uncertainty: By treating the eight largest systematic uncertainties one at a time as uncertain (varying $\epsilon_s$ ), the study found the central value of the combination to be highly robust, shifting by less than 0.1 GeV across the explored range of $\epsilon_s$ . The confidence interval remained stable, showing only a $\sim 10\%$ increase when the largest systematic (b-JES) was treated as uncertain with a 50% relative error.
Outlier Sensitivity: A hypothetical scenario was introduced by adding a fictitious outlier measurement ( $m_t = 174.5$ $m_{t} = 174.5$ GeV). In this case, the GVM exhibited distinct behavior:
- As the error-on-error parameter for the new measurement increased, the central value shifted back toward the original combination, indicating the model's ability to reduce the weight of inconsistent data.
- The confidence interval inflated significantly (up to 25%), reflecting the GVM's interpretation of internal data tension as an additional source of uncertainty.
Analytical vs. Numerical: The perturbative analytical results were shown to agree closely with numerical maximization results, even in the presence of the outlier, validating the utility of the derived formulas.

Significance
The paper argues that the GVM provides a necessary statistical framework for modern particle physics combinations where systematic uncertainties are large and their own uncertainties are non-negligible. The authors claim that while the assumption of gamma-distributed variance estimates is an approximation, it offers a more realistic inference than assuming variances are exactly known. The primary significance lies in demonstrating that incorporating errors-on-errors makes combination results more robust to discrepancies between input data (outliers) while correctly penalizing the precision (widening confidence intervals) when internal consistency is poor. This methodology serves as a general tool to identify which systematic uncertainties a combination is sensitive to and to assess the robustness of results against uncertain systematic assignments.

More like this