Covering Unknown Correlations in Bayesian Priors by Inflating Uncertainties

This paper proposes a method to inflate uncertainties in Bayesian priors to conservatively account for unknown correlations between nuisance parameters when combining experiments with different parametrizations, thereby preventing underestimated uncertainties in the posterior results.

The Gist

Here is an explanation of the paper "Covering Unknown Correlations in Bayesian Priors by Inflating Uncertainties" using simple language and everyday analogies.

The Big Picture: The "Mystery Team" Problem

Imagine you are a detective trying to solve a case. You have two different witnesses (let's call them Experiment A and Experiment B). Both witnesses saw the same event, but they describe the confusing details (the "nuisance parameters") in completely different languages.

• Witness A says: "The suspect was wearing a red hat and a blue coat."
• Witness B says: "The suspect had a warm head covering and a heavy outer layer."

You know they are talking about the same person, but you don't know exactly how "red hat" translates to "warm head covering." Are they 100% the same thing? Are they totally different? Or are they slightly related?

In science, when we combine data from different experiments, we have to guess how these "witnesses" relate to each other. If we guess wrong, we might think we know the answer (the "parameters of interest") with too much confidence. We might say, "I'm 99% sure the suspect is John," when we should really only be 80% sure.

The Core Problem: The "Hidden Link"

The paper argues that when you combine these different experiments, there is a hidden danger: Unknown Correlations.

If the two witnesses are actually describing the exact same thing, their errors are linked. If one is wrong, the other is likely wrong in the same way. If we ignore this link and treat them as totally independent, we might accidentally cancel out their mistakes, making our final result look super precise when it's actually shaky.

The Analogy:
Imagine you are trying to guess the weight of a watermelon.

• Scale A says it's 10 lbs, give or take 1 lb.
• Scale B says it's 10 lbs, give or take 1 lb.

If Scale A and Scale B are totally independent, you can average them and say, "It's definitely 10 lbs, give or take 0.7 lbs." You feel very confident.

But, what if both scales were bought from the same factory and have the same broken spring? If Scale A is off by +1 lb, Scale B is also off by +1 lb. They are 100% correlated. In this case, averaging them doesn't help at all. The watermelon is still 10 lbs, give or take 1 lb.

If you didn't know about the broken spring (the unknown correlation), you would have underestimated your uncertainty. You thought you were more precise than you actually were.

The Solution: "The Safety Margin"

The author, Lukas Koch, asks: "How do we fix this without spending years figuring out exactly how the two scales are linked?"

His solution is surprisingly simple: Just add a "Safety Margin" to your uncertainty.

Instead of trying to guess the exact relationship between the two experiments, he suggests we assume they are completely unconnected (which usually makes us too confident) and then inflate (increase) our uncertainty to cover our bases.

The Magic Number ($n_B$):
The paper proves that if you have $n_B$ different experiments (blocks of data), you can simply multiply your uncertainty by that number to be safe.

• The Analogy: Imagine you are packing for a trip. You have 3 different suitcases (3 experiments). You don't know if they will all get lost together or just one.
  • Standard approach: You pack for the average risk.
  • Koch's approach: You pack as if all 3 suitcases might get lost at the exact same time. You bring extra clothes.

By "inflating" the uncertainty (making the "give or take" range bigger), you ensure that even if the hidden links between the experiments are the worst-case scenario, your final conclusion is still conservative. You won't be fooled into thinking you know more than you do.

Why This Works (The "Linear" Rule)

The paper relies on a key assumption: Linearity.

Think of the relationship between the "broken spring" and the "weight" as a straight line. If the spring breaks a little, the weight goes up a little. If it breaks a lot, the weight goes up a lot.

• If the relationship is a straight line: The "Safety Margin" (inflating the uncertainty) works perfectly. It guarantees you won't underestimate your error.
• If the relationship is curved (Non-linear): Things get a bit trickier. Imagine the spring gets so broken that it snaps, and the scale reads zero. That's a curve. The paper admits that in these weird, curved cases, the simple math might not be perfect, but it shows that for most real-world physics problems, the "straight line" assumption holds up well enough.

The Takeaway for Everyday Life

You don't need to be a physicist to use this logic. It applies to any situation where you are combining different sources of information that might be secretly related:

1. The Problem: You have two reports on a project. They use different metrics. You don't know how much they overlap.
2. The Risk: If you combine them naively, you might think the project is safer than it is.
3. The Fix: Don't try to solve the mystery of how they overlap. Instead, assume the worst-case overlap and add a buffer to your risk assessment.

In short: When in doubt about how different pieces of data are connected, be more humble about your precision. Widen your margin of error by the number of sources you are combining. It's better to be slightly less precise but 100% safe, than to be very precise and wrong.

Summary of the Paper's Conclusion

• Don't guess the correlation: It's too hard and easy to get wrong.
• Assume zero correlation: Start by treating the experiments as independent.
• Inflate the uncertainty: Multiply the uncertainty by the number of experiments ($n_B$).
• Result: You get a "conservative" result. You might not be as precise as you could be if you knew the secret links, but you will never be dangerously overconfident.

Technical Summary

Here is a detailed technical summary of the paper "Covering Unknown Correlations in Bayesian Priors by Inflating Uncertainties" by Lukas Koch.

1. Problem Statement

In combined Bayesian analyses of multiple experiments (e.g., joint neutrino oscillation analyses like T2K and NOvA), a significant challenge arises when different experiments use different parameterizations for their nuisance parameters to describe overlapping or related physics.

• The Core Issue: If two experiments describe the exact same physics, their nuisance parameters should be 100% correlated. If they describe independent physics, they should be uncorrelated. However, when parameterizations differ (e.g., one scales cross-sections, while another scales particle yields after Final State Interactions), the mapping between them is non-trivial, and the joint prior distribution is unknown.
• The Risk: Assuming zero correlation (independence) between these differently parameterized but physically related nuisance parameters can lead to underestimated posterior uncertainties for the parameters of interest. This occurs because "attrition" from many small, unaccounted-for correlations can accumulate, significantly reducing the reported uncertainty even if no single correlation is dominant.
• Limitations of Current Methods: Explicitly studying correlations for all possible combinations is computationally and laboriously intensive, often leading to only a subset of correlations being analyzed.

2. Methodology

The paper proposes a conservative approach to handle unknown correlations without needing to explicitly determine the correlation matrix. The methodology relies on inflating the prior covariance of the nuisance parameters.

A. Mathematical Framework

The author utilizes the Law of Total Variance to decompose the posterior variance of a parameter of interest ($\theta$) given data ($x$) and nuisance parameters ($\phi$):
$$\text{Var}[\theta|x] = \underbrace{E[\text{Var}[\theta | x, \phi] | x]}_{\text{Intrinsic Variance}} + \underbrace{\text{Var}[E[\theta | x, \phi] | x]}_{\text{Extrinsic Variance}}$$

• Assumption 1 (Linearity): The conditional expectation $E[\theta|x, \phi]$ is approximated as a linear function of $\phi$ on the scale of uncertainties.
• Assumption 2 (Constant Intrinsic Variance): The intrinsic variance does not depend on the specific values of $\phi$.

Under these assumptions, the extrinsic variance is determined by the gradient vector $a$ (sensitivity of $\theta$ to $\phi$) and the covariance matrix of the nuisance parameters $\Sigma_\phi$:
$$\text{Var}_{\text{extrinsic}} = a^T \Sigma_\phi a$$

B. The Inflation Strategy

The paper considers a scenario where the nuisance parameters are divided into $n_B$ blocks (e.g., one block per experiment). Within each block, correlations are known. Between blocks, correlations are unknown.

1. Whitening Transform: A block-wise whitening transform $W$ is applied to the known covariance blocks $\Sigma_{\phi,0}$ (assuming zero inter-block correlation) to create a normalized space where diagonal blocks are identity matrices.
2. Eigenvalue Analysis: The author analyzes the matrix $\Sigma_W = W \Sigma_\phi W^T$. The diagonal blocks are identity matrices, while off-diagonal blocks represent unknown correlations.
3. Bounding the Variance: By analyzing the eigenvalues of $\Sigma_W$, the paper proves that the maximum possible extrinsic variance (achieved by optimally tuning unknown correlations) is bounded by a factor of $n_B$ (the number of blocks) relative to the variance calculated assuming zero correlation.

The Proposed Solution:
To ensure conservative coverage, one should simply inflate the uncorrelated prior covariance matrix by the number of blocks ($n_B$):
$$\Sigma_{\phi, \text{conservative}} = n_B \Sigma_{\phi, 0}$$

3. Key Contributions

1. Analytical Proof of Conservatism: The paper provides a rigorous mathematical proof that inflating the prior covariance by $n_B$ guarantees that the resulting posterior variance will be at least as large as the variance obtained from any possible configuration of unknown correlations between the blocks, provided the linear approximation holds.
2. Handling Higher-Order Effects: The author investigates the validity of the method when relaxing the linearity assumption:
   • Quadratic Terms in Variance: If the intrinsic variance has quadratic dependence on $\phi$, inflating the prior is still conservative because it increases the average intrinsic variance, covering potential reductions caused by specific correlation choices.
   • Quadratic Terms in Expectation: If the expectation value $E[\theta|x, \phi]$ has quadratic terms, the inflation strategy remains safe for the variance of the posterior. However, it may introduce a bias in the posterior mean. The paper derives an upper bound for this potential bias ($\Delta \mu_\theta$) to help users judge acceptability.
3. Practical Applicability: The method offers a "plug-and-play" solution for combined analyses where re-parameterizing to a common basis is difficult or impossible.

4. Results

• Theoretical Bound: The maximum increase in extrinsic variance due to unknown correlations is bounded by $n_B$. Therefore, multiplying the uncorrelated prior variance by $n_B$ is sufficient to cover the worst-case scenario.
• Safety of Inflation:
  • For linear dependencies, the method is strictly conservative.
  • For quadratic dependencies in the variance function, the method remains conservative.
  • For quadratic dependencies in the mean function, the method conservatively covers the variance but may shift the mean. The shift is bounded and can be compared against the posterior uncertainty to determine if it is negligible.
• Limitations: If the nuisance parameters are the dominant source of uncertainty for the parameter of interest, a simple inflation (e.g., doubling or tripling variance) might be too aggressive or unacceptable. In such cases, a bespoke solution involving detailed physics overlap studies and re-parameterization is required.

5. Significance

• Robustness in Multi-Experiment Analyses: This paper addresses a critical gap in high-energy physics and other fields combining datasets. It prevents the "false precision" that arises from ignoring correlations between differently parameterized nuisance parameters.
• Computational Efficiency: It eliminates the need for labor-intensive, explicit correlation studies for every possible combination of nuisance parameters, offering a mathematically justified shortcut.
• Conservative Philosophy: In scientific reporting, underestimating uncertainty is a severe error. This method prioritizes the safety of the uncertainty estimate, ensuring that reported confidence intervals are robust against unknown systematic correlations.
• Generalizability: While motivated by neutrino oscillation analyses (T2K/NOvA), the statistical framework applies to any Bayesian analysis combining datasets with heterogeneous nuisance parameter definitions.

In conclusion, Koch proposes that when combining experiments with unknown correlations between nuisance parameter blocks, the safest and most straightforward approach is to assume independence and then scale the prior covariance by the number of experiments ($n_B$). This ensures that the final reported uncertainties are conservative and robust against the "attrition" of unmodeled correlations.