Bring the noise: exact inference from noisy simulations in collider physics

This paper introduces an exact-approximate Markov Chain Monte Carlo method utilizing a novel unbiased Poisson likelihood estimator to achieve exact statistical inferences from noisy Monte Carlo simulations in collider physics, offering robust results at a computational cost comparable to existing approximate methods.

The Gist

Imagine you are a detective trying to solve a mystery: Is there a new, invisible particle hiding in the data from the Large Hadron Collider (LHC)?

To find out, you need to compare what the LHC actually saw (the "observed" data) against what your computer simulations predict should happen if a new particle exists.

The Problem: The "Noisy" Crystal Ball

In the real world, you can't calculate the exact answer. The math is too complex. Instead, you use a computer to run millions of "what-if" scenarios (simulations) to guess the answer.

Think of this like trying to guess the average height of everyone in a stadium by asking just a few people.

• The Traditional Method (MLE): You decide to ask exactly 1,000 people. You count how many are tall, divide by 1,000, and use that number as your "truth."
  • The Flaw: Because you forced a fixed number of people, your guess is slightly biased. It's like a scale that always reads 1 pound too heavy. If you don't ask enough people, your guess is wildly wrong. If you ask too many, it takes forever.
• The Noise: Every time you run the simulation, the result is a little bit "noisy" (random). Sometimes you get a lucky high number, sometimes a low one.

The Solution: The "Exact-Approximate" Trick

The authors of this paper introduced a clever new method called Exact-Approximate MCMC (or "Pseudo-Marginal").

Here is the analogy:
Imagine you are trying to weigh a bag of gold, but your scale is broken and gives a random number every time you step on it.

• Old Way: You step on the scale 1,000 times, take the average, and assume that average is the true weight. It's close, but not perfect.
• New Way (The Paper's Method): Instead of forcing the scale to weigh exactly 1,000 times, you tell the scale: "Weigh the bag a random number of times, following a specific pattern (a Poisson distribution)."

By letting the number of weighings vary randomly, the authors discovered a mathematical trick that cancels out the errors. Even though every single measurement is noisy and random, the final conclusion you draw from the chain of measurements is mathematically perfect.

The "Magic" Ingredients

1. The Unbiased Estimator (The Fair Coin)
The old method (Maximum Likelihood Estimator) was like a coin that was slightly weighted to land on heads. No matter how many times you flipped it, the average would still be slightly off.
The new method (UMVUE) uses a "magic coin" that is perfectly fair. It achieves this by changing how the coin is flipped. Instead of flipping it a fixed number of times, you flip it until a random timer stops you. This randomness is the key to removing the bias.

2. The "Sticky" Chain
The authors also noticed a side effect. Because the new method is so random, the computer sometimes gets "stuck."

• Analogy: Imagine walking through a foggy forest. If you take a step and suddenly see a giant mountain (a huge random spike in your data), you might be scared to take the next step. You get stuck in one spot.
• The Fix: The paper shows that if you don't generate enough "fog" (simulations), the chain gets very sticky and inefficient. But if you generate just the right amount of fog, the new method is just as fast as the old one, but 100% accurate.

Why This Matters

• Before: Scientists had to run massive, expensive simulations to make the "noise" small enough that the "bias" didn't matter. They were guessing, hoping they guessed enough.
• Now: Scientists can get exact answers with the same amount of computer power. They don't need to guess how many simulations to run; they just run the new algorithm, and it guarantees the result is correct, even if the individual steps are messy.

The Bottom Line

The authors built a new mathematical tool that turns a "noisy, approximate" computer simulation into a "perfect, exact" result. It's like upgrading from a blurry, hand-drawn map to a GPS that gives you the exact location, even if the satellite signal is a bit shaky.

They tested this on a search for invisible particles (neutralinos and charginos) and proved that you can get the right answer without wasting billions of dollars on extra computer time.

Technical Summary

1. Problem Statement

In searches for new physics at the Large Hadron Collider (LHC), statistical inference relies on comparing observed event counts in a "signal region" against expected background and signal counts. The likelihood function for these counting experiments follows a Poisson distribution. However, the expected signal count ($s$) depends on selection efficiencies ($\epsilon$) that cannot be calculated analytically; they must be estimated via noisy Monte Carlo (MC) simulations (e.g., using Pythia, Herwig, Delphes).

The Core Issue:

• Bias in Standard Methods: Traditional approaches use a Maximum Likelihood Estimator (MLE) where a fixed number of MC events ($n_{MC}$) are generated. This results in a binomial distribution of simulated events. However, the physical experiment is a fixed-time process expecting a Poisson distribution. This mismatch introduces a systematic bias in the likelihood estimator.
• Approximate Inference: To mitigate this bias, researchers typically generate massive numbers of MC events ($n_{MC} \gg n_{LHC}$) to make the bias negligible. This is computationally expensive and still yields only approximate inferences.
• The Goal: The authors aim to perform exact Bayesian inference despite using noisy, approximate likelihood estimators derived from MC simulations.

2. Methodology: Exact-Approximate MCMC

The paper pioneers the application of Exact-Approximate Markov Chain Monte Carlo (MCMC), also known as Pseudo-Marginal MCMC, to collider physics.

• Theoretical Basis: Pseudo-marginal MCMC algorithms converge to the exact posterior distribution even when the likelihood is replaced by a noisy estimator ($\hat{L}$), provided two conditions are met:

  1. The estimator is unbiased: $\langle \hat{L} \rangle = C \cdot L$ (where $C$ is a constant).
  2. The estimator is non-negative: $\hat{L} \geq 0$.

• The Novel Estimator (UMVUE):
  The authors construct a new estimator for the Poisson likelihood that satisfies these conditions.

  • Standard MLE Failure: Generating a fixed number of events $n_{MC}$ leads to a biased estimator because the expectation of the inverse of a binomial variable does not equal the inverse of the probability.
  • The Solution: Instead of fixing $n_{MC}$, the authors draw the number of MC events ($k_{MC}$) from a Poisson distribution with mean $n_{MC}$ ($k_{MC} \sim \text{Po}(n_{MC})$).
  • The Estimator:
    • For background $b=0$: $\hat{L}_{UMVUE} = \binom{k}{o} f^o (1-f)^{k-o}$, where $f = n_{LHC}/n_{MC}$ and $k$ is the number of simulated events passing selection.
    • For background $b>0$: A convolution of Poisson and Binomial terms is used.
  • Properties: This estimator is the Uniformly Minimum Variance Unbiased Estimator (UMVUE). It is unbiased for any choice of $n_{MC}$.

• Handling Negative Estimates:
  When the ratio $f = n_{LHC}/n_{MC} > 1$, the UMVUE can yield negative values (the "sign problem"). To handle this in MCMC:

  • The algorithm uses the absolute value $|\hat{L}|$ for the acceptance ratio.
  • The sign ($\sigma = \pm 1$) is stored and used to weight the samples when calculating posterior expectations and Effective Sample Size (ESS).

3. Key Contributions

1. Unbiased Poisson Likelihood Estimator: Introduction of a UMVUE for Poisson likelihoods in the context of collider physics, which requires drawing MC event counts from a Poisson distribution rather than a fixed number.
2. Exact-Approximate Framework: Demonstration that exact Bayesian inference is achievable in LHC analyses without the need for prohibitively large MC samples to suppress bias.
3. Implementation: Public release of C++ and Python libraries (via the ideal package) and integration into the GAMBIT framework (specifically the ColliderBit module) to facilitate adoption by the community.
4. Theoretical Proof: Rigorous proof that the proposed estimator is the UMVUE and that no unbiased estimator exists if a fixed number of MC events is used.

4. Results

The authors tested the method on toy models based on the ATLAS-SUSY-2019-09 search for neutralino-chargino production ($TChiWZ$ topology).

• Bias vs. Variance Trade-off:
  • MLE (Fixed $n_{MC}$): Shows significant bias when $n_{MC}$ is small (specifically when $n_{MC} < n_{LHC}$). To achieve results indistinguishable from the exact likelihood, $n_{MC}$ must be $\approx 50 \times n_{LHC}$.
  • UMVUE (Poisson $n_{MC}$): Returns exact inferences regardless of $n_{MC}$.
• Computational Efficiency:
  • The UMVUE performs best when $n_{MC} \approx n_{LHC}$.
  • At this optimal point, the UMVUE achieves the same computational cost (in terms of MC events simulated per posterior sample) as the biased MLE, but with zero bias.
  • If $n_{MC}$ is too small ($< n_{LHC}$), the UMVUE suffers from high variance and "stickiness" in the MCMC chain due to negative likelihood estimates, reducing efficiency.
• Performance Metrics:
  • In 2D parameter space tests ($m_1, m_2$), the UMVUE posterior distributions matched the exact likelihood results perfectly.
  • The MLE showed noticeable shifts in the 95% credible regions and posterior means compared to the exact solution.
  • Kolmogorov-Smirnov tests confirmed that MLE samples were statistically distinct from exact samples ($p < 10^{-9}$), while UMVUE samples were not ($p \approx 0.5$).

5. Significance

• Paradigm Shift: This work challenges the standard practice of "brute-forcing" MC statistics to reduce bias. It proves that one can obtain exact statistical results with significantly fewer computational resources by using a mathematically rigorous unbiased estimator.
• Robustness: The method is robust against the number of MC events generated per point, provided the mean is chosen appropriately ($n_{MC} \approx n_{LHC}$). This prevents the "faulty inferences" that occur when users unknowingly use insufficient MC statistics with biased estimators.
• Generalizability: While focused on LHC physics, the methodology applies to any field involving Poisson likelihoods estimated via noisy simulations.
• Practical Utility: By integrating into GAMBIT/ColliderBit, the authors provide a ready-to-use tool for the high-energy physics community to transition from approximate to exact inference in their global fits and exclusion limit calculations.

In summary, the paper demonstrates that "bringing the noise" (using a noisy, unbiased estimator) allows for exact inference, provided the noise is managed correctly via Pseudo-Marginal MCMC, offering a superior alternative to traditional biased estimators in collider physics.