Robust Calibration of Non-Perturbative Models with History Matching

This paper introduces the first application of Bayes Linear Emulation and History Matching to calibrate non-perturbative models in Monte Carlo event generators, offering a robust method to identify all data-consistent parameter regions rather than relying on traditional best-fit estimates, as demonstrated with hadronisation models in Sherpa.

The Gist

The Big Picture: Tuning the Universe's Simulator

Imagine you have a super-complex video game engine that simulates how particles smash together in a giant collider (like the Large Hadron Collider). This engine is called SHERPA. It's great at predicting what happens when particles collide, but it has a "fuzzy" part: it doesn't know exactly how particles stick together to form new matter (a process called hadronization).

To make the game realistic, the developers have to "tune" about 20 to 23 knobs (parameters) on the engine. If they turn these knobs the wrong way, the simulation looks nothing like the real world. If they get them right, the simulation matches the data from real experiments perfectly.

The Problem:
Traditionally, scientists try to find the one perfect setting for these knobs. They guess, check, and adjust until they find a "best fit." But this is like trying to find a single needle in a haystack. The problem is that there might not be just one needle; there could be several different piles of hay that all look like needles from a distance. Also, the "haystack" is so huge that checking every single spot would take longer than the age of the universe.

The Solution:
This paper introduces a new, smarter way to tune the engine called History Matching. Instead of looking for the one best needle, they look for all the places in the haystack where a needle could be hiding. They don't just find the best spot; they map out the entire "safe zone" where the knobs can be turned without breaking the simulation.

---

The Analogy: The "Impossible" Recipe

Think of the particle simulation like a giant, complicated recipe for a cake.

• The Ingredients: The 20+ knobs (parameters) are things like "amount of sugar," "baking time," "oven temperature," etc.
• The Real Cake: The actual data from the particle collider (what we know is real).
• The Simulation: The cake you bake in your kitchen based on the recipe.

The Old Way (Monte Carlo Tuning):
You try to bake the perfect cake. You guess a temperature, bake it, taste it, and say, "Hmm, a bit dry. Let's lower the temp." You keep doing this until you find one set of settings that makes a cake that tastes exactly like the real one.

• The Flaw: Maybe there are two different recipes (one with high heat/short time, one with low heat/long time) that both make a perfect cake. The old method might find one and ignore the other, thinking it's the only solution.

The New Way (History Matching):
Instead of trying to find the perfect cake, you start with a giant list of every possible combination of ingredients.

1. The "Emulator" (The Taste-Tester): You can't bake 100,000 cakes to test them all. So, you build a super-smart AI "Taste-Tester" (an emulator). This AI learns from baking just a few hundred cakes and can guess what the result would be for any other combination without actually baking it.
2. The "History Match" (The Eliminator): You ask the AI: "Which of these 100,000 combinations would definitely result in a burnt or raw cake?"
3. The Sweep: The AI says, "If you use more than 2 cups of sugar, the cake is definitely too sweet. If you bake below 300 degrees, it's definitely raw."
4. The Result: You throw away all those bad combinations. You are left with a smaller, manageable list of "plausible" recipes. You repeat this process, getting more specific each time, until you have a small "safe zone" of recipes that could make a perfect cake.

What Did They Actually Do?

The authors applied this method to two different "recipes" for particle physics:

1. AHADIC: A built-in model in SHERPA (like a classic family recipe).
2. PYTHIA: A famous external model (like a celebrity chef's recipe).

They used data from old experiments at the LEP collider (where electrons and positrons smashed together) to test these models.

The Key Findings:

• No Single Best Answer: They discovered that there isn't just one perfect setting. There are actually distinct "islands" of settings that work equally well. For example, you can have a high value for "knob A" and a low value for "knob B," OR a low value for "A" and a high value for "B," and both produce the same perfect cake. Traditional methods would miss one of these islands.
• Robust Uncertainty: By mapping out the whole "safe zone," they can now say, "Our uncertainty isn't just a small circle around one point; it's this whole weird shape." This gives physicists a much more honest and robust idea of how uncertain their predictions really are.
• Model Comparison: They found that while both models (AHADIC and PYTHIA) make great cakes, they use slightly different ingredients to get there. This helps scientists understand the differences between the two theories.

Why Does This Matter?

In the past, if a scientist wanted to know how uncertain a prediction was, they would just wiggle the knobs a little bit around their "best guess." If the model had multiple "best guesses" (multiple islands), they would miss the others, leading to overly confident (and potentially wrong) predictions.

With History Matching, they have mapped the entire landscape.

• For Scientists: It means they can trust their error bars more. They know exactly where the model works and where it might fail.
• For the Future: It allows them to design better experiments. If they know there are two "islands" of possibilities, they can design a new experiment specifically to see which island is the real one.

The Bottom Line

This paper is like upgrading from a treasure hunter who digs in one spot hoping to find gold, to a cartographer who maps the entire island to show every place gold could be. It's a more honest, thorough, and powerful way to tune the complex machines that help us understand the universe.

Technical Summary

1. Problem Statement

Monte Carlo Event Generators (MCEGs) like SHERPA, PYTHIA, and HERWIG are essential for particle physics analyses (e.g., at the LHC). They combine perturbative QCD calculations with phenomenological models for non-perturbative phases, such as hadronisation (the transition of quarks/gluons to hadrons).

• The Challenge: These non-perturbative models rely on high-dimensional parameter spaces (approx. 20 parameters) that must be calibrated ("tuned") to experimental data.
• Limitations of Current Methods: Standard "Monte Carlo tuning" typically seeks a single "best-fit" parameter set and estimates uncertainties via local variations (often assuming an ellipsoidal shape around the minimum). This approach fails to:
  • Reliably identify multiple, distinct local minima (disjoint regions of parameter space).
  • Handle cases where different parameter combinations yield comparable goodness-of-fit.
  • Provide robust uncertainty estimates when extrapolating to new dynamic domains.
  • Efficiently explore high-dimensional spaces without prohibitive computational costs (requiring millions of simulator runs).

2. Methodology: Bayes Linear Emulation and History Matching

The authors introduce History Matching (HM) combined with Bayes Linear (BL) Emulation as a novel framework for calibrating MCEGs.

• Philosophy of History Matching: Instead of searching for the "best" parameters, HM systematically rules out regions of the parameter space that are "implausible" (i.e., inconsistent with observational data given uncertainties). The goal is to identify the entire "non-implausible" space.
• The Framework:
  • Model Structure: $f(x) = z + e + \epsilon(x)$, where $f(x)$ is the simulator output, $z$ is the observation, $e$ is observational error, and $\epsilon(x)$ is model discrepancy.
  • Implausibility Measure ($I(x)$): A metric quantifying how unlikely a parameter set $x$ is to produce the observed data $z$, accounting for emulator uncertainty, observational error, and model discrepancy.
    $$I(x)^2 = \frac{(E_D[g(x)] - z)^2}{Var_D[g(x)] + Var[e] + Var[\epsilon(x)]}$$
  • Bayes Linear Emulators: To avoid running the computationally expensive SHERPA simulator millions of times, the authors use statistical surrogates (emulators). These are trained on a small set of simulator runs (design of experiments) to predict outputs and uncertainties across the parameter space.
    • They use active variables to reduce dimensionality, identifying which parameters significantly influence specific outputs.
    • They employ wave-based iteration: An initial emulator rules out implausible regions; new simulator runs are generated in the remaining "non-implausible" region to train a more accurate emulator for the next wave. This continues until the non-implausible space stabilizes or becomes empty.
• Application Specifics:
  • Models: Two hadronisation models in SHERPA: AHADIC (cluster fragmentation, 19 parameters) and PYTHIA (Lund string fragmentation, 23 parameters).
  • Data: 432 observable bins from LEP experiments ($e^+e^- \to \text{hadrons}$ at $\sqrt{s}=91.2$ GeV), including event shapes, fragmentation functions, and particle multiplicities.
  • Stochasticity: The authors account for the intrinsic stochastic variability of the Monte Carlo generator by treating it as an $x$-independent contribution to model discrepancy.

3. Key Contributions

1. First Application in Particle Physics: This is the first application of Bayes Linear emulation and History Matching to non-perturbative models in MCEGs.
2. Robust Uncertainty Quantification: The method provides a systematic quantification of parametric uncertainties that captures multi-modal and disjoint regions of the parameter space, which traditional "best-fit" methods miss.
3. Model Comparison: By keeping the perturbative components identical and only varying the hadronisation model, the authors isolate and quantify the uncertainty arising specifically from the choice of physics model (Cluster vs. String).
4. Efficiency: The approach drastically reduces the number of required simulator runs (using ~300–350 runs per model for training) while exploring the full high-dimensional space, overcoming the "curse of dimensionality."

4. Results

• Parameter Space Reduction:
  • AHADIC: 3 waves of HM reduced the parameter space volume by ~3 orders of magnitude.
  • PYTHIA: 5 waves reduced the volume by ~13 orders of magnitude.
• Structure of Non-Implausible Space:
  • The final non-implausible regions exhibited complex, multi-modal structures (e.g., "banana-shaped" distributions and distinct peaks).
  • Strong correlations were found between parameters (e.g., between baryon fraction and di-quark spin ratios in AHADIC; between Lund parameters $a$ and $b$ in PYTHIA).
  • Traditional tuning would likely settle in one mode, missing viable alternative solutions.
• Observable Predictions:
  • Both models provided good overall agreement with LEP data.
  • AHADIC showed larger variations in the $b$-quark fragmentation function, while PYTHIA was more constrained but slightly undershot data in the central region.
  • For inclusive observables (charged multiplicity, event shapes), the two models yielded nearly indistinguishable predictions.
  • Discrepancies remained in heavy-flavour fragmentation and specific hadron yields (e.g., $\omega$, $K^*$, $D_s$), highlighting areas for future model refinement.
• Goodness-of-Fit: The reduced $\chi^2$ for the final wave members peaked around 1.5 for both models, with no members exceeding 2.5 (AHADIC) or 3.0 (PYTHIA), indicating a consistent level of agreement across the entire non-implausible ensemble.

5. Significance

• Paradigm Shift: Moves particle physics calibration from a "single best-fit" approach to a rigorous identification of all viable parameter regions, providing a more honest and robust assessment of model uncertainty.
• Handling Complexity: Successfully demonstrates that high-dimensional, non-linear, and stochastic problems in particle physics can be solved efficiently using emulation and iterative exclusion.
• Future Impact: The derived non-implausible regions can be used to:
  • Propagate non-perturbative uncertainties into LHC analyses.
  • Identify which observables are most constraining for future experiments.
  • Diagnose specific deficiencies in hadronisation models (e.g., heavy-flavour production).
  • Serve as a foundation for developing on-the-fly reweighting algorithms for non-perturbative uncertainties.

In conclusion, the paper establishes History Matching as a superior methodology for calibrating complex non-perturbative models, offering a robust framework to quantify uncertainties and explore the full landscape of viable physics parameters in high-energy collision simulations.