Model bias and parameter optimisation with the example of INCL/ABLA

This paper explores the complementarity of parameter optimization and model bias estimation within a Bayesian framework, demonstrating their application to the INCL/ABLA high-energy spallation models to improve accuracy and precision for experimental design and data analysis.

The Gist

Imagine you are a master chef trying to recreate a famous, complex dish (like a perfect soufflé) based on a recipe book. However, the recipe book is old, the measurements are vague, and you've never actually tasted the original dish. You have to rely on your best guess to make it.

This is exactly what nuclear physicists face when they try to predict how atomic nuclei behave during high-energy collisions. They use computer models (like INCL and ABLA) as their "recipe books," but these models aren't perfect. Sometimes they predict the dish will be too salty (too much energy), or too dry (wrong particle count).

This paper is about two new ways to fix the recipe so the final dish is delicious and reliable.

The Two Problems

1. The Ingredients are Wrong: Maybe the recipe says "add 5 grams of salt," but it should really be 4 grams. The physics is right, but the numbers (parameters) are off.
2. The Recipe is Missing Steps: Maybe the recipe forgets to mention that you need to let the dough rest for 10 minutes. No matter how much salt you add, the dish will still be wrong because a whole step is missing. This is called "model bias."

The Two Solutions (The "Fixes")

The authors developed two methods to fix these problems, and they found that using them together is like having a super-chef and a taste-tester working in tandem.

1. Tuning the Knobs (Parameter Optimisation)

Think of the computer model as a giant radio with thousands of dials.

• The Problem: The dials are set to the factory default, but the signal is fuzzy.
• The Fix: The team uses a smart algorithm (called Gaussian Process Regression) to listen to real experimental data (the "music") and automatically turn the dials until the model's prediction matches the real music perfectly.
• The Result: They find the "sweet spot" for the dials. This makes the model more accurate because it's now using the right ingredients.

2. Adding a "Correction Note" (Model Bias Estimation)

Sometimes, even with the perfect dials, the radio still has a static hiss because the radio itself is broken (the physics model is missing a step).

• The Problem: The model is consistently 10% too loud.
• The Fix: Instead of trying to fix the broken radio, the team calculates exactly how much it is off. They create a "correction note" that says, "Whatever this model predicts, subtract 10%."
• The Result: They can now predict the future with high confidence, even if they know the model has a flaw, because they know exactly how to correct for it.

The Magic Synergy: Why Do Both?

The paper shows that doing just one isn't enough.

• If you only tune the knobs, you might fix the numbers, but you might not realize the model is missing a whole step.
• If you only add a correction note, you are fixing the symptom, but you aren't understanding why the model is wrong.

The Analogy:
Imagine you are trying to hit a target with a bow and arrow.

• Tuning the knobs is like adjusting the sights on your bow so the arrow flies straight.
• Bias estimation is like realizing there is a strong wind blowing you to the left, so you aim slightly to the right to compensate.

If you only adjust the sights but ignore the wind, you'll miss. If you only aim for the wind but your sights are broken, you'll also miss. But if you do both, you hit the bullseye every time.

The "Secret Sauce": The Covariance Matrix

To make these calculations work, the team uses a mathematical tool called a "Covariance Matrix."

• Think of this as a "Relationship Map." It tells the computer: "If the model is wrong about the temperature, it's probably also wrong about the humidity in a similar way."
• The authors had to be very careful here. If they map the relationships wrong, the computer gets confused and gives bad advice. They tested different "map styles" (kernels) and found that a specific style (called the Matérn kernel) was the most robust, meaning it didn't get tricked by random noise in the data.

Why Does This Matter?

These models are used for real-world things like:

• Space Travel: Predicting how cosmic rays hit astronauts.
• Medicine: Designing cancer treatments (hadron therapy) that kill tumors without hurting healthy tissue.
• Energy: Building safer nuclear reactors.

By combining "tuning the dials" with "adding a correction note," the authors have created a system that gives scientists not just a prediction, but a prediction with a confidence score. They can say, "We think the result is X, and we are 95% sure it's between Y and Z."

The Catch (Limitations)

The authors admit this method isn't magic.

1. Garbage In, Garbage Out: If the experimental data they use to tune the model is messy or has hidden errors, the model will learn those errors.
2. Heavy Lifting: Doing these calculations requires a lot of computer power. It's like trying to solve a massive puzzle while the pieces are constantly moving.
3. The Map is Hard to Draw: Creating the perfect "Relationship Map" (Covariance Matrix) is difficult. If you guess the relationships wrong, the whole system breaks.

Summary

In short, this paper teaches us how to take a flawed, complex computer model of the atomic world and make it trustworthy. They do this by fine-tuning the internal settings and mathematically correcting the remaining errors, all while keeping a close eye on how confident we can be in the results. It's a recipe for turning "best guesses" into "reliable science."

Technical Summary

1. Problem Statement

High-energy nuclear physics relies heavily on accurate transport and activation models (such as INCL for intra-nuclear cascades and ABLA for de-excitation/ablation) to interpret experimental data and design facilities. However, experimental data is scarce, particularly above 20 MeV, forcing reliance on model predictions.
Current challenges include:

• Model Inaccuracy: Nuclear models often fail to reproduce experimental data across wide ranges of observables and energies.
• Uncertainty Quantification: Existing models lack well-defined uncertainties, making it difficult to assess the reliability of predictions for applications like hadron therapy, neutrino physics, and space radiation shielding.
• Limitations of Single Approaches: Previous works addressed this via either parameter optimization (tuning model inputs) or model bias estimation (correcting output errors), but the synergy between these two methods had not been fully explored.

2. Methodology

The authors propose a unified Bayesian framework based on Gaussian Process (GP) regression to address both parameter optimization and model bias estimation. The core mathematical foundation is the conditional probability of observables given experimental data, assuming multivariate normal distributions.

A. Parameter Optimization (GP Phase + Gibbs Sampling)

• Goal: Find an optimal set of model parameters ($\vec{p}_{op}$) that minimizes the discrepancy between model predictions and experimental data.
• Process:
  1. Linearization: The model is locally linearized using a Taylor expansion around a prior parameter set ($\vec{p}_0$).
  2. Iterative Update: The optimal parameters are updated iteratively using the GP regression formula (Eq. 7), which incorporates the Jacobian (sensitivity matrix) and covariance matrices of parameters and data.
  3. Convergence: The process repeats until the $\chi^2$ converges.
  4. Uncertainty Estimation: If the model likelihood is non-Gaussian or highly non-linear, a secondary phase using a Gibbs sampling-like approach is employed to sample the posterior parameter distribution, providing robust uncertainty estimates.

B. Model Bias Estimation

• Goal: Quantify and correct the inherent bias of the model (systematic errors not fixed by parameter tuning) without altering the model's internal physics.
• Process:
  1. Bias Definition: The bias is treated as an observable ($y_1$) with a prior mean of zero.
  2. Covariance Construction: A covariance matrix ($K$) is constructed using Marginal Likelihood Optimization (MLO). This matrix combines:
     • Experimental uncertainties (statistical + systematic).
     • Model stochasticity.
     • Physical correlations between observables (modeled via kernels).
  3. Kernel Selection: The authors utilize a composite kernel consisting of:
     • Diagonal kernel: For uncorrelated statistical errors.
     • Constant kernel: For global systematic biases.
     • Matérn covariance function ($\nu=3/2$): To model smooth physical correlations between observables. The Matérn kernel was chosen over the standard Square Exponential kernel because it is more robust against "fake correlations" (spurious patterns) in experimental data.
  4. Correction: The posterior bias is predicted and subtracted from the model output, with the posterior covariance providing the "systematic" uncertainty.

C. Combined Approach

The paper investigates the synergy of applying parameter optimization before bias estimation. The hypothesis is that optimizing parameters improves the physical coherence of the model, thereby reducing the residual bias and narrowing the uncertainty bounds of the bias correction.

3. Key Contributions

1. Unified Bayesian Framework: Demonstrated that parameter optimization and bias estimation are complementary orthogonal approaches that can be integrated within a single GP regression framework.
2. Robust Kernel Selection: Validated the use of the Matérn ($\nu=3/2$) kernel for bias estimation, showing it is less prone to overfitting and producing unrealistic results compared to the Square Exponential kernel when dealing with complex experimental data.
3. Algorithmic Efficiency: Proposed techniques to handle computational costs, including:
   • Reusing the Jacobian matrix across iterations if changes are within tolerance.
   • Utilizing pseudo-input points (Sparse Gaussian Processes) to reduce the $O(N^3)$ complexity of matrix inversion.
4. Handling "Unknown Unknowns": Addressed the difficulty of defining experimental covariance matrices by proposing a "non-unrealistic" matrix that accounts for both reported uncertainties and estimated unthought sources of error.

4. Results

The methods were tested on proton-induced fission cross-sections for various nuclei (Ho, Ta, Au, Pb, Bi, Th, U, Np, Pu) using the INCL/ABLA model combination.

• Case Study: The authors intentionally degraded the ABLA model by altering the fission-dissipation coefficient (from 4.5 to 7).
• Parameter Optimization: The algorithm successfully recovered the coefficient to $4.40 \pm 0.59$ and adjusted the level density curvature. The $\chi^2$/DoF improved significantly from 4.8 to 1.7.
• Synergy:
  • Bias Estimation Alone: Corrected the model to match data ($\chi^2 < 1$) but left large uncertainties, especially in regions without experimental data.
  • Combined Approach: After parameter optimization, the bias estimation yielded reduced uncertainties, particularly in extrapolation regions (away from experimental data points).
  • Conclusion: Optimizing parameters first improves the model's physical coherence, which in turn allows the bias estimation to provide more precise predictions with tighter uncertainty bounds.

5. Significance and Limitations

Significance:

• The study provides a pathway to generate reliable, high-precision nuclear data with quantified uncertainties, essential for the design of future facilities (e.g., spallation sources) and safety assessments.
• It offers a systematic way to distinguish between errors caused by incorrect model parameters (fixable via optimization) and errors caused by missing physics (requiring bias correction).
• The approach is applicable to a wide range of nuclear applications, from fundamental physics (neutrinos, antimatter) to applied fields (hadron therapy).

Limitations:

• Data Quality: The method is highly sensitive to the quality of experimental covariance matrices. Underestimated experimental uncertainties or unaccounted correlations between datasets can lead to biased results.
• Computational Cost: The approach requires inverting large covariance matrices ($O(N^3)$), making it computationally expensive for datasets with thousands of points, though sparse techniques mitigate this.
• Covariance Structure: The accuracy of the bias estimation depends heavily on the user-defined kernel structure. If the chosen kernel fails to capture the true physical correlations, the results will be incorrect.

In summary, this work establishes a robust, two-step Bayesian methodology that significantly enhances the predictive power and reliability of the INCL/ABLA nuclear model combination, offering a template for uncertainty quantification in complex physical simulations.