Technical Summary: Defining a Minimum Resolution for Unbinned Analyses

Problem Statement

High-energy physics (HEP) collider analyses increasingly rely on Machine Learning (ML) models to estimate background processes in high-dimensional, unbinned feature spaces. While these techniques enhance statistical power, they introduce a critical vulnerability: background model misspecification.

When a data-driven or simulation-based background model $p(x|\theta_b)$ does not perfectly match the true background $p_{true}(x)$, it can lead to systematic biases. Specifically, if the model underestimates the probability density in regions where signal and background overlap (often due to training artifacts or architectural biases "spilling" density into the tails), the signal-to-background likelihood ratio is overestimated. This results in a false overestimation of the signal strength ($\hat{\mu}_s$), potentially leading to false discoveries. Quantifying and correcting these biases is difficult because they are often non-Gaussian, hard to parameterize, and depend on the specific training distribution of the ML model.

Methodology: Minimum Resolution Likelihood (MRL)

The paper proposes the Minimum Resolution Likelihood (MRL) method as a conservative solution to mitigate signal overestimation caused by background mismodeling. The core philosophy is to trade systematic uncertainties for statistical uncertainties by defining a Fiducial Signal Region (FSR) that excludes regions where the background model is unreliable.

The methodology proceeds as follows:

1. Calibration Region: A region containing only background (or a distribution closer to the true background than the model) is required. This region is used to validate the background model.
2. Likelihood Volume Comparison: The method compares the cumulative distribution of the background model's likelihood values, $L = p(x|\theta_b)$, between:
   • Samples generated by the background model (Toys).
   • Samples from the Calibration Region (representing the true background).
3. Determination of $L_{min}$: A critical likelihood threshold, $L_{min}$, is defined as the value where the discrepancy between the model's predicted survival function and the true background's survival function is maximal:
   $$L_{min} \equiv \arg \max_{L_c} | f(L_c | \theta_b) - f(L_c | true_b) |$$
   where $f(L_c)$ is the fraction of events with likelihood greater than $L_c$.
   • Interpretation: $L_{min}$ corresponds to the "crossing point" where the background model transitions from underestimating to overestimating the background density. Events with likelihoods below this threshold are considered unreliable.
4. Fiducial Signal Region (FSR): Events in the Signal Region with $p(x|\theta_b) < L_{min}$ are discarded. The statistical analysis is then performed only on the remaining events.
5. Modified Likelihood: The extended likelihood function is redefined to include efficiency terms ($\epsilon$) that account for the fraction of signal and background retained within the FSR. These efficiencies are estimated using the Calibration Region and toy simulations.

Key Contributions

• Definition of MRL: A formal prescription to define a data-driven cut ($L_{min}$) that isolates the region where a background model is trustworthy, effectively converting unquantifiable systematic biases into statistical uncertainties due to reduced sample size.
• Theoretical Justification: The paper demonstrates that $L_{min}$ mathematically corresponds to the crossing point between regions where the model underestimates and overestimates the background, providing a robust criterion for exclusion.
• Integration with Nuisance Parameters: The framework accommodates nuisance parameters (e.g., shape uncertainties) by re-weighting the likelihood, allowing for profiling even after the FSR cut is applied.
• Diagnostic Utility: The method serves as a diagnostic tool to identify when a model's bias is significant enough to invalidate an analysis, forcing a conservative estimate (consistent with zero) rather than a biased discovery.

Results

The authors validate the MRL method using two toy examples and a realistic application to di-Higgs searches using the HI-SIGMA technique.

Toy Examples

1. 1D Gaussian Case:
   • When the background model's standard deviation was misspecified, a standard fit significantly overestimated the signal strength.
   • Applying MRL reduced the bias, yielding signal strength estimates consistent with the true value (or zero for small signals), albeit with increased uncertainty due to the reduced event count.
   • Coverage tests showed that while the biased fit failed to cover the true parameter, the MRL-corrected fit restored coverage for moderate-to-large signals, though it remained conservative for very small signals.
2. 2D Exponential/Gaussian Case:
   • In a 2D scenario mimicking di-Higgs kinematics, the biased model again led to significant signal overestimation.
   • MRL successfully corrected the bias across different signal strengths. However, for "Rare" signal scenarios (low statistics), the method became highly conservative, often resulting in estimates consistent with zero and undercoverage, highlighting the trade-off between bias removal and statistical power.

Realistic Application: HI-SIGMA (Di-Higgs Search)

• The method was applied to a di-Higgs dataset ($hh \to b\bar{b}\gamma\gamma$) using conditional Normalizing Flows (cNF) for background estimation.
• Without Masking: The initial application of MRL without sideband masking resulted in an overly conservative cut that removed most signal events, leading to large uncertainties and a loss of sensitivity.
• With Masking: When combined with a sideband masking technique (previously shown to improve HI-SIGMA), MRL effectively removed the problematic sideband regions. The resulting fit was unbiased (or consistent with zero) with uncertainties smaller than a mass-only fit, though larger than the standard HI-SIGMA fit.
• Pseudo-experiments: In a limited set of pseudo-experiments, MRL consistently prevented false discoveries in low-signal regimes, forcing the signal strength estimate to be consistent with zero when the background bias was too severe to be trusted.

Significance and Claims

The paper claims that MRL offers a conservative "band-aid" for high-dimensional, unbinned analyses where background models are imperfect. Its primary significance lies in:

1. Bias Mitigation: It prevents the dangerous scenario of false signal discoveries caused by background model underestimation in the signal region.
2. Uncertainty Trade-off: It explicitly trades systematic model bias for statistical uncertainty (reduced effective statistics), a trade-off the authors argue is preferable to unquantified systematic errors.
3. Diagnostic Power: It provides a quantitative metric ($L_{min}$ and the associated fraction difference) to assess the reliability of a background model before drawing physics conclusions.

The authors are modest about the method's scope, noting it is not a "be-all, end-all" solution. They acknowledge that for very low signal strengths, the method may be too conservative, potentially masking real signals. They suggest future work could explore softer cuts (masking) rather than hard exclusions and further testing with higher statistics to refine the definition of $L_{min}$. The method is envisioned to be applicable not only to data-driven backgrounds but also as a diagnostic for Simulation-Based Inference (SBI) and Anomaly Detection tasks.