Neural Fake Factor Estimation Using Data-Based Inference

Technical Summary: Neural Fake Factor Estimation Using Data-Based Inference

Problem Statement
In high-energy physics (HEP) analyses, "fake" backgrounds arise from events that fail formal signal selection criteria but are accepted due to mis-reconstructed or mis-identified particles, such as non-prompt leptons or hadronic jets mistaken for leptons. Traditionally, these backgrounds are estimated using data-driven techniques, most notably the Fake Factor method. This method extrapolates the fake lepton contribution from a kinematically adjacent, looser selection region (Control Region, CR) to the Signal Region (SR) using a scale factor (the "fake factor").

The conventional implementation of this method relies on binned estimation, where the fake factor is calculated as the ratio of two histograms (tight vs. loose selections) in a low-dimensional space (typically transverse momentum $p_T$ and pseudorapidity $\eta$). This approach faces several limitations:

1. Binning Artifacts: The choice of binning significantly impacts results; coarse bins lose kinematic features, while fine bins suffer from statistical fluctuations, empty bins, or negative values.
2. Dimensionality Limits: Due to limited statistics, the method is typically restricted to a few variables, preventing the capture of complex correlations with other event topology variables (e.g., missing transverse energy $E^{miss}_T$ or jet multiplicity).
3. Extrapolation Uncertainty: Discontinuities caused by binning and the inability to model high-dimensional dependencies degrade the accuracy of extrapolating background estimates to the signal region.

Methodology
The authors propose a novel Machine Learning (ML)-based Fake Factor method that replaces histogramming with neural density ratio estimation. This approach, termed Data-Based Inference (DBI), estimates a continuous, unbinned fake factor function on a per-event basis.

The method is structured in two primary steps:

1. Subtraction Step (Real Lepton Removal):
   Since the fake factor must be derived from fake leptons only, the contribution of real (prompt) leptons must be subtracted from both the tight and loose data samples. The authors train two independent binary classifiers to estimate the ratio of data to Monte Carlo (MC) simulation in the tight and loose regions separately ($r_{T,L} = N^{data}/N^{MC}$).

   • These classifiers are trained to distinguish data events (label 1) from MC events (label 0).
   • The output is used to reweight data events (or MC events) to obtain "real-subtracted" densities.
   • To ensure physical validity (positive weights), a soft absolute activation function is applied to the classifier's logit output, ensuring the ratio $r > 1$ and the resulting weights remain positive.

2. Ratio Step (Fake Factor Estimation):
   A third binary classifier is trained to distinguish between the tight (numerator) and loose (denominator) real-subtracted samples.

   • The training dataset consists of reweighted events from both regions.
   • The classifier learns the likelihood ratio between the two hypotheses.
   • The final fake factor $F(x)$ for an event with features $x$ is estimated as the exponential of the classifier's output: $F(x) = \exp(q(x))$.
   • This yields a continuous function dependent on a high-dimensional feature space (e.g., $p_T, \eta, E^{miss}_T, N_{jets}, m_T$).

Model Architecture and Training

• Architecture: The authors utilize a pre-activation ResNet with four residual blocks, each containing two layers of 128 neurons. This architecture mitigates vanishing gradients and allows for stable training of deeper networks compared to standard feed-forward networks.
• Input Processing: Numerical features are standardized, and categorical features are label-encoded and embedded. An embedding layer maps features to a higher-dimensional space, followed by mean pooling.
• Loss Function: The training uses binary cross-entropy with a squared regularization term to prevent exploding densities. For the subtraction classifiers, a soft absolute activation ensures non-negative outputs; for the ratio classifier, a linear activation is used.
• Training: The model is trained using the AdamW optimizer with early stopping based on validation loss.

Key Contributions

1. Continuous, Unbinned Estimation: The method provides a per-event fake factor, eliminating binning artifacts and discontinuities inherent in histogram-based methods.
2. High-Dimensional Flexibility: By leveraging neural networks, the method can incorporate multiple correlated kinematic variables simultaneously, capturing complex dependencies that traditional binned methods cannot due to the "curse of dimensionality."
3. Improved Extrapolation: The continuous nature of the estimator allows for smoother and more stable extrapolation from the control region to the signal region.
4. Validation Framework: The authors demonstrate a robust two-step validation procedure (subtraction and ratio) using ATLAS Open Data, ensuring the method correctly handles real-lepton contamination.

Results
The method was validated using an analysis of $W \to e\nu$ events from ATLAS Run 2 data.

• Control Region (CR): The ML-based method showed good agreement with the traditional binned method in the CR. While the binned method performed slightly better in low-$p_T$ regions with high statistics, the ML method demonstrated superior modeling in variables like $E^{miss}_T$ and $m_T$, which are difficult to include in binned analyses due to statistical constraints.
• Signal Region (SR): When extrapolating to the SR ($m_T > 60$ GeV), the ML-based method provided significantly better predictions in both shape and normalization compared to the binned method. The binned method exhibited larger discrepancies and systematic mis-modeling, particularly in distributions of $E^{miss}_T$ and $m_T$, due to its reliance on coarse binning and limited variable inclusion.
• Stability: The ML approach produced smoother distributions with reduced statistical fluctuations, particularly in regions with lower event counts or complex correlations.

Significance and Claims
The paper claims that the ML-based Fake Factor method represents a significant advancement in data-driven background estimation for high-energy physics. By moving from discrete, low-dimensional binning to continuous, high-dimensional density ratio estimation, the method:

• Mitigates common limitations such as binning selection bias and extrapolation uncertainties.
• Enhances the ability to model complex correlations between variables.
• Improves the sensitivity of searches for rare signals by providing more accurate background estimates, thereby reducing the risk of spurious signals arising from mis-modeling.

The authors emphasize that while the method was demonstrated on a simple $W$-boson analysis, its framework is inherently adaptable to multi-lepton final states and other mis-identified objects. They note that future work will focus on integrating systematic uncertainty estimation and applying the method to more complex LHC analyses searching for new physics. The code for the implementation is made publicly available.