Searching for New Physics with Reinforcement Learning

Technical Summary: Searching for New Physics with Reinforcement Learning

Problem Statement
The primary challenge addressed in this work is the identification of New Physics (NP) within the framework of the Standard Model Effective Field Theory (SMEFT). While the Standard Model (SM) is successful at TeV scales, it fails to explain phenomena such as neutrino masses and dark matter. When experimental anomalies arise (discrepancies between low-energy observables and SM predictions), physicists must determine which SMEFT operators can explain them.

This task is computationally and logically difficult for two reasons:

1. Combinatorial Explosion: The Warsaw basis contains 59 baryon-number-conserving dimension-6 operators, which expand to 2,499 operators when individual flavor indices are counted.
2. Complex Correlations: Operators are not independent. Through renormalization-group (RG) running, a single SMEFT operator contributes to many Weak Effective Theory (WET) operators at the one-loop level. Conversely, a specific WET operator required to explain an anomaly may be generated by numerous SMEFT operators.

Human analysis typically relies on phenomenological intuition to select relevant operators. However, as the number of anomalies increases, the search space becomes too vast for intuition or trial-and-error methods to navigate effectively without bias toward specific UV theories.

Methodology
The authors formulate the search for NP operators as a Reinforcement Learning (RL) problem. The goal is to learn a policy that maps the current "state" of the fit to an "action" (proposing a new SMEFT operator) to maximize a "reward" (improvement in the global fit).

• RL Mapping:

  • State ($s_t$): The set of currently proposed and accepted SMEFT operators, the individual pulls of the observables, and the current minimum $\chi^2$ value.
  • Action ($a_t$): Proposing a specific SMEFT operator from the space of 912 baryon- and lepton-number-conserving operators.
  • Environment: A computational pipeline using flavio to calculate theoretical predictions and Minuit to perform a global fit. The environment evaluates the proposed operator by finding the Wilson Coefficient (WC) that minimizes $\chi^2$.
  • Reward ($r_t$): Defined as $r_t = \Delta\chi^2 - c_P$, where $\Delta\chi^2 = \chi^2_{SM} - \chi^2_{min}$ represents the improvement in the fit. A penalty $c_P$ is applied for "dead steps" where the operator fails to reduce $\chi^2$ by a threshold (0.001).

• Algorithm:
  The authors employ an Actor-Critic algorithm parameterized by a transformer-based neural network.

  • The Actor ($\pi_\theta$) proposes operators based on the current state.
  • The Critic ($V_\phi$) estimates the expected future return (value) of the state to guide the policy updates.
  • The training objective minimizes a combined loss: $L_{total} = L_{actor} + \lambda_V L_{critic}$.
  • The policy is trained over trajectories of length $T=10$, where a sequence of $T$ operators is proposed.

The method is designed to be "bottom-up," requiring no prior knowledge of which operators are relevant; it relies solely on experimental data and theoretical predictions to drive the search.

Key Contributions and Results
The paper validates the RL method through two distinct case studies:

1. Reproduction of Known Results (CDF W-Mass Anomaly):

   • The method was tested on the CDF W-mass anomaly using 27 Z-pole observables and the W-mass measurement.
   • The RL policy successfully identified the operator combination $[O_{ll}]_{1221}$ and $[O_{HD}]$, reproducing the results of previous literature (Ref. [25]) with a $\Delta\chi^2 = 27.6$.
   • Improvement: The RL method discovered a new viable operator combination ($[O_{lu}]_{1133}$ and $[O_{Hu}]_{33}$) with $\Delta\chi^2 = 20.6$, which was missed in previous analyses. The authors attribute this to the RL policy's ability to systematically explore the full SMEFT space including RG running effects, which were neglected in the reference study.

2. Complex Multi-Anomaly Scenario:

   • A synthetic dataset was created with 32 observables spanning flavor and electroweak sectors, including 10 "fictitious" anomalies (deviations of 3–5$\sigma$ from the SM).
   • Comparison: The Actor-Critic policy was compared against a "No Critic" (REINFORCE-only) baseline and a uniform random search.
   • Performance:
     • The Actor-Critic policy converged faster and achieved a higher final average reward ($\sim 153$) compared to the No Critic policy ($\sim 141$).
     • In a full search space of 912 operators, the RL policy found 560 unique operator sets with $\Delta\chi^2 \ge 120$ and 26 sets with $\Delta\chi^2 \ge 150$. In contrast, a random search of equal size (75,000 samples) found zero solutions.
     • Even in a reduced search space (219 operators), the RL policy significantly outperformed random sampling, finding 876 sets with $\Delta\chi^2 \ge 120$ versus 46 for random search.
   • The best solution found by the policy in the full space consisted of 10 operators with a $\Delta\chi^2$ of 156.4, resulting in a $\chi^2_{min}/d.o.f. = 1.0$.

Significance
The paper claims that Reinforcement Learning offers a robust, unbiased, and efficient alternative to human intuition for navigating the vast and highly correlated SMEFT operator space. The authors demonstrate that:

• RL can reproduce known physics results.
• RL can discover non-trivial solutions that human intuition or simplified searches might miss, particularly when multiple anomalies and complex correlations are involved.
• The method scales effectively to high-dimensional search spaces where random sampling fails completely.

The work establishes a framework where AI can autonomously perform the "bottom-up" analysis of anomalies, systematically identifying the SMEFT operators that explain experimental data without relying on preconceived theoretical biases.