Technical Summary: Optimizing Social Utility in Sequential Experiments

Problem Statement

In high-stakes domains such as drug development, regulatory approval relies on Randomized Controlled Trials (RCTs) to establish safety and efficacy. However, the substantial financial costs of these trials often deter developers from pursuing "moonshot" products with high potential social utility, particularly when they lack absolute certainty regarding a product's efficacy. While adaptive trials and targeted subsidies have been proposed to mitigate this, their effectiveness depends on how developers strategically respond to incentives. Existing literature often models regulatory approval as a principal-agent game but typically focuses on static hypothesis testing or contract design to disincentivize participation by agents with ineffective products. There is a gap in understanding how to optimally subsidize sequential experimentation when both the regulator (principal) and the developer (agent) are uncertain about the product's true efficacy, aiming to maximize social utility rather than just controlling error rates.

Methodology

The authors propose a statistical protocol for sequential experimentation where the agent conducts a sequence of RCTs, and the principal (regulator) partially subsidizes the experimentation costs conditional on approval. The framework is built upon three core components:

1. Sequential Hypothesis Testing with e-values:
   Instead of traditional p-values, the protocol utilizes e-values to ensure "any-time validity." The principal sets a baseline efficacy $\theta_b$ and a false positive rate bound $\kappa$. At each step $t$, the agent selects a sample size $n_t$. The principal computes an e-value $E(X_t, n_t)$ based on the outcome $X_t$ (successes in the trial). These e-values are aggregated multiplicatively into a test process $M_t$. The null hypothesis $H_0$ (efficacy $\theta^* < \theta_b$) is rejected (and the product approved) as soon as $M_t \geq 1/\kappa$. This structure guarantees that the probability of a false positive is bounded by $\kappa$ regardless of when the experiment is stopped.

2. Belief Markov Decision Process (MDP) for the Agent:
   Since the agent does not know the true efficacy $\theta^*$, they act based on their posterior belief, modeled as a Beta distribution $B_t = \text{Beta}(\alpha_t, \beta_t)$. The agent's decision problem is modeled as a belief MDP ($M_\epsilon$) where:

   • State: Characterized by belief parameters $(\alpha, \beta)$, accumulated cost $C$, and time step.
   • Action: Selecting a sample size $n$ or opting out ($n=0$).
   • Transition: Updates the belief via Bayesian inference and accumulates cost.
   • Reward: Includes the benefit of approval ($\rho_A$) plus the subsidy ($\epsilon$) minus the cost of the trial.
     The agent seeks a policy $\pi$ that maximizes their anticipated utility. The authors show that the optimal policy can be computed efficiently using dynamic programming (value iteration).

3. Principal's Optimization via Divide-and-Conquer:
   The principal aims to select a subsidy level $\epsilon$ that maximizes social utility ($U^S$), defined as the social benefit of approval ($\rho_S$) minus the expected subsidy cost.

   • Structural Properties: The authors prove that the agent's optimal anticipated utility is a piecewise linear and convex function of the subsidy $\epsilon$. Consequently, the principal's anticipated social utility is a piecewise linear function over intervals where the agent's optimal policy remains constant.
   • Algorithm: Leveraging these properties, the authors develop a divide-and-conquer algorithm (Algorithm 1) to find the optimal subsidy $\epsilon^*$. The algorithm iteratively identifies the "vertices" of the piecewise linear utility function by solving the agent's MDP at candidate subsidy points and checking for policy changes.

Key Contributions

1. Novel Protocol: Introduction of a sequential approval protocol combining e-value-based hypothesis testing with partial cost subsidies, designed to align the incentives of developers and regulators under uncertainty.
2. Computational Efficiency:
   • Demonstration that the agent's optimal strategy in the belief MDP can be found efficiently using dynamic programming, with a state space size of $O((n_{max})^2 \cdot T^3)$.
   • Proof that the social utility function is piecewise linear and convex with respect to the subsidy, enabling the use of an efficient divide-and-conquer algorithm to find the socially optimal subsidy without exhaustive search.
3. Theoretical Characterization:
   • Establishment of the monotonicity of the optimal value function with respect to belief parameters and cost.
   • Proof that the agent's optimal policy exhibits a threshold structure regarding the decision to opt out.
   • Demonstration that the agent's anticipated utility is linear in the subsidy within specific intervals, allowing for the decomposition of the value function.

Experimental Results

The authors validated the protocol using simulation experiments based on publicly available data regarding antibiotic development and approval.

• Setup: The scenario involved an antibiotic with a true efficacy $\theta^* = 0.65$ and a baseline $\theta_b = 0.5$. The agent could conduct up to four trials with a maximum sample size of 200. Costs were modeled based on real-world estimates (fixed cost \approx \48.9$M, per-patient cost \approx \66$k).
• Findings:
  • Under the optimal subsidy ($\epsilon^* \approx 0.108$), the protocol increased social utility by approximately 5.5% compared to a no-subsidy scenario and reduced the agent's probability of opting out by ~22%.
  • When compared to standard non-sequential protocols (where the agent is restricted to a single trial), the sequential subsidized protocol yielded substantial gains. Specifically, it increased social utility by ~50–60% relative to non-sequential protocols without subsidies, and by >35% relative to non-sequential protocols with optimal subsidies.
  • The optimal subsidy was found to increase with the ratio of social benefit to agent benefit ($\rho_S/\rho_A$), reflecting stronger incentives for the principal to subsidize when social stakes are higher.
  • The algorithm was computationally efficient, solving for the optimal subsidy in approximately 271 seconds for the fiducial setting.

Significance and Claims

The paper claims to provide a principled framework for designing subsidy mechanisms that improve social utility in regulatory approval processes. By modeling the interaction as a Stackelberg game within a belief MDP, the authors demonstrate that it is possible to:

• Incentivize experimentation for high-potential but uncertain products without compromising statistical rigor (via e-values).
• Compute optimal strategies and subsidies efficiently, making the approach computationally tractable for real-world application.
• Substantially outperform traditional non-sequential designs, particularly in scenarios characterized by market failure and underinvestment (such as antibiotic development).

The authors position this work as a contribution to the broader direction of regulatory agencies and developers exploring flexible, data-driven approval methodologies (including Bayesian approaches) to better balance safety and innovation. They explicitly note that while the current model uses a Beta-Binomial framework, the approach can be extended to more general settings, though this introduces computational challenges. The paper concludes that their sequential, subsidized protocol offers a viable path to aligning agent incentives with social objectives, potentially informing future policy discussions.