Robust Sequential Experimental Design for A/B Testing

Technical Summary: Robust Sequential Experimental Design for A/B Testing

Problem Statement
A/B testing is a cornerstone of data-driven decision-making in technology, yet standard independent random assignment often suffers from finite-sample covariate imbalances, leading to inefficient estimation of the Average Treatment Effect (ATE). While sequential experimental designs have been developed to maintain covariate balance, existing approaches face two critical limitations:

1. Model Sensitivity: They typically rely on correctly specified linear models with additive treatment effects and specific covariate distributions. In practice, true data-generating processes often contain unknown nonlinear components, rendering these designs suboptimal or biased.
2. Myopia and Carryover Effects: Many sequential designs are "myopic," optimizing only immediate imbalance without considering how current assignments influence future covariate states. Furthermore, they often ignore carryover effects (where past treatments influence future outcomes), which are prevalent in dynamic settings like ride-sharing or time-series experiments.

Methodology
The paper proposes a unified framework for Robust Sequential Design (RSD) that addresses model misspecification and carryover effects through three core components:

1. Orthogonalization for Robustness:
   The authors derive a worst-case Mean Squared Error (MSE) upper bound for the ATE estimator under a misspecified linear working model. By employing an orthogonalization technique, they decompose the MSE into a variance term and a bias term arising from model misspecification. They approximate the unknown nonlinear error function using a sieve representation and enforce orthogonality conditions to derive an upper bound that depends only on the design and known basis functions, rather than the specific unknown nonlinear coefficients. This allows the design to hedge against model misspecification without needing to know the exact form of the nonlinearity.

2. Dynamic Programming (DP) for Sequential Allocation:
   To optimize the derived MSE upper bound in a sequential setting, the problem is formulated as a Dynamic Programming (DP) task. The state is defined by low-dimensional summary statistics (imbalance measures) of the covariates and basis functions. The value function represents the expected future MSE. Since exact backward induction is computationally infeasible for high-dimensional states, the authors approximate the value functions using Deep Neural Networks (DNNs) trained via synthetic rollouts. This enables the algorithm to make allocation decisions that account for the entire history of observations, thereby mitigating finite-sample randomness.

3. Hierarchical Design for Dynamic Settings:
   For environments with carryover effects (e.g., daily experiments with multiple time intervals), the authors propose a hierarchical scheme:

   • Macro-level (Across-day): A DP approach optimizes the day-level objective, treating each day as a single decision stage.
   • Micro-level (Within-day): Reinforcement Learning (RL), specifically an actor-critic algorithm, is used to determine optimal treatment assignments within a day, maximizing the negative of the day-level loss.
     This decomposition allows the method to handle complex temporal dependencies and large state spaces (e.g., hundreds of decision stages) that are intractable for standard DP.

Key Contributions

• Robustness to Misspecification: The framework explicitly accounts for unknown nonlinear components in the data-generating process, providing a theoretical guarantee on the worst-case MSE of the ATE estimator without requiring correct model specification.
• Finite-Sample Awareness: Unlike designs relying on large-sample approximations (e.g., Neyman allocation), the proposed method utilizes the full data history to directly mitigate finite-sample randomness in covariate distributions.
• Unified Framework: The approach covers both contextual bandit settings (no carryover) and dynamic settings (with carryover effects), offering a single methodology for diverse experimental environments.
• Algorithmic Scalability: By combining DP with deep learning approximations and a hierarchical DP-RL structure, the method scales to practical experimental sizes and complexities.

Experimental Results
The authors evaluate the proposed RSD using synthetic data and real-world data from a leading ride-sharing company.

• Contextual Bandits: In simulations with both additive and interactive treatment effects under nonlinear data-generating processes, RSD consistently achieves the lowest MSE compared to baselines (including non-robust DP, random designs, switchback designs, and biased-coin designs). The advantage is particularly pronounced in small-sample settings.
• Dynamic Settings: In dynamic environments with varying degrees of nonlinearity and time horizons, RSD outperforms state-of-the-art baselines like TMDP (Time-series MDP) and NMDP. The method remains stable even as the degree of model misspecification increases.
• Real-World Simulation: Using a simulation environment calibrated on real ride-sharing data, RSD demonstrates superior performance over existing methods, confirming its practical effectiveness.
• Efficiency: Under correct model specification, RSD incurs only a minimal efficiency loss (approx. 97–99% relative efficiency) compared to an oracle non-robust design, indicating that the robustness comes at a negligible cost when the model is correct.

Significance and Claims
The paper claims to advance the literature on experimental design by bridging the gap between robust statistics and sequential decision-making. It argues that existing sequential designs are inherently fragile to model misspecification and often fail to account for the long-term impact of current assignments on future states. By providing a unified framework that is robust, finite-sample aware, and versatile across static and dynamic settings, the authors offer a more reliable approach for A/B testing in complex, real-world scenarios where model assumptions are frequently violated. The work emphasizes that while A/B testing is routine, the design of these experiments has often been overlooked in favor of estimation techniques, and this paper seeks to rectify that by improving the data generation process itself.