Bayes Factor Group Sequential Designs

Technical Summary: Bayes Factor Group Sequential Designs

Problem Statement
Sequential designs, which allow for early stopping based on interim analyses, are highly desirable in fields like clinical trials and animal experiments to reduce sample sizes and ethical burdens. While Bayesian methods, particularly those utilizing Bayes factors, are naturally suited for sequential analysis due to their inherent updating mechanism and lack of sensitivity to multiplicity issues, their practical adoption has been hindered by computational challenges. Specifically, designing sequential Bayes factor studies requires determining key characteristics such as the probability of obtaining conclusive evidence (analogous to power and type-I error) and the expected sample size. Currently, computing these metrics typically relies on computationally intensive Monte Carlo simulations because no closed-form or efficient numerical methods exist. This reliance on simulation makes prototyping and comparing different design options time-consuming and subject to Monte Carlo error, limiting the ability of researchers to optimize study designs.

Methodology
The authors propose a novel framework that extends classical group sequential design (GSD) theory to sequential Bayes factor designs, enabling fast, accurate, and simulation-free computation of design characteristics. The core methodological innovation involves three steps:

1. Mapping Bayes Factors to Z-Statistics: The authors express the Bayes factor ($BF_{01}$) as a function of the $z$-statistic ($z = (\hat{\theta} - \theta_0)/\sigma$), where $\hat{\theta}$ is the parameter estimate and $\sigma$ is its standard error. For various common Bayes factor types (e.g., point null vs. directional alternative, point null vs. point alternative), they derive critical $z$-values corresponding to specific evidence thresholds ($k_0$ and $k_1$). In cases where a closed-form solution for the critical $z$-value is unavailable, numerical root-finding is employed.
2. Defining Stopping Regions: Based on these critical values, the authors define stopping regions in the space of accumulating $z$-statistics. For Bayes factors with a single critical value, these regions form $i$-dimensional hyper-rectangles. For those with two critical values (e.g., two-sided alternatives), the regions consist of unions of hyper-rectangles. The probability of stopping at a specific interim analysis is equivalent to the probability of the vector of cumulative $z$-statistics falling within these specific regions.
3. Multivariate Normal Integration: Assuming the vector of accumulating $z$-statistics follows a canonical multivariate normal distribution (or a marginal prior-predictive distribution when a design prior is specified), the authors compute the probabilities of these stopping regions using multivariate normal integration. This approach utilizes recursive one-dimensional integration algorithms or modern implementations of multivariate normal integration (e.g., via the mvtnorm R package) to calculate stopping probabilities deterministically.

The framework accommodates both fixed parameter values (frequentist operating characteristics) and design priors (Bayesian predictive power/assurance) by adjusting the mean and covariance structure of the $z$-statistic distribution.

Key Contributions and Results

• Simulation-Free Computation: The primary contribution is a method that eliminates the need for Monte Carlo simulations. The authors demonstrate that design characteristics can be computed in seconds, a significant reduction compared to the minutes or hours required for simulation-based approaches.
• Open-Source Implementation: The method is implemented in the open-source R package bfpwr, making the planning of sequential Bayes factor designs as straightforward as classical GSDs.
• Application to Diverse Scenarios: The paper illustrates the method through several case studies:
  • Low-PV Clinical Trial: Re-analyzing a trial on ropeginterferon alfa-2b, the authors show how to compute the probability of conclusive evidence and determine the required sample size to achieve 90% probability of correct evidence. They demonstrate that increasing the number of interim analyses can improve efficiency but must be balanced against the risk of misleading evidence.
  • Rat Weight Loss Experiment: In a preclinical setting, the method is used to plan a replication study. By using a design prior based on original data, the authors show how to optimize sample sizes across different dose groups, potentially saving animals while maintaining high probabilities of informative inferences.
  • Sequential Bayes Factor t-Test: The framework is extended to the sequential $t$-test (including the Jeffreys-Zellner-Siow Bayes factor). The authors replicate a psychological study design involving 61 interim analyses, showing that their method yields results nearly identical to simulation-based estimates (e.g., 70.3% vs. 70.6% for stopping for $H_1$) but with drastically reduced computation time.

Significance and Limitations
The paper claims that this approach makes exploring sequential Bayes factor designs "as straightforward as classical group sequential designs," thereby removing a major barrier to their adoption. By providing a unified framework that handles both Bayesian predictive characteristics and frequentist operating characteristics, the method facilitates the design of informative and efficient experiments.

The authors remain modest regarding the scope of their method. They acknowledge that the approach relies on the assumption that the Bayes factor can be expressed as a function of the $z$-statistic and that the accumulating statistics follow (at least approximately) a canonical multivariate normal distribution. Consequently, the method may not apply directly to scenarios with small sample sizes and binary data or extreme probabilities, where these assumptions might fail, potentially necessitating a return to simulation or other numerical methods. Future research directions include developing modifications for these non-canonical situations.