📊 statistics

adabay: an R package for rapid evaluation and calibration of Bayesian group sequential designs across common endpoint types

The paper introduces **adabay**, an open-source R package that enables the rapid evaluation and calibration of Bayesian group sequential designs across various endpoint types by utilizing a semi-simulation framework that combines Monte Carlo data path simulation with analytical or deterministic posterior computations, achieving speed improvements of several orders of magnitude over existing tools while maintaining accuracy.

Original authors: Zhangyi He, Feng Yu

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhangyi He, Feng Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of a single clue, you are collecting a stream of evidence over time. In the world of medicine, this is what happens during a clinical trial: doctors test a new treatment on patients, checking in at specific moments to see if the drug is working or if it's a waste of time. This is called a "group sequential design." The goal is to stop the trial early if the answer is clear, saving money, time, and sparing patients from unnecessary exposure to a bad drug.

For decades, scientists have used a strict, rule-based method (called "frequentist") to decide when to stop. It's like following a rigid recipe: if the evidence hits a specific number, you stop. But there's a newer, more flexible way called "Bayesian" design. Instead of just counting numbers, Bayesian methods ask, "What is the probability that this drug works?" It's like having a detective who updates their gut feeling every time a new clue arrives, allowing them to incorporate outside knowledge and make decisions based on how likely something is to be true. This sounds great, but there's a catch: figuring out if these flexible Bayesian rules are safe and fair is incredibly hard to calculate. It's like trying to predict the weather for a thousand different future scenarios, where each prediction requires running a complex computer simulation inside another simulation. It's so slow that it often feels impossible to use these smart methods for real-world drug testing.

This is where a new tool called adabay comes in. Think of adabay as a high-speed turbocharger for these Bayesian detectives. The authors, Zhangyi He and Feng Yu, built a free software package that lets researchers design and test these flexible trials in seconds instead of days. They achieved this by using a clever trick: instead of running the heavy, slow computer simulations for every single step of the trial, they use a "semi-simulation" approach. They simulate the flow of patients and data as usual, but when it comes to calculating the probability of the drug working, they use a mathematical shortcut that is almost instant. It's like having a super-fast calculator that can instantly tell you the odds of winning a game, whereas the old way required you to play the game a million times just to get the odds.

The paper shows that adabay is not just fast; it's also accurate. The authors tested it against other existing software tools and found that it produces the same correct answers but runs thousands of times faster. For example, while an older tool might take hours to simulate a single trial, adabay can do it in a fraction of a second. They also showed that this new tool works for all the common types of medical data, including continuous measurements (like blood pressure), yes/no outcomes (like whether a patient survived), counts (like the number of infections), and even time-to-event data (like how long it takes for a tumor to shrink).

The most exciting part is that adabay doesn't just speed things up; it makes the "smart" Bayesian approach actually usable for real drug trials. Before this, the computational cost was so high that many researchers avoided these flexible designs. Now, with adabay, they can quickly test different scenarios, find the perfect stopping rules, and ensure the trial is safe and efficient. The authors demonstrated this by recreating real-world trials, like those for depression and cancer, and showing that their tool can handle the complexity without breaking a sweat. It's a game-changer that turns a slow, clunky process into a rapid, precise one, potentially helping bring life-saving treatments to patients faster.

Technical Summary of adabay: an R package for rapid evaluation and calibration of Bayesian group sequential designs across common endpoint types

Problem Statement
Bayesian group sequential designs (GSDs) offer significant advantages over frequentist approaches, including clinically interpretable probability statements and principled incorporation of external evidence. However, their adoption in confirmatory Phase III trials has been hindered by the computational cost of evaluating frequentist operating characteristics (OCs), particularly the Type I error rate. Traditional calibration requires nesting Markov chain Monte Carlo (MCMC) or other approximate posterior inference within a Monte Carlo trial-simulation loop. This "MCMC-in-MCMC" structure is computationally prohibitive, often rendering the evaluation of large design grids infeasible. Existing software solutions either lack flexibility in prior specification (e.g., restricted to conjugate families), lack coverage of common endpoint types (e.g., time-to-event), or remain orders of magnitude slower than conjugate-based methods.

Methodology: The Semi-Simulation Framework
The paper introduces adabay, an open-source R package implementing a semi-simulation framework that decouples trial data generation from posterior computation. The core methodology, detailed in a companion paper (He et al., 2026), relies on three pillars:

Monte Carlo Data Generation: Trial data paths are generated via standard Monte Carlo simulation.
Analytic or Deterministic Posterior Updates: Instead of MCMC, per-look posteriors and tail probabilities are computed analytically or via low-dimensional deterministic quadrature. This is achieved by approximating any user-specified prior (provided it can be sampled or evaluated numerically) as a finite mixture of conjugate components.
Conjugate Mixture Approximation: The package approximates non-conjugate priors using a finite mixture of conjugate kernels (e.g., Beta, Normal, Gamma, Normal-Inverse-Gamma). The mixture weights and hyperparameters are fitted by minimizing an empirical forward Kullback–Leibler (KL) divergence. Crucially, the package includes diagnostics to flag inadequate approximations at decision thresholds, issuing runtime warnings if tail-probability errors exceed tolerances.

Key Contributions
The adabay package provides a unified interface for Bayesian GSDs across four primary endpoint types: continuous, binary, count, and time-to-event. Its specific technical contributions include:

Unified Interface: A single programming API handling absolute and relative effect scales across all four endpoint types.
Flexible Prior Specification: Support for arbitrary user-specified priors via the finite-mixture approximation, preserving conjugacy at each look. This allows for the analytic or one-dimensional deterministic evaluation of posterior tail probabilities.
Complex Decision Rules: Support for posterior-probability decision rules with multiple efficacy and futility criteria, configurable per look, under both binding and non-binding futility regimes.
Precomputation and Caching Architecture: A strategy that decouples threshold calibration and look-time selection from the simulation pass. By caching per-look posterior tail probabilities over a grid of candidate thresholds, the package allows subsequent design evaluations to occur at negligible marginal cost (milliseconds) after a single simulation pass.
Reproducibility: Implementation of parallel simulation using L'Ecuyer's combined multiple-recursive generator, ensuring bit-identical reproducibility across different worker counts when explicitly requested.

Results and Benchmarks
The paper validates adabay against existing tools (gsbDesign, BATSS, and adaptr) and analytic references across four case studies:

Accuracy: In continuous and binary case studies, adabay reproduces the operating characteristics of BATSS and adaptr within Monte Carlo error. For continuous endpoints, it matches the analytic values from gsbDesign to within 0.05 percentage points for Type I error and power. In the count case study, it reproduces BATSS results within Monte Carlo error.
Time-to-Event: adabay is the only tool among the comparators to support time-to-event endpoints under the specified conventions, as BATSS (GLM-only interface) and adaptr (Gaussian/Binomial only) do not support this endpoint type.
Computational Speed: The semi-simulation framework yields dramatic speedups. adabay runs approximately four to five orders of magnitude faster than BATSS and one to over two orders of magnitude faster than adaptr per virtual trial on eight cores.
- Example: For a binary endpoint with $10^4$ trials, adabay completed the evaluation in 1.6 seconds, compared to 62.7 seconds for adaptr and ~~32,317 seconds (~~9 hours) for BATSS.
Calibration Efficiency: The caching architecture allows a grid of 800 design configurations to be evaluated in ~34 seconds (including the initial simulation pass), whereas direct simulation would require ~5,500 seconds.

Significance and Claims
The authors claim that adabay addresses a critical unmet need in clinical trial design by providing a tool that simultaneously offers conjugate-speed posterior updates, flexible prior specification beyond natural conjugate families, unified coverage of the four most common endpoint types, and a scalable calibration architecture.

The paper modestly asserts that adabay lowers the computational barrier to deploying Bayesian GSDs in confirmatory trials. It does not claim to solve all design challenges; limitations are acknowledged, including the approximation error inherent in the mixture prior (mitigated by diagnostics), the continued reliance on simulation-based calibration (subject to Monte Carlo error), and the current restriction to two-arm fixed-allocation designs. The authors position the package as a practical engineering solution that makes the semi-simulation framework accessible for routine use, enabling the evaluation of large design grids that were previously computationally intractable.

Technical Summary of adabay: an R package for rapid evaluation and calibration of Bayesian group sequential designs across common endpoint types

More like this