📊 statistics

E-values for Adaptive Clinical Trials: Anytime-Valid Monitoring in Practice

This paper serves as a practical methodology guide for implementing anytime-valid e-values in adaptive clinical trials, demonstrating through numerical studies and the open-source R package `evalinger` how they provide robust Type I error control under continuous monitoring while offering a viable alternative to traditional group sequential and Bayesian approaches.

Original authors: Alexandra Sokolova, Vadim Sokolov

Published 2026-02-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexandra Sokolova, Vadim Sokolov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge overseeing a long-running trial to see if a new medicine works better than a placebo. In the old days, statisticians said, "Wait until the very end, count all the patients, and then make one final decision." But in real life, trials don't work that way. Doctors and safety committees want to peek at the data as it comes in to stop early if the drug is amazing (to save lives) or if it's clearly failing (to stop wasting resources).

The problem with "peeking" is that it's like rolling a die over and over. If you roll a six once, that's luck. If you roll a six ten times in a row, you might think the die is loaded. But if you keep rolling until you finally get a six, you trick yourself into thinking you found a pattern when it was just random noise. In statistics, this is called "Type I error" (a false alarm).

This paper introduces a new tool called E-Values (Evidence Values) to solve this problem. Here is how it works, using simple analogies:

1. The Betting Game (The Core Idea)

Imagine you are a gambler betting against the idea that the new drug does nothing (the "Null Hypothesis").

The Setup: You start with $1. Every time a new patient finishes the trial, you get to bet a portion of your money on whether the drug worked.
The Rule: If the drug doesn't work (the Null is true), the game is fair. On average, you will neither win nor lose money. Your wealth will bounce around but stay near $1.
The Win: If the drug does work, your bets will start paying off. Your wealth (your "E-value") will grow.
The Threshold: You decide in advance: "If my wealth grows to $40, I will declare the drug a winner."

Why is this special?
In traditional statistics, if you check your bank account every day, you have to lower your winning threshold every time you look, or you'll eventually get lucky and think you won when you didn't.
With E-values, you can check your bank account as often as you want. Even if you look every second, the math guarantees that if the drug is fake, your wealth will almost never reach $40. It's like a "magic" bet that stays fair no matter how many times you peek.

2. The Three Competitors

The paper compares three different ways to run this trial:

The "Naive" Approach (The Reckless Gambler): This is checking the data and saying, "Hey, the numbers look good!" without any special math. The paper shows this is dangerous. If you peek 20 times, you will falsely declare a winner about 15% of the time (when the drug is actually useless).
The "Group Sequential" Approach (The Strict Accountant): This is the current standard. You plan exactly when you will look (e.g., at 25%, 50%, and 75% of the trial). You have a strict budget for how much "risk" you can spend at each look.
- Pros: It's very powerful (good at finding real cures) if you stick to the plan.
- Cons: If you change the plan or look at the data at weird times, the math breaks, and you have to start over.
The "E-Value" Approach (The Flexible Watchman): This is the paper's focus. It uses the betting game described above.
- Pros: You can look at the data whenever you want (even continuously, like a live stream). You don't need a pre-set schedule. It's "anytime-valid."
- Cons: To be safe enough to allow unlimited peeking, it is slightly less "powerful" than the Strict Accountant when you do stick to a fixed schedule. It takes a bit longer to reach the $40 threshold.

3. What the Numbers Say

The authors ran thousands of computer simulations to see how these methods performed:

Fixed Schedule: If you plan to look exactly 20 times, the "Strict Accountant" (Group Sequential) finds the truth about 86% of the time. The "Flexible Watchman" (E-Value) finds it about 72% of the time. The E-Value is slightly slower, but it's still very good.
Continuous Monitoring: If you look at the data every single time a patient finishes (which is impossible for the Strict Accountant without breaking the rules), the Strict Accountant's power crashes to 10% (it becomes too scared to call a winner). The Flexible Watchman (E-Value) stays strong at 75%.
The "Futility" Check: The paper also shows how to use this to stop a trial early if the drug is failing. If the betting wealth starts shrinking or staying low, you know the drug isn't working, and you can stop to save money and patients.

4. The Software and The Future

The authors didn't just write theory; they built a free software package called evalinger (and a website) that lets doctors and statisticians run these betting simulations themselves.

They also looked at the "Regulatory Landscape" (the FDA). They note that while the FDA loves the old "Strict Accountant" methods, they are opening the door for new methods. The paper argues that E-Values fit well into the FDA's new 2026 draft guidance because they provide a clear, mathematical guarantee that you aren't faking the results, even if you change your mind about when to look at the data.

Summary

Think of E-Values as a universal "Truth Meter" for clinical trials.

Old way: You have to promise exactly when you will check the meter, or the reading becomes unreliable.
New way (E-Values): You can check the meter whenever you want, as often as you want, and the reading is guaranteed to be honest.

The paper concludes that while E-Values might be slightly slower to find a cure in a perfectly planned trial, they are the superior tool for the messy, real-world reality of clinical trials where plans change, data arrives late, and safety committees need to peek whenever necessary.

Technical Summary: E-Values for Adaptive Clinical Trials

Problem Statement

Adaptive clinical trials increasingly rely on interim analyses, flexible stopping rules, and data-dependent design modifications to improve ethical efficiency and operational speed. However, these features complicate statistical guarantees. Classical frequentist methods, such as group sequential designs, require a prespecified monitoring schedule and alpha-spending function to control Type I error; repeated inspection of fixed-horizon test statistics without such corrections leads to substantial error inflation. Similarly, while Bayesian adaptive designs offer flexibility through posterior and predictive probabilities, standard threshold rules (e.g., $P(\text{effect} > 0) > 0.975$ ) do not automatically control frequentist Type I error under optional stopping and require extensive simulation-based calibration for each specific design.

There is a need for a methodology that provides anytime-valid inference—statistical guarantees that remain valid regardless of when or how often data are inspected, without requiring a fixed sample size or a pre-specified stopping rule.

Methodology

The paper proposes the use of e-values and e-processes as a practical alternative for interim monitoring.

Definitions: An e-value is a non-negative statistic $E$ such that $E_{H_0}[E] \le 1$ . An e-process is a sequence of e-values $(E_1, E_2, \dots)$ computed as data accumulates.
The Betting-Martingale Construction: For two-arm randomized controlled trials (RCTs) with binary endpoints, the authors utilize a "testing by betting" framework. A bettor starts with wealth 1 and bets against the null hypothesis ( $H_0: p_T = p_C$ ) at each patient pair. The betting fraction $\lambda$ is chosen based on a design alternative (anticipated effect size). The wealth process $E_n = \prod_{i=1}^n (1 + \lambda D_i)$ , where $D_i$ is the outcome difference, forms a non-negative martingale under $H_0$ .
Anytime-Validity: By Ville's inequality, if $(E_t)$ is a non-negative supermartingale under $H_0$ , then $P_{H_0}(\sup_t E_t \ge 1/\alpha) \le \alpha$ . This implies that rejecting $H_0$ when $E_t \ge 1/\alpha$ controls the Type I error at level $\alpha$ for any stopping time, planned or unplanned.
Extensions:
- Survival Endpoints: The paper utilizes the "safe logrank test," which constructs an e-process via a mixture likelihood ratio over the hazard ratio, providing valid inference under optional stopping.
- Futility Monitoring: Two mechanisms are proposed: (1) inverting the e-process to generate an anytime-valid confidence sequence and stopping if the upper bound falls below the minimum clinically important difference (MCID); and (2) constructing a "reciprocal" e-process to test the reverse hypothesis (that the effect is at least MCID).
- Platform Trials: E-values support multiplicity control via arithmetic means (for dependent hypotheses) or products (for independent hypotheses), enabling False Discovery Rate (FDR) control using procedures like e-BH.

Key Contributions

The paper serves as a methodology guide for practitioners, offering the following specific contributions:

Practical Constructions: Explicit development of betting-martingale e-processes for two-arm trials with binary and survival endpoints, including guidance on choosing the "GROW-optimal" (Growth-Rate Optimal) betting fraction $\lambda$ to maximize power under a design alternative.
Numerical Comparison: A comprehensive simulation study comparing five monitoring rules in a two-arm binary RCT:
- Naive repeated p-values.
- Calibrated group sequential (O'Brien–Fleming-like).
- Betting-martingale e-process.
- Naive Bayesian posterior threshold.
- Calibrated Bayesian posterior threshold (simulation-tuned).
Extended Simulations: Analysis of the power gap between group sequential and e-value methods across varying monitoring schedules (fixed, irregular, and continuous) and design parameters.
Futility Framework: Implementation and evaluation of futility monitoring via confidence sequences and reciprocal e-processes.
Real-Data Illustration: A re-analysis of the Novick (1965) ulcer trial, comparing e-value trajectories against original classical and Bayesian analyses.
Software Implementation: The release of the evalinger R package and an accompanying interactive Shiny web application. These tools implement e-process construction, GROW-optimal calibration, real-time monitoring, confidence sequences, futility analysis, and hybrid comparisons with group sequential and Bayesian methods.

Results

The numerical study (50,000 simulations, 20 interim looks, $\alpha=0.025$ ) yielded the following findings:

Error Control: Naive repeated p-values and naive Bayesian posterior thresholds (e.g., $>0.975$ ) inflated Type I error to approximately 13–15%. In contrast, the calibrated group sequential, calibrated Bayesian, and e-value methods successfully controlled Type I error (at or below 2.5%).
Power Comparison (Fixed Schedule): Under a fixed schedule of 20 looks:
- Calibrated Group Sequential: 86.1% power.
- E-value (Betting): 72.3% power.
- Calibrated Bayesian: 68.8% power.
- The e-value method was more conservative than group sequential but less conservative than the calibrated Bayesian rule in this specific design.
Monitoring Schedule Sensitivity: The power gap is highly dependent on the monitoring schedule. Under continuous monitoring (every patient pair), the group sequential power collapsed to 10% (due to the need for extreme conservatism to maintain Type I error), while the e-value power increased to 75%. Under irregular schedules, e-values maintained stable power without re-calibration, whereas group sequential boundaries required re-calibration for each realized schedule.
Parameter Sensitivity: The power gap between e-values and group sequential methods narrowed for large effect sizes (e.g., 2% gap for $\delta=0.20$ ) but widened for small effects at moderate base rates (up to 29% gap for $\delta=0.10$ ).
Futility: The reciprocal e-process was more sensitive for detecting futility (53.5% detection rate) compared to the confidence sequence approach (14.2%) in a scenario with a sub-clinical effect.
Real-Data (Novick 1965): The e-process accumulated evidence ( $E \approx 15.3$ ) but did not cross the rejection threshold ( $E \ge 40$ ), aligning with the original classical sequential analysis which found no significant difference. This contrasted with the original Bayesian analysis, which reported a posterior credibility of 0.98, highlighting the "price of anytime validity" where e-values require stronger evidence to reject under optional stopping.

Significance and Claims

The paper positions e-values as a principled framework for anytime-valid inference that addresses the operational realities of modern adaptive trials.

Regulatory Context: The authors situate the methodology within the context of the January 2026 FDA draft guidance on Bayesian methodology. They argue that e-values map naturally to the guidance's "Type I error calibration" pathway, offering an analytic guarantee of error control that avoids the extensive simulation burden required for posterior-threshold approaches. They also suggest e-values can serve as a "frequentist safety layer" or evidential ledger alongside Bayesian decision rules.
Practical Utility: The paper claims e-values are most appropriate when:
- Interim looks are irregular, opportunistic, or continuous.
- Optional continuation (extending a trial beyond its planned horizon) is a realistic possibility.
- An auditable, time-uniform validity guarantee is a primary requirement.
Limitations: The authors modestly acknowledge that e-values are not a universal replacement. They note a power penalty relative to optimally calibrated group sequential designs when the monitoring schedule is fixed and known in advance. They also highlight that constructing powerful e-processes for complex endpoints (e.g., non-proportional hazards) remains a research frontier and that the methodology requires careful tuning of the betting fraction to avoid under-powering.

In conclusion, the paper advocates for a hybrid approach: using Bayesian predictive probabilities for operational decision-making, group sequential boundaries for power-optimized confirmatory analysis at pre-planned looks, and e-processes as a robust, anytime-valid evidential ledger that remains valid regardless of monitoring frequency or schedule changes.