📊 statistics

Likelihood-Ratio E-Value Monitoring for Benchmark-Based Decisions in Early-Phase Oncology Trials

This paper proposes Calibrated E-Value Monitoring (CEVAM), a novel likelihood-ratio framework for early-phase oncology trials that establishes protocol-ready, sample-size-independent decision boundaries relative to clinical benchmarks, offering improved efficiency and anytime-valid interpretations compared to existing Bayesian optimal designs.

Original authors: Masahiro Kojima

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Masahiro Kojima

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef testing a new, experimental recipe for a soup. You have a "gold standard" recipe that is known to be mediocre (let's say it's just "okay" soup), and you have a "dream" recipe that would be absolutely delicious. Your goal is to taste the new soup as you cook it and decide: Is this new soup good enough to serve to customers, or is it so bad that we should stop cooking and throw it away?

In the world of early cancer trials, the "soup" is a new drug, the "tasting" is checking if patients get better, and the "gold standard" is a benchmark rate of success that doctors agree is the minimum acceptable.

This paper introduces a new tool called CEVAM (Calibrated e-value Monitoring) to help chefs (researchers) make these decisions faster and more clearly than before.

The Old Way: The "Gambler's Coin"

Previously, many trials used a method called BOP2. Imagine this as a gambler flipping a coin. To decide if the soup is good, the gambler calculates a complex "probability score" based on a hidden rulebook (called a "Bayesian prior").

The Problem: The rules for stopping depend on how many people you've tasted so far. If you've tasted 10 people, the rule is different than if you've tasted 20. It's like a game where the winning score changes every time you take a bite. Also, the score relies on that hidden rulebook, which can be confusing to explain to a non-expert.

The New Way: CEVAM (The "Evidence Meter")

The author, Masahiro Kojima, proposes CEVAM. Instead of a changing probability score, CEVAM uses a simple Evidence Meter based on a "Likelihood Ratio."

Think of this as a balance scale:

The "Bad" Side: One side of the scale holds the weight of the "mediocre soup" (the benchmark we want to beat).
The "Good" Side: The other side holds the weight of the "dream soup" (the target we hope for).
The Tasting: Every time a patient responds (a "taste"), you add a weight to the scale.
- If the patient gets better, the "Good" side gets heavier.
- If they don't, the "Bad" side gets heavier.

How CEVAM works:

Stopping for Success (Efficacy): If the "Good" side gets heavy enough to tip the scale past a fixed line, you stop early and say, "This is a winner!"
Stopping for Failure (Futility): If the "Bad" side gets heavy enough to tip the scale the other way, you stop early and say, "This is a loser; let's not waste more time."

Why is this better?

The paper claims CEVAM has three main superpowers:

No Hidden Rulebooks: Unlike the old method, you don't need a complex "Bayesian prior" (a hidden starting assumption). The evidence is calculated directly from the data and the benchmarks. It's like weighing the soup ingredients directly rather than guessing based on a secret recipe.
Simple Counting: The result is just a number. "If you have 25 successes out of 44 patients, stop." It creates a simple table that doctors can look at before the trial even starts. No complex math needed during the trial.
Two Versions for Two Needs:
- The "Fixed" Version: This is like a strict rule that works even if you decide to taste the soup at random, unexpected times. It guarantees you won't be fooled by luck, no matter when you check.
- The "Calibrated" Version: This is for trials where you plan to taste the soup at specific times (e.g., after 10, 20, and 30 people). The author "tuned" this version to be the most efficient, meaning it finds the answer (good or bad) using the fewest number of patients possible, while still keeping the risk of being wrong very low.

The Results: The "Tuned" Version Wins

The author ran thousands of computer simulations (like running the soup test 10,000 times in a virtual kitchen).

The Finding: The "Tuned" CEVAM version (called CEVAM-T) was the most efficient. It stopped the trials earlier than the old methods (BOP2) in almost every scenario.
The Trade-off: It stopped slightly earlier, but it didn't miss out on finding good drugs. It kept the success rate of finding a "winner" almost exactly the same as the old methods, just with fewer people involved.

Real-World Test: The TREND Trial

To prove it works in the real world, the author looked at a past breast cancer trial called TREND.

The Situation: The trial stopped early because 30 out of 44 patients got better.
The Test: The author applied the CEVAM rules to this data.
The Result: CEVAM agreed! It said, "Yes, 30 successes is enough evidence to stop early and call it a success." This showed that CEVAM can handle real-world data just like the old methods, but with a clearer, more direct logic.

Summary

In short, this paper says: "Stop using complex, changing probability rules to decide if a cancer drug works. Instead, use a simple, direct 'Evidence Meter' that compares the drug's performance against a fixed benchmark. This new method (CEVAM) lets you make decisions faster, uses fewer patients, and is easier to explain, without losing accuracy."

Technical Summary: Likelihood-Ratio E-Value Monitoring for Benchmark-Based Decisions in Early-Phase Oncology Trials

Problem Statement
Early-phase oncology trials frequently require protocol-ready interim rules to determine whether an experimental regimen demonstrates sufficient activity relative to prespecified clinical benchmarks. While existing Bayesian optimal phase II designs (such as BOP2 and BOP2-FE) provide calibrated count boundaries for these decisions, they rely on posterior probabilities derived from an analysis prior. These designs utilize sample-size-dependent posterior-probability cutoff functions to define stopping rules. The author identifies three limitations in this approach: (1) evidence is quantified through posterior probabilities even when vague priors are used; (2) the operative stopping rules are tied to posterior-probability thresholds rather than direct evidence measures; and (3) the validity of these rules relies on design-stage calibration rather than direct sequential validity arguments, limiting their robustness under arbitrary optional stopping.

Methodology
The paper proposes Calibrated E-Value Monitoring (CEVAM), a likelihood-ratio evidence framework designed to replace posterior-probability evidence with likelihood-ratio evidence on an e-value scale. The method is tailored for single-arm phase II trials monitoring binary efficacy endpoints against prespecified benchmarks ( $p_0$ for unacceptable response rates and $p_1$ for clinically meaningful alternatives).

The core methodology involves constructing two specific e-processes:

Efficacy E-Process ( $ELR_n$ ): A likelihood-ratio process comparing the design alternative $p_1$ against the least favorable point of the composite null $p \le p_0$ .
Reverse Futility E-Process ( $FLR_n$ ): A reverse likelihood-ratio process comparing a non-promising rate $p_0$ against the target rate $p_1$ .

The author establishes that both processes are nonnegative supermartingales under their respective null hypotheses. By Ville's inequality, crossing fixed thresholds ( $1/\alpha$ for efficacy, $1/\beta$ for futility) provides time-uniform Type I error control under optional monitoring.

A critical feature of CEVAM is the derivation of monotone response-count boundaries. Because the likelihood-ratio statistics are strictly monotone functions of the cumulative number of responses ( $R_n$ ), the continuous evidence thresholds can be converted into integer boundaries ( $r_{eff}$ and $r_{fut}$ ) for each interim look. This allows the design to be presented as a pretabulated boundary table, a format standard in clinical protocols.

The paper distinguishes between two implementations:

Fixed-Threshold Version: Uses prespecified evidence thresholds ( $\alpha, \beta$ ) directly. This version retains "anytime-valid" properties, meaning it is valid under arbitrary optional stopping and irregular monitoring schedules.
Planned-Look Calibrated Versions (CEVAM-NT and CEVAM-T): Designed for conventional trials with fixed interim looks. These versions calibrate the evidence-threshold parameters ( $\alpha_{eff}, \beta$ $α_{e f f}, β$ ) via grid search to meet specific operating characteristics (Type I error, power, expected sample size) at the planned looks.
- CEVAM-NT: Fixes the futility parameter ( $\beta$ ) and calibrates only the efficacy threshold.
- CEVAM-T: Calibrates both efficacy and futility thresholds simultaneously to optimize efficiency.

Key Contributions

Evidence Scale Shift: CEVAM replaces posterior-probability cutoffs with likelihood-ratio evidence defined directly relative to clinical benchmarks, eliminating the need for a Bayesian analysis prior.
Protocol-Ready Format: The method preserves the practical utility of BOP2-type designs by converting e-processes into monotone, integer response-count boundaries that can be tabulated before trial initiation.
Dual Implementation Framework: The paper provides both an anytime-valid fixed-threshold rule (for flexible/adaptive monitoring) and planned-look calibrated rules (for standard finite-look trials), bridging the gap between rigorous sequential theory and practical trial design.
Extension to Complex Endpoints: While the main text focuses on binary endpoints, the Supplementary Material outlines how the projected-evidence principle applies to ordinal, nested, co-primary, and joint efficacy-toxicity endpoints.

Simulation Results
Monte Carlo simulations compared CEVAM against existing designs: BOP2 (futility only), BOP2-FE (efficacy and futility with power-function cutoffs), and BOP2-FE with O'Brien-Fleming-type efficacy cutoffs.

Type I Error: All designs, including CEVAM-NT and CEVAM-T, successfully controlled the nominal Type I error (0.10) under the design null ( $p_0 = 0.20$ ).
Efficiency: The tuned CEVAM-T design achieved the smallest expected sample size across all evaluated scenarios (both under the null and alternative hypotheses) compared to the other designs.
Power: CEVAM-T maintained rejection probabilities (power) close to those of BOP2-FE, indicating that the efficiency gains did not materially compromise the ability to detect promising regimens.
Error Spending: CEVAM designs tended to allocate Type I error probability earlier in the trial compared to BOP2-F designs, which concentrated error spending at the final look.

Application to TREND Trial
The method was applied retrospectively to the TREND trial (a phase II study of neoadjuvant tislelizumab for triple-negative breast cancer). Using the trial's reported benchmarks ( $p_0=0.41, p_1=0.56$ ) and the final efficacy-evaluable result (30 pathological complete responses in 44 patients), both CEVAM-NT and CEVAM-T classified the result as sufficient evidence for early success stopping. This demonstrated that CEVAM rules could be specified using standard reported design parameters and would support the same early-stopping decision as the original Bayesian design.

Significance and Claims
The paper claims that CEVAM offers an analysis-prior-free, protocol-ready likelihood-ratio evidence scale for benchmark monitoring in early-phase oncology. Its primary significance lies in decoupling the evidence scale from posterior probabilities while retaining the practical boundary-table format required for clinical protocols.

The author modestly notes that the method is specifically targeted at the "narrower but practically important" problem of arm-level phase II benchmark monitoring, rather than general adaptive two-arm randomized settings. The paper concludes that CEVAM-T improves trial efficiency by reducing expected sample sizes without sacrificing power, providing a flexible alternative to posterior-cutoff monitoring. Future work is suggested to address delayed outcomes, alternative look schedules, and integration with multi-arm dose-optimization designs.