📊 statistics

Risk-inclusive Contextual Bandits for Early Phase Clinical Trials

This paper proposes a risk-inclusive contextual bandit algorithm that integrates participant-specific data using dual Thompson samplers and generalized asymptotic confidence sequences to optimize dose allocation in early-phase clinical trials by effectively balancing safety and efficacy.

Original authors: Rohit Kanrar, Chunlin Li, Zara Ghodsi, Margaret Gamalo

Published 2026-02-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rohit Kanrar, Chunlin Li, Zara Ghodsi, Margaret Gamalo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to perfect a new recipe for a soup that needs to be both delicious (efficacy) and safe to eat (safety). You have five different spice levels to test.

In a traditional clinical trial, you would act like a rigid robot: you'd give the soup to 100 people, randomly handing out each spice level to 20 people each. You'd wait until the very end to taste the results.

The Problem: This is wasteful. If Spice Level 1 tastes terrible and makes people sick, you still fed it to 20 people. If Spice Level 5 is amazing, you only fed it to 20 people, when you could have fed it to 80. You also ignore that some people have sensitive stomachs (covariates) while others don't.

This paper introduces a smarter way to run these trials using a concept called "Risk-inclusive Contextual Bandits." Let's break it down with a simple story.

1. The "Smart Waiter" (The Algorithm)

Imagine a waiter in a busy restaurant who has to serve soup to customers one by one. Instead of flipping a coin to decide which spice level to serve, this waiter is a super-smart AI.

Contextual: The waiter looks at the customer first. "Oh, this person has a sensitive stomach (covariate)."
The Bandit: The waiter remembers what happened to previous customers. "Last time, the spicy soup made the person with the sensitive stomach cry, but the mild soup was boring."
The Decision: The waiter uses this history to make a guess. "I think this customer will love the medium-spicy soup, and it won't hurt them."

This is the Contextual Bandit part. It learns as it goes, giving better soup to more people as the trial progresses, rather than sticking to a rigid plan.

2. The "Two-Headed Coin" (Risk-Inclusive)

Most smart waiters only care about one thing: Taste (Efficacy). They might serve a super-spicy soup because it tastes great, even if it burns the customer's tongue.

This paper introduces a Risk-Inclusive waiter. This waiter has two heads:

Head A (Taste): "How delicious is this?"
Head B (Safety): "Will this make the customer sick?"

The waiter has a dial (a weight parameter) that the chef can turn.

If the chef is testing a cancer drug, the dial is turned toward Taste (we accept some risk for a cure).
If the chef is testing a skin cream, the dial is turned toward Safety (we can't risk a rash).

The waiter balances these two heads. It won't serve the "best tasting" soup if it's too dangerous, and it won't serve the "safest" soup if it tastes like water. It finds the sweet spot.

3. The "Magic Safety Net" (AsympCS)

Here is the tricky part. Because the waiter is changing the menu based on what they learn, the data gets "messy." If you just look at the numbers at the end, you might get the wrong answer because the waiter was too clever.

The authors invented a Magic Safety Net called AsympCS (Asymptotic Confidence Sequences).

The Analogy: Imagine the waiter is walking a tightrope. Usually, you only check if they fell at the very end. But with this safety net, you can check every single step they take.
The Benefit: If the waiter starts walking toward a cliff (a dangerous dose), the net catches them immediately. You can stop the trial early because you are statistically sure the soup is bad.
The "Model-Assisted" Trick: The waiter uses a simple guess (a parametric model) to decide what to serve. But the Magic Safety Net is so strong that even if the waiter's guess is wrong, the final conclusion about which soup is best is still 100% correct. It's like having a GPS that might give you a slightly wrong turn-by-turn, but a backup system that guarantees you arrive at the right destination.

4. Why This Matters

For Patients: Fewer people get the "bad" soup. More people get the "good" soup. It's more ethical.
For Drug Companies: They don't need to test as many people to find the winner. They can stop the trial earlier, saving millions of dollars and years of time.
For Science: It handles the real world. People are different (covariates), and we don't know the perfect recipe yet (uncertainty). This method adapts to that chaos.

Summary

Think of this paper as upgrading a clinical trial from a rigid, blindfolded game of chance to a smart, adaptive learning system.

It looks at the patient (Context).
It balances taste and safety (Risk-Inclusive).
It learns from every patient to improve the next one (Bandit).
It has a super-secure safety net that lets you stop the trial the moment you find the winner, without needing to trust that your initial guesses were perfect (AsympCS).

It's about finding the best medicine faster, safer, and with less waste.

1. Problem Statement

Early-phase clinical trials (specifically Phase IIb dose-ranging studies) face a critical challenge: selecting the optimal drug dose that balances efficacy and safety while accounting for participant heterogeneity (covariates).

Limitations of Traditional Designs: Standard Equal Randomization (ER) exposes many participants to sub-optimal doses because it ignores individual covariates and treats all arms equally. This leads to larger sample sizes and prolonged development times.
Limitations of Existing Adaptive Designs: While Response Adaptive Randomization (RAR) and Contextual Bandits exist, they often:
- Focus solely on efficacy, neglecting safety constraints.
- Fail to incorporate participant covariates effectively.
- Rely on parametric models that, if misspecified, lead to invalid statistical inference (biased effect size estimates).
- Lack "anytime-valid" inference capabilities, meaning they cannot support continuous monitoring for early stopping without inflating Type I error rates.

The authors aim to develop a framework that simultaneously optimizes dose allocation (balancing safety/efficacy using covariates) and provides robust, model-agnostic statistical inference for effect sizes during the trial.

2. Methodology

The proposed framework consists of two main components: Trial Optimization and Trial Evaluation.

A. Trial Optimization: Risk-inclusive Thompson Sampling (RiTS)

The authors frame the trial as a Contextual Multi-Armed Bandit (CMAB) problem.

Dual Thompson Samplers: Unlike standard Thompson Sampling (TS) which optimizes a single objective, RiTS employs two separate Bayesian models:
1. One for Efficacy ( $R_n$ ).
2. One for Safety ( $S_n$ ).
Weighted Utility: A pre-specified weight parameter $w \in (0,1)$ $w \in (0, 1)$ combines the posterior samples from both models to define a utility function:
$\omega(x, b_0, b, g_0, g; w) = w \cdot (\text{Efficacy Prediction}) + (1-w) \cdot (\text{Safety Prediction})$
- $w$ allows domain experts to tune the trade-off (e.g., higher $w$ for oncology where efficacy is prioritized; lower $w$ for dermatology where safety is paramount).
Clipping (Propensity Constraints): To ensure valid inference later, the algorithm enforces a "clipping" parameter $\delta$ . The probability of assigning any arm $a$ is constrained to $[\delta, 1-\delta]$ . This forces the algorithm to explore sub-optimal arms sufficiently, preventing the data distribution from becoming too skewed.
Covariate Integration: The allocation probability $q_n(a, x)$ depends on the participant's covariate vector $X_n$ , enabling personalized dose assignment.

B. Trial Evaluation: Asymptotic Confidence Sequences (AsympCS)

To address the need for continuous monitoring and valid inference under adaptive allocation, the authors extend Asymptotic Confidence Sequences (AsympCS) to the CMAB setting.

Cross-Fitting & AIPW: The method uses a cross-fitting approach where data is split into training ( $D_{trn}$ $D_{t r n}$ ) and evaluation ( $D_{eval}$ $D_{e v a l}$ ) sets. It constructs Augmented Inverse Propensity Weighted (AIPW) pseudo-outcomes.
- This allows the method to borrow information across arms via regression estimators while correcting for the bias introduced by adaptive randomization using inverse propensity weighting.
Model-Assisted Inference:
- Optimization Phase: Uses parametric models (e.g., linear regression) for efficient learning.
- Inference Phase: The validity of the confidence sequences does not depend on the parametric model being correctly specified. Even if the model is misspecified, the AsympCS provides valid coverage guarantees for the effect sizes ( $\Delta(a)$ ) as long as the regression estimators converge uniformly to some function.
Anytime-Valid Stopping: The method generates confidence intervals that maintain coverage guarantees at any stopping time $n$ , allowing for early stopping for efficacy or futility without inflating Type I error.

3. Key Contributions

Risk-Inclusive Framework (RiTS): A novel adaptation of Thompson Sampling that explicitly balances efficacy and safety using a weighted composite utility, addressing a gap in existing multi-objective bandit literature for clinical trials.
Robust Inference under Adaptivity: Theoretical proof that AsympCS remains valid in a contextual bandit setting even with model misspecification. This is achieved by combining AIPW estimators with cross-fitting and propensity clipping.
Anytime-Valid Stopping Rules: Provides a statistically principled method to stop trials early based on continuous monitoring of confidence sequences, saving time and resources while maintaining rigorous error control.
Practical Implementation Guidelines: Detailed recommendations on hyperparameters (burn-in size $m$ , clipping $\delta$ , initial randomization $n_0$ ) to ensure the asymptotic properties hold in finite samples.

4. Results

The authors validated the method through extensive simulations and a real-world application.

Simulation Studies (High-SNR, Low-SNR, Null-Efficacy):
- Regret: RiTS achieved the lowest cumulative safety regret and comparable utility regret compared to standard TS (which ignored safety) and Equal Randomization (ER). TS had the lowest efficacy regret but dangerously high safety regret.
- Inference Quality: RiTS-AIPW provided valid time-uniform coverage (cumulative miscoverage < nominal level) across all scenarios.
- Efficiency: While RiTS required slightly wider confidence intervals than ER (due to adaptive allocation skew), it was significantly more efficient than TS in identifying the "winner dose" while maintaining safety.
- Robustness: Under model misspecification, RiTS-AIPW maintained control over cumulative miscoverage, whereas standard methods (like Rand-OF) failed to control Type I error rates.
Real-World Application (Alopecia Areata Phase IIb Trial):
- Applied to a dataset of 654 participants with 5 active doses and a placebo.
- Findings: RiTS successfully identified the most efficacious dose (Arm 6) but allocated more participants to a dose with a better safety profile (Arm 4), aligning with the actual commercial decision made in the study.
- Comparison: RiTS outperformed TS in balancing safety and efficacy, demonstrating that the method can recover the true "winner" while protecting patient safety.

5. Significance

This paper bridges the gap between Reinforcement Learning (RL) and Clinical Trial Design in a critical way:

Patient Safety: It moves beyond "efficacy-only" optimization, ensuring that adaptive trials do not expose patients to unsafe doses in the pursuit of better outcomes.
Statistical Rigor: It solves the "inference problem" in adaptive trials. Historically, adaptive designs made post-hoc statistical inference difficult or invalid. This method provides valid, anytime-valid inference even when the underlying dose-response model is unknown or misspecified.
Regulatory Relevance: By offering a framework that supports early stopping and continuous monitoring with rigorous error control, it facilitates faster drug development cycles without compromising statistical integrity or patient safety.

In summary, the authors propose a model-assisted, risk-inclusive approach that leverages modern RL techniques for trial optimization while relying on robust causal inference tools for evaluation, offering a practical solution for the complex trade-offs inherent in early-phase clinical trials.