📊 statistics

How to optimize dynamic borrowing in basket trials - A utility-based framework and results of a comparison study

This paper proposes a utility-based framework for optimizing dynamic information borrowing in basket trials, demonstrating through a comparison study that no uniformly most powerful design exists and that transparent optimization is essential to balance power and type-I error rates based on specific trial goals.

Original authors: Lukas D Sauer, Alexander Ritz, Meinhard Kieser

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lukas D Sauer, Alexander Ritz, Meinhard Kieser

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of one suspect, you have a whole lineup of people who might be guilty. In the world of medicine, this is like testing a new drug on different groups of patients, called "strata," who might have slightly different versions of a disease. This is the world of basket trials. The big question for statisticians is: How do you learn from one group to help solve the mystery for another? If the groups are very similar, it makes sense to let them "borrow" clues from each other to get a clearer picture faster. But if they are totally different, borrowing clues might lead you to the wrong culprit. The challenge is finding the perfect balance: borrowing just enough to be smart, but not so much that you get tricked. This paper dives into the math of how to tune that balance, asking: "What is the best way to mix these clues so we catch the real villains (effective treatments) without falsely accusing the innocent (safe but ineffective treatments)?"

The authors of this paper, Lukas D Sauer, Alexander Ritz, and Meinhard Kieser, decided to stop guessing and start building a "recipe book" for these trials. They created a framework—a checklist of all the ingredients needed to design a basket trial—and then put it to the test. They didn't just look at one way to mix the clues; they tried twelve different "flavors" of mixing strategies, which they call utility functions. Think of these as different goals: one strategy might say, "I want to catch as many villains as possible, even if I risk a few false alarms," while another might say, "I will never accuse an innocent person, even if it means I miss a few villains."

To see which strategy was the best, the researchers ran a massive simulation study. They acted like video game designers, creating seven different "worlds" (scenarios) with varying numbers of patient groups (from 3 to 20) and different numbers of patients (from 15 to 480). In these virtual worlds, they tested their twelve strategies against the standard way of doing things. They found that there is no single "magic wand" or "super-strategy" that works perfectly in every situation. In fact, they proved mathematically that such a perfect design is impossible. Instead, the best strategy depends entirely on the specific situation.

Here is what they discovered in their virtual worlds:

The "Brute Force" Winner: When trying to find the best settings for their strategies, the simplest method, called grid search (which is like checking every single combination of settings one by one), turned out to be the fastest and most reliable. It beat out complex, fancy algorithms that tried to be too clever.
The "Goldilocks" Strategies: They found that different utility functions led to different behaviors. Some strategies were very conservative (very careful, rarely making false accusations, but sometimes missing real cures). Others were liberal (very eager to find cures, but willing to risk more false alarms). The "just right" or moderate strategies, specifically those using a utility function called u_ecd_avg_pen, seemed to offer the best compromise in many situations, giving a good boost to finding cures without letting the false alarm rate get too high.
The Size Matters Rule: The number of patient groups and the number of patients changed everything.
- In trials with few groups and few patients, borrowing information was very helpful, and moderate borrowing strategies worked well.
- In trials with many groups and many patients, the researchers found that borrowing information often became a bad idea. The groups were so numerous that mixing their data tended to confuse the results. In these big trials, the best approach was often to stop borrowing entirely and treat each group separately, just like a standard trial.
The "Jack of All Trades" Myth: The paper explicitly rules out the idea that there is one perfect design that works for everyone. They showed a mathematical counterexample proving that you cannot have a design that is the absolute best at finding cures in every possible scenario while also being perfect at avoiding false alarms. You always have to make a trade-off.

In the end, the paper suggests that the key to a successful basket trial isn't finding a new, fancy mathematical trick. Instead, it's about being transparent and careful. Researchers need to clearly state what they are trying to optimize (catching cures vs. avoiding false alarms) and choose a strategy that fits their specific trial's size and structure. If you have a small trial, borrowing clues can be a superpower. If you have a huge trial with many groups, it might be safer to keep your clues separate. The authors hope that by providing this clear framework and checklist, more clinical trials will be able to use these smart borrowing methods safely and effectively, rather than avoiding them because the math seems too confusing.

Technical Summary: Optimizing Dynamic Borrowing in Basket Trials

Problem Statement
Basket trials investigate a single treatment across multiple subpopulations (strata) within a unified trial structure, a design common in early-phase oncology and rare disease research. While analyzing strata separately preserves type-I error control, it often results in low statistical power due to small sample sizes. Conversely, pooling all data increases power but risks inflating the type-I error rate (TOER) in inactive strata. Information borrowing methods have been proposed to dynamically share information between strata based on response similarity, aiming to balance power gains against type-I error inflation. However, the performance of these methods depends heavily on tuning parameters, and there is no universally optimal design for all possible outcome scenarios. The paper addresses the lack of a systematic framework for optimizing these designs and the uncertainty regarding which optimization algorithms and target functions (utility functions) yield the best trade-offs.

Methodology
The authors propose a structured framework for optimizing basket trial designs, treating the selection of tuning parameters as a constrained optimization problem. The methodology is divided into three main components:

Optimization Framework: The authors define a checklist for planning basket trials, including trial-specific aspects (number of strata, endpoint type, scenario sets), operating characteristics (targets like power, constraints like TOER), and the analysis strategy. The core of the framework involves defining a utility function ( $u$ ) to maximize (e.g., expected number of correct decisions) subject to constraints ( $v \leq c$ ) on error rates.
Comparison Study Design: A pre-specified comparison study was conducted to evaluate:
- Part I: The reliability and efficiency of various optimization algorithms (Grid Search, Simulated Annealing, COBYLA, Differential Evolution, Particle Swarm Optimization) in finding optimal tuning parameters.
- Part II: The statistical performance of twelve distinct utility functions across seven different outcome scenario sets. These scenarios varied in the number of strata ( $I=3$ to $20$) and sample sizes.
Specific Design and Implementation: The study utilized the basket trial design by Fujikawa et al. (2020), which employs a Bayesian beta-binomial model with borrowing weights determined by the Jensen-Shannon divergence. The design has three tuning parameters: a decision threshold ( $\lambda$ ) and borrowing shape parameters ( $\varepsilon, \tau$ ). The utility functions varied in their targets (Experiment-wise Power, Expected Number of Correct Decisions, local power) and included penalty terms for exceeding type-I error thresholds. Simulations were performed using R on a high-performance computing cluster, with exact calculations for smaller scenarios and Monte Carlo approximations for larger ones.

Key Contributions

A Systematic Framework: The paper provides a comprehensive checklist and mathematical formulation for optimizing basket trial designs, explicitly addressing the specification of scenario sets, utility functions, and optimization algorithms.
Algorithmic Evaluation: The study empirically compares optimization algorithms, identifying Grid Search as the most reliable and efficient method for the specific three-parameter design tested, particularly when parallelized.
Utility Function Analysis: The authors demonstrate that different utility functions lead to distinct borrowing behaviors (conservative, moderate, or liberal). They introduce and evaluate penalized utility functions (e.g., $u\_ecd\_avg\_pen$ ) that enforce strict type-I error constraints.
Mathematical Limitation: The paper presents a mathematical counterexample proving that no Uniformly Most Powerful (UMP) test exists for basket trials. This establishes that no single design can be optimal across all possible alternative hypotheses while strictly controlling type-I error.

Results

Optimization Algorithms: Grid search outperformed stochastic meta-heuristic algorithms (like Simulated Annealing and PSO) in both runtime and consistency in finding optimal parameters for the Fujikawa design. Stochastic algorithms often struggled with "flat" regions in the utility function landscape.
Utility Function Performance:
- Conservative Borrowing: Utility functions like $u\_ewp\_avg\_pen$ (Experiment-wise Power with penalty) resulted in conservative borrowing, keeping Family-Wise Error Rate (FWER) close to frequentist levels.
- Moderate Borrowing: Functions like $u\_ecd\_avg\_pen$ (Expected Correct Decisions with penalty) offered a moderate compromise, gaining power in scenarios with multiple active strata while controlling FWER within acceptable limits (e.g., $\leq 1.5 \times$ nominal FWER).
- Liberal Borrowing: Functions like $u\_2pow\_avg\_pen$ led to more aggressive borrowing, resulting in higher power but also higher FWER inflation.
- Scenario Dependence: The optimal utility function depended on the trial setting. In scenarios with many strata and many patients, the optimization often resulted in no borrowing at all, suggesting that the design's parameters were tuned to rely solely on the decision threshold $\lambda$ to control error rates. In these large settings, the Fujikawa design's lack of explicit borrowing limits made it susceptible to being "overwhelmed" by information from other arms.
Default Parameters: The default parameters suggested by Fujikawa et al. ($fuj1$) were found to be too liberal in most scenarios, while $fuj2$ offered a more moderate approach but still required tuning of the decision threshold to ensure strict error control.

Significance and Claims
The authors claim that while information borrowing can yield power gains, there is no "one-size-fits-all" optimal design. The significance of their work lies in the demonstration that transparent optimization procedures are crucial for planning basket trials. They argue that the choice of utility function dictates the borrowing behavior and the resulting trade-off between power and type-I error.

The paper emphasizes that because no UMP test exists, researchers must explicitly define their target scenarios and acceptable error inflation levels. The framework allows for the communication of these trade-offs, which is essential given that many existing trials lack transparency in how borrowing parameters were chosen. The authors modestly conclude that their results are specific to the Fujikawa design and the tested scenarios, but the framework is generalizable to other borrowing mechanisms (such as Bayesian Hierarchical Models) and trial settings. They note that for strict type-I error control, frequentist testing without borrowing remains the safest option, but optimized borrowing can be tuned to respect constraints for specific, pre-specified scenarios.

Technical Summary: Optimizing Dynamic Borrowing in Basket Trials

More like this