📊 statistics

Externally Controlled Trials: A Review of Design and Borrowing Through a Causal Lens

This paper presents a unified causal framework and six-step roadmap to organize, synthesize, and evaluate modern methodologies for externally controlled trials, clarifying how to effectively integrate external data into single-arm and hybrid trial designs while balancing efficiency and robustness.

Original authors: Ke Zhu, Rima Izem, Peng Yang, Ying Yuan, Herbert Pang, Mark van der Laan, Lei Nie, Birol Emir, Pallavi Mishra-Kalyani, Hana Lee, Shu Yang

Published 2026-05-06

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ke Zhu, Rima Izem, Peng Yang, Ying Yuan, Herbert Pang, Mark van der Laan, Lei Nie, Birol Emir, Pallavi Mishra-Kalyani, Hana Lee, Shu Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to prove that your new secret sauce makes a soup taste better. The gold standard way to do this is a Randomized Controlled Trial (RCT): you invite 100 people, flip a coin for each one, and give half the new sauce and half the old sauce. You then ask them which soup they liked. Because the coin flip ensures the two groups are identical in every way (age, taste buds, hunger level), any difference in the soup is definitely due to the sauce.

However, sometimes you can't do this. Maybe the soup is for a very rare disease where you can't find 100 people. Maybe it's unethical to give a "bad" sauce to people who are starving. Maybe you need an answer yesterday, and waiting to recruit 100 people takes too long.

This is where Externally Controlled Trials (ECTs) come in. Instead of flipping a coin to create a control group, you say: "We will give our new sauce to our 50 patients, and we will compare them to a group of 500 people who ate the old sauce in a different hospital five years ago, or in a database of insurance records."

The problem? That old group isn't a perfect match. They might be older, sicker, or the soup might have been served differently. This paper is a "roadmap" for statisticians on how to use these "external" groups without getting fooled by the differences.

Here is the paper's guide, broken down into simple concepts:

1. The Two Main Recipes

The paper organizes these trials into two main types:

The "Solo Chef" Trial (Single-Arm Trial): You have no control group in your own kitchen. You only have your 50 patients with the new sauce. You must compare them to an external group (the "Solo Chef" relies entirely on the outside world). This is risky because if your patients are just naturally healthier than the old group, you might think your sauce is magic when it's just the patients.
The "Hybrid" Trial: You have your 50 patients with the new sauce, and you also have 50 patients in your own kitchen eating the old sauce (the internal control). But your internal group is too small to be sure. So, you "borrow" extra people from the external group to make your comparison stronger. This is safer because you have a local control group to check if the external data is lying to you.

2. The Two Big Traps

When you mix your fresh data with old data, two things can go wrong. The authors call these Covariate Shift and Outcome Drift.

Covariate Shift (The "Different Ingredients" Problem): Imagine your new patients are all young and fit, but the old data is full of elderly people with weak immune systems. If you just compare the results, the new sauce looks great, but only because the patients were younger. You have to mathematically "re-weight" the old data to make it look like it came from young, fit people.
Outcome Drift (The "Different Seasons" Problem): Imagine you are comparing your soup to data from 10 years ago. Maybe the old data was collected when winter was colder, or the hospitals used different thermometers. Even if the patients are identical, the results might be different just because the world changed. This is "drift." If you don't account for this, you might think your sauce is bad when it's actually fine, or vice versa.

3. The Six-Step Roadmap

The authors propose a six-step checklist to make sure you don't get tricked:

Define the Question: What exactly are we trying to prove? (e.g., "Does the sauce help young adults?")
Check the Data: Do we have the right ingredients? (Are the old records detailed enough?)
Make Assumptions: We have to guess that the old data is similar enough to our new data. The paper calls this "Identifiability."
Do the Math: Turn those guesses into a statistical formula.
Run the Models: There are two main ways to do this math:
- The Bayesian Way (The "Flexible Chef"): This approach starts with a "hunch" based on the old data. If the new data agrees with the hunch, the chef trusts the old data heavily. If the new data disagrees, the chef ignores the old data. It's like a dynamic conversation between the past and the present.
- The Frequentist Way (The "Strict Judge"): This approach is more rigid. It tries to prove the old data is safe to use by running strict tests. If the data fails the test, it throws the old data away. It's like a judge who only accepts evidence if it passes a specific legal standard.
Sensitivity Analysis (The "What If" Test): This is the most important step. The authors say: "Let's pretend the old data is slightly wrong. How much wrong can it be before our conclusion changes?" If a tiny error changes your result, your conclusion is fragile. If it takes a huge error to change your result, your conclusion is robust.

4. The "Borrowing" Strategies

How do we actually mix the data? The paper reviews many tools, which can be grouped into:

Matching: Finding an old patient who looks exactly like a new patient (like finding a twin in a database).
Weighting: Giving more importance to old patients who look like your new patients and less importance to those who don't.
Discounting (The Bayesian Special): If the old data looks a bit suspicious, you don't throw it away; you just "turn down the volume" on it. You listen to it, but you don't let it shout over your new data.
Testing First: You run a test to see if the old data is compatible. If it passes, you mix it in. If it fails, you use only your new data.

5. The Bottom Line

The paper concludes with three main lessons:

Randomization is still King: Nothing beats a coin flip. If you can't randomize, you have to work much harder to prove your results are real.
Speed vs. Safety: The methods that give you the biggest boost in "power" (making your trial look bigger and faster) usually require the strongest assumptions. If you relax those assumptions to be safer, you lose some of that speed boost. It's a trade-off.
Transparency is Key: You must be honest about your assumptions. You can't just say "we used old data." You have to explain how you checked it, what you assumed, and how you tested if you were wrong.

In short, this paper is a manual for how to use "borrowed" data to speed up medical research without accidentally cooking up a fake result. It tells statisticians: "You can use the old recipes, but you must taste-test them carefully before serving them to the world."

Technical Summary: Externally Controlled Trials: A Review of Design and Borrowing Through a Causal Lens

1. Problem Statement

Randomized Controlled Trials (RCTs) are the gold standard for evaluating medical interventions, yet they face significant limitations in rare diseases, oncology, pediatrics, and post-approval research due to recruitment challenges, ethical constraints, high costs, and the need for timely evidence. Consequently, there is a growing reliance on Externally Controlled Trials (ECTs), which compare experimental treatments against controls drawn from external sources such as historical trials, disease registries, or real-world data (RWD).

While methodological research in causal inference, Bayesian dynamic borrowing, and hybrid designs has expanded rapidly, the literature remains fragmented. Different approaches often employ distinct estimands, assumptions, and inferential goals, making it difficult for practitioners and regulators to select appropriate methods or understand the trade-offs between efficiency and robustness. The core challenge lies in integrating external data without introducing bias from covariate shift (systematic differences in baseline covariates) or outcome drift (residual differences in outcomes even after conditioning on covariates).

2. Methodological Framework

The authors propose a unified six-step scientific roadmap to organize ECT methodology, grounded in the potential outcomes framework and causal inference principles. This roadmap structures the analysis of two primary ECT designs:

Single-Arm Trials with External Controls (SAT+EC): A single treatment arm compared entirely to external controls.
Hybrid Controlled Trials (HCT): An RCT where the internal control arm is augmented with external controls.

The six steps are:

Scientific Question & Estimand: Defining the target population, causal estimand (e.g., Average Treatment Effect on the Treated, ATT, or Average Treatment Effect, ATE), and intercurrent events.
Observed Data Structure: Characterizing the data sources (Trial vs. External Control), variables (covariates $X$ , treatment $A$ , outcome $Y$ ), and sampling mechanisms.
Identifiability Assumptions: Establishing assumptions required to link the causal estimand to observed data, such as consistency, no unmeasured confounding, and positivity.
Identification: Expressing the causal estimand as a functional of the observed data distribution (e.g., via outcome modeling, inverse probability weighting, or doubly robust combinations).
Statistical Models & Estimation: Selecting estimation strategies (Bayesian or frequentist) and handling nuisance parameters.
Sensitivity Analysis: Evaluating the robustness of conclusions to violations of untestable assumptions (e.g., unmeasured confounding or outcome drift).

3. Key Contributions

The paper synthesizes modern ECT methodology through this causal lens, offering three main contributions:

A. Unification of Approaches via a Causal Roadmap

The authors demonstrate that apparent differences between Bayesian and frequentist methods often stem from distinct exchangeability assumptions rather than fundamental inferential incompatibilities. By mapping methods to the six-step roadmap, the paper clarifies how modeling and borrowing strategies trade off efficiency and robustness, particularly under covariate shift and outcome drift.

B. Systematic Review of Designs and Methods

The paper provides a comprehensive review of methods for both SAT+EC and HCT settings:

For SAT+EC: The focus is on estimating the ATT. Methods include propensity score matching/weighting, outcome modeling, and doubly robust estimators (AIPW, TMLE). The paper emphasizes that without randomization, these methods rely heavily on the assumption of no unmeasured confounding, necessitating rigorous sensitivity analysis (e.g., E-values, bias modeling).
For HCT: The paper categorizes borrowing strategies into four classes based on assumptions and data usage:
1. RCT-only analysis: Serves as a benchmark.
2. Borrowing under no outcome drift: Methods like propensity score matching/weighting and outcome modeling that assume external controls are exchangeable with RCT controls given covariates.
3. Bayesian borrowing under outcome drift: Utilizes informative priors (Power priors, Elastic priors, Commensurate priors, Mixture priors) to dynamically discount external data based on compatibility. The paper highlights Self-Adapting Mixture (SAM) priors and Elastic priors for their ability to achieve information-borrowing consistency (borrowing fully when compatible, refraining when not).
4. Frequentist borrowing under outcome drift: Includes Prognostic adjustment (using EC data to improve nuisance estimation without assuming exchangeability), Test-then-pool (pre-testing for compatibility), Selective borrowing (unit-level selection of exchangeable controls), Bias-model-based integration (modeling and correcting bias), and Estimator averaging.

C. Software and Practical Guidance

The authors catalog 19 R packages (Table 3) implementing these methods, noting their supported outcomes (continuous, binary, time-to-event) and dependencies. They also discuss the regulatory landscape, emphasizing the need for prespecification, transparent justification of assumptions, and operating characteristic evaluations (Type I error, power) via simulation.

4. Results and Findings

The review does not present new empirical results but synthesizes existing theoretical and simulation-based findings:

Efficiency vs. Robustness: There is an inherent tension between efficiency gains and robustness. Methods that maximize precision (e.g., full borrowing) rely on strong exchangeability assumptions. Methods that relax these assumptions (e.g., prognostic adjustment, selective borrowing) sacrifice some efficiency to protect against bias.
Dynamic Borrowing: Contrary to some critiques suggesting dynamic borrowing is illusory in small samples, the paper argues that while small RCT control arms may lack power to detect all biases, methods like prognostic adjustment and randomization inference can still provide consistency and Type I error control even when dynamic borrowing is imperfect.
Sensitivity Analysis: Sensitivity analysis is identified as a core inferential component, not an afterthought. Frameworks like the "b-value" and E-values are crucial for quantifying the magnitude of unmeasured bias required to overturn conclusions.
Regulatory Considerations: The paper notes that regulatory adoption requires "fit-for-purpose" data and clear communication of assumptions. Methods that rely on adaptive borrowing to accommodate incompatibility may face regulatory hurdles if the operating characteristics are not well-calibrated.

5. Significance and Future Directions

The paper claims significance by establishing a coherent foundation for integrating external data into regulatory and clinical decision-making. By reframing ECT methodology through a causal lens, it bridges the gap between disparate statistical traditions (Bayesian vs. frequentist) and provides a structured approach for practitioners.

The authors identify several open challenges and opportunities for future research:

Complex Innovative Designs: Adapting ECT methods for master protocols, platform trials, and basket trials where borrowing occurs across multiple dimensions (time, subgroups, arms).
Synthetic and AI-Generated Data: Integrating digital twins and foundation models while addressing validity and efficiency challenges.
Bayesian-Frequentist Synthesis: Developing methods that leverage the small-sample performance of Bayesian approaches while retaining asymptotic validity.
Trial Planning: Incorporating borrowing strategies into sample size calculations and adaptive decision rules at the design stage.
Long-term Outcomes: Addressing time-dependent confounding and informative censoring when external controls have longer follow-up than the RCT.
Data Constraints: Developing methods for settings with only summary statistics or federated learning due to privacy regulations.
Rare Diseases: Refining approaches for small, heterogeneous populations where bias risks are amplified.

In conclusion, the paper argues that while ECTs offer a practical complement to conventional trials, their reliability hinges on principled data selection, rigorous analysis, and transparent sensitivity analysis. The proposed roadmap serves as a guide to navigate the trade-offs between efficiency and robustness in this evolving field.