Reproducing comparative therapeutic efficacy in relapsed refractory multiple myeloma using an external control arm derived from historical clinical trial data

Technical Summary: Reproducing Comparative Therapeutic Efficacy in Relapsed Refractory Multiple Myeloma Using an External Control Arm

Problem Statement
In oncology drug development, randomized controlled trials (RCTs) remain the gold standard for evaluating safety and efficacy. However, single-arm trials are increasingly employed when concurrent control arms pose ethical, scientific, or feasibility challenges. While external control arms (ECAs) derived from non-randomized data can supplement evidence, their use is often limited by concerns regarding bias from measured and unmeasured confounding. A significant portion of previous ECA applications has relied on real-world data (e.g., electronic medical records), which often suffer from missing data and inconsistent variable definitions. This study addresses the feasibility of constructing a robust ECA using patient-level data from historical clinical trials (HCTs)—a source with standardized data collection—to reproduce the treatment effect observed in a contemporaneous RCT for relapsed/refractory multiple myeloma (RRMM).

Methodology
The study utilized de-identified patient-level data from Medidata's Enterprise Data Store (MEDS), encompassing phase 3 historical RCTs conducted between 2010 and 2017.

• Data Sources: The analysis included a "target" RCT (investigational therapy vs. dexamethasone control) and a pool of historical RCTs where patients received dexamethasone control.
• Cohort Selection: Eligibility criteria were harmonized across datasets, requiring patients to be ≥18 years old, have RRMM, have received at least two prior lines of treatment, and have prior exposure to lenalidomide and bortezomib.
• Matching Algorithm: Due to the limited number of available historical controls, the authors employed optimal full matching (a propensity-score-based method) rather than greedy 1-1 matching. This approach subdivides subjects into strata to maximize the use of the investigational arm patients while minimizing average differences in propensity scores between cohorts.
• Covariates: The model adjusted for clinically important baseline covariates, including age, sex, race, region, ECOG performance status, cytogenetic risk, prior stem cell transplant, number of prior lines of therapy, and disease-specific markers (e.g., bone lesions, time since last progression).
• Statistical Analysis:
  • Primary Outcome: Overall Survival (OS).
  • Comparative Metrics: Kaplan-Meier survival curves, log-rank tests, and weighted Cox proportional hazards models to estimate Hazard Ratios (HR) and 95% Confidence Intervals (CI).
  • Balance Assessment: Standardized Mean Differences (SMD) were calculated pre- and post-matching, with an absolute SMD ≤0.20 considered indicative of negligible difference.
  • Sensitivity Analyses: To evaluate robustness against unmeasured confounding, the authors performed tipping point analyses (adjusting for hypothetical unobserved confounders) and calculated E-values (the minimum strength of association an unmeasured confounder would need to have with both treatment and outcome to explain away the observed effect).

Key Results

• Sample Characteristics: After optimal full matching, the analytic sample comprised 290 patients in the matched investigational arm and a weighted sample of 290 patients in the ECA. This retained 99% of the original target RCT investigational arm patients.
• Covariate Balance: Matching substantially improved balance; the propensity score SMD decreased from 0.827 (pre-match) to 0.171 (post-match). All available baseline covariates achieved an absolute SMD ≤0.20, indicating high comparability between the matched groups.
• Treatment Effect Reproduction:
  • Target RCT Comparison: The investigational arm vs. its internal randomized dexamethasone control yielded an HR of 0.74 (95% CI: 0.60–0.92; log-rank p=0.006).
  • External Control Comparison: The matched investigational arm vs. the matched ECA yielded a nearly identical HR of 0.76 (95% CI: 0.63–0.91; log-rank p=0.0158).
• Sensitivity Analysis:
  • Tipping Point: Statistical significance was maintained unless the unmeasured confounder prevalence in the ECA reached 35% (assuming an HR of 1.5 for the confounder). A "clinical tipping point" (reversal of effect direction) required an 80% prevalence.
  • E-value: The calculated E-value was 1.71, indicating that an unmeasured confounder would need to be associated with a 1.71-fold increased risk for both exposure and outcome to nullify the observed treatment effect.

Significance and Claims
The paper claims that an ECA constructed from harmonized historical trial data, when analyzed with a propensity-score-based full matching algorithm, can produce an overall survival estimate that closely matches the findings of a contemporaneous RCT. The study demonstrates that under favorable conditions—specifically when historical trial data is high-quality, standardized, and well-aligned with the target trial's population and design—external controls can yield credible comparative effectiveness evidence.

The authors emphasize that while these results support the potential utility of ECAs in oncology, they do not eliminate the risk of unmeasured confounding. Consequently, the paper advocates for rigorous confounding assessment, transparency in design, and the mandatory inclusion of sensitivity analyses (such as tipping point and E-value calculations) when relying on external controls for regulatory or clinical decision-making. The study positions itself as an empirical demonstration project that validates the methodology for using historical trial data as a viable alternative to concurrent controls in specific, well-defined contexts.