📊 statistics

Regression-Based Proximal Reconciliation of Conflicting Trials with Unmeasured Effect Modifiers

This paper introduces a causal inference framework utilizing regression-based tests and proxy variables to formally evaluate and quantify the reconcilability of conflicting randomized controlled trials caused by unmeasured effect modifiers, demonstrating its application through an analysis of the Meis and PROLONG trials.

Original authors: Daniel A Xu, Eric J Tchetgen Tchetgen, Enrique F Schisterman, Sean C Blackwell, Ellen C Caniglia

Published 2026-08-06

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel A Xu, Eric J Tchetgen Tchetgen, Enrique F Schisterman, Sean C Blackwell, Ellen C Caniglia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of a crime scene, your clues are medical studies. In the world of science, specifically a field called causal inference, researchers try to figure out if a specific treatment (like a new medicine) actually causes a good outcome (like getting better). Usually, they do this with Randomized Controlled Trials (RCTs), which are like perfectly fair coin-flip experiments where patients are randomly assigned to get the medicine or a fake pill. The goal is to see if the medicine works.

But here's the twist: sometimes, two different studies testing the exact same medicine, with the exact same rules, come to completely different conclusions. One study says, "This medicine is a miracle!" while the other says, "This medicine does nothing!" This is a huge problem for doctors and regulators who need to know what to prescribe. The big question is: Are these studies actually contradicting each other, or are they just looking at different groups of people?

To understand this, you need to know about effect modifiers. Think of these as hidden variables that change how a medicine works. For example, a painkiller might work great for someone who is young and healthy, but not at all for someone who is elderly and has other health issues. If Study A mostly enrolled young, healthy people, and Study B mostly enrolled older, sicker people, they might get different results even if the medicine works the same way for each specific type of person. The challenge for scientists is to mathematically prove whether the difference in results is just because the "ingredients" of the two groups were different, or if the studies are truly incompatible.

The Detective's New Toolkit: Reconciling the Confusing Trials

In this paper, a team of statisticians led by Daniel Xu and colleagues tackles a famous medical mystery: the conflicting results of two trials testing a drug called 17-alpha-hydroxyprogesterone caproate (17OHP-C), which was supposed to prevent premature birth.

The first trial, called Meis, found that the drug was a huge success, reducing the risk of premature birth by nearly 19%. But a second, larger trial called PROLONG found absolutely no benefit. The drug was eventually pulled from the market because of this conflict. The big question was: Did the drug work for the women in the Meis trial but not the PROLONG trial? Or was the drug actually the same, but the women in the two trials were just different in ways the researchers couldn't see?

The authors developed a new statistical "detective kit" to solve this. They call it Regression-Based Proximal Reconciliation. Let's break down what that means using a simple analogy.

The "Proxy" Problem

Imagine you are trying to figure out why two groups of students got different test scores. You know that Study A had mostly students who studied all night, while Study B had mostly students who slept well. You also suspect there is a hidden factor, let's call it "Brain Power," that affects scores. But you can't measure "Brain Power" directly; it's invisible.

However, you do have some clues, or proxies, that are related to "Brain Power." Maybe you know how many hours they spent reading (Proxy 1) and how many video games they played (Proxy 2). Even though you can't see "Brain Power," these proxies might give you a hint about it.

The authors' method uses these proxies (like reading habits or video game time) to mathematically "reconstruct" the invisible "Brain Power" and see if, once you account for it, the two groups of students actually perform the same. In the medical trial, the "invisible factor" was something like cervical length (a biological measure of pregnancy risk), which wasn't measured in the first trial but was suspected to be different between the groups. The proxies were things like how many prior premature births a woman had or her pre-pregnancy weight.

The Two Ways to Check: "Conditional" vs. "Marginal"

The paper introduces two ways to check if the trials can be reconciled:

Conditional Reconcilability (The "Same Person" Test): This asks, "If we took a specific woman from the Meis trial and a woman from the PROLONG trial who had the exact same characteristics (age, weight, and the invisible 'Brain Power'), would the drug work the same for both?" If the answer is yes, the trials are reconcilable. The authors developed a powerful new math test to check this, which is much better at spotting differences than older methods.
Marginal Reconcilability (The "Average Person" Test): This asks, "If we take the results from the Meis trial and mathematically adjust them to look like the PROLONG trial's population, do the results match what the PROLONG trial actually found?" This is a weaker test because it only cares about the average, not the specific individuals.

The "Equivalence" Twist

Usually, scientists try to prove that two things are different. But here, the authors wanted to prove that two things are similar enough to be considered the same. They used a technique called Equivalence Testing.

Think of it like a judge deciding if two suspects are the same person. Instead of asking, "Are they different?" (which is easy to prove), the judge asks, "Are they so similar that the difference doesn't matter?" The judge sets a "margin of error" (like 1 week of pregnancy). If the difference between the two trials is smaller than that margin, they are considered "reconciled." The authors also created a new score called the Reconciliation Proportion, which tells you what percentage of the conflict between the trials can be explained by the differences in the groups. A score of 100% means the groups explain everything; 0% means the groups explain nothing.

What Did They Find?

The authors applied their new toolkit to the Meis and PROLONG trials, using different sets of "proxies" (like prior birth history and body weight) to guess at the invisible "cervical length" factor.

The Result: The new, powerful "Conditional" test (the "Same Person" test) found statistically significant evidence that the trials were NOT reconcilable in one of their analyses. This suggests that even after accounting for the visible differences and the invisible factors guessed by the proxies, there are still other hidden differences between the two groups that explain why the drug worked in one trial but not the other.
The "Average" Test: When they looked at the "Average Person" (Marginal Reconcilability), the results were mixed. The math suggested that the invisible factors might explain some of the difference, but the evidence wasn't strong enough to say for sure.
The "Equivalence" Test: When they tried to prove the trials were similar enough (within a 1-week margin of pregnancy), they failed. The difference between the transported results and the actual results was too big to be ignored.

The Bottom Line

The paper concludes that the selected proxies (the clues they used to guess at the invisible factors) were likely too weak to fully explain the conflict. The "Reconciliation Proportion" scores were low or had huge ranges of uncertainty, meaning the math couldn't confidently say, "Ah, it was just the different groups!"

Instead, the findings suggest that the conflict between the Meis and PROLONG trials is likely due to other hidden factors that the researchers didn't measure or couldn't guess at with their proxies. These could be things like differences in how the hospitals were run, access to quality care, or other biological factors.

In short, the authors built a sophisticated new mathematical microscope to look for hidden reasons why medical trials disagree. When they pointed it at the famous 17OHP-C drug trials, the microscope showed that the "different groups" theory wasn't the whole story. The trials likely remain irreconcilable because there are still invisible forces at play that we haven't figured out how to measure yet. The paper doesn't solve the mystery of the drug, but it provides a much better way to ask the right questions about why studies disagree.

Technical Summary: Regression-Based Proximal Reconciliation of Conflicting Trials with Unmeasured Effect Modifiers

Problem Statement
Randomized controlled trials (RCTs) with similar protocols often yield conflicting findings when the distribution of relevant effect modifiers differs across study populations. While the transportability literature provides frameworks for generalizing treatment effects, no formal statistical framework exists to define and assess whether conflicting trials are reconcilable—specifically, whether differences in study populations explain the discrepancy. A critical gap in existing methods is their reliance on measured covariates; if unmeasured effect modifiers differ in distribution across studies, standard transportability approaches may fail to detect or explain conflicts. Furthermore, existing tests often lack power for conditional reconcilability and rely on null hypothesis significance testing (NHST), where failure to reject the null does not provide evidence for reconcilability.

Methodology
The authors propose a causal inference framework that evaluates both conditional and marginal reconcilability in the presence of unmeasured effect modifiers ( $U$ ) using proxy variables.

Definitions of Reconcilability:
- Conditional Reconcilability: Treatment effects conditional on both measured ( $X$ ) and unmeasured ( $U$ ) effect modifiers are identical across trials.
- Marginal Reconcilability: The marginal treatment effect observed in one trial can be recovered by transporting the conditional causal effect from the other trial and marginalizing over the target population's distribution of $(X, U)$ .
Proximal Causal Framework:
The method leverages a pair of proxy variables: a reweighting proxy ( $Z$ ) and an adjustment proxy ( $W$ ), which are hypothesized to be associated with the unmeasured effect modifier $U$ . The framework relies on standard causal assumptions (consistency, exchangeability, positivity) and specific proxy structure assumptions (e.g., $Z$ and $W$ share no common causes other than $X$ and $U$ , and $Z$ does not affect the outcome $Y$ directly).
Statistical Testing Approaches:
- Additive and Multiplicative Scales: The authors develop tests for reconcilability on both scales.
- NHST for Conditional Reconcilability: Instead of relying on nonparametric bridge functions (which suffer from slow convergence), the authors adopt a parametric structural model approach. By assuming linear structural models for the outcome and proxies, they derive expressions for the observed data conditional mean that eliminate dependence on the unobserved $U$ . This reduces the problem of testing conditional reconcilability to a coefficient testing procedure (Wald test) on specific regression coefficients in a pooled model. This approach is shown to have greater power than transportability-based methods for testing conditional equality.
- NHST for Marginal Reconcilability: The authors extend existing transportability methods to test if transported marginal effects match observed marginal effects, accounting for unmeasured modifiers via the proxy structure.
- Equivalence Testing: To address the limitation that failing to reject the null does not confirm reconcilability, the authors reframe the problem as an equivalence test. They define a prespecified equivalence margin ( $\delta$ ) representing the smallest clinically meaningful difference. Rejection of the null hypothesis (that the difference is $\ge \delta$ ) provides evidence in favor of marginal reconcilability.
- Reconciliation Proportion: A new metric is introduced to quantify the degree of marginal reconciliation, defined as the reduction in discrepancy between transported and observed effects relative to the original discrepancy between trials.

Key Results (Application to Meis and PROLONG Trials)
The methods were applied to the conflicting Meis and PROLONG trials regarding 17-alpha-hydroxyprogesterone caproate (17OHP-C) for preventing recurrent preterm birth. The analysis hypothesized that baseline cervical length was an unmeasured effect modifier, using proxies such as the number of prior preterm births and gestational age at prior preterm birth.

Analysis 1 (Proxies: Prior PTB count and Gestational Age at prior PTB): Neither the coefficient testing approach nor the transportability approach yielded statistically significant results against the null of reconcilability at $\alpha=0.05$ . However, equivalence tests failed to reject the null of non-reconcilability at a 1-week margin, and Least Equivalent Allowable Differences (LEADs) were large (approx. 2 weeks). The reconciliation proportions were high but had wide confidence intervals.
Analysis 2 (Proxies: Gestational Age at prior PTB and Pre-pregnancy BMI): The coefficient testing approach yielded a statistically significant result ( $p=0.022$ ), rejecting the null of conditional reconcilability. This suggests the presence of additional, conditionally imbalanced unmeasured effect modifiers not captured by the proxies. Equivalence tests also failed to provide evidence for marginal reconcilability at the 1-week margin.

Significance and Claims
The paper claims to provide the first formal statistical framework for defining and assessing the reconcilability of conflicting RCTs in the presence of unmeasured effect modifiers. Key contributions include:

Formalization: Defining distinct concepts of conditional and marginal reconcilability.
Power Improvement: Demonstrating that a regression-based coefficient testing approach offers superior power for testing conditional reconcilability compared to traditional transportability methods.
Evidence for Reconcilability: Introducing an equivalence testing framework that allows researchers to provide evidence supporting reconcilability, rather than merely failing to reject irreconcilability.
Quantification: Providing the reconciliation proportion to measure the extent to which population differences explain trial discrepancies.

The authors conclude that while their methods provided mixed evidence for the specific Meis/PROLONG case (potentially due to weak proxies for cervical length), the framework offers a principled way for regulators and clinicians to evaluate whether population differences explain conflicting trial findings. They acknowledge limitations, including reliance on parametric models and the need for valid proxies, and suggest future work in semiparametric extensions and sensitivity analyses.