📊 statistics

Improving Precision of RCT-Based CATE Estimation using Data Borrowing with Double Calibration

The paper proposes R-OSCAR, a robust two-stage framework that leverages large observational studies to improve the precision of conditional average treatment effect estimation in randomized controlled trials by calibrating outcome predictions and correcting bias, thereby significantly reducing the sample size required to detect heterogeneous treatment effects while maintaining unbiasedness.

Original authors: Amir Asiaee, Chiara Di Gravio, Cole Beck, Yuting Mei, Samhita Pal, Jared D. Huling

Published 2026-07-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Amir Asiaee, Chiara Di Gravio, Cole Beck, Yuting Mei, Samhita Pal, Jared D. Huling

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out which medicine works best for which type of person. This is the goal of personalized medicine. Usually, the gold standard for solving this mystery is a Randomized Controlled Trial (RCT). Think of an RCT as a super-strict, perfectly organized experiment where a small group of volunteers is randomly assigned to different treatments. Because the group is small and carefully controlled, the results are trustworthy, but they often lack the numbers to spot subtle differences between patients. It's like trying to find a specific needle in a haystack when you only have a tiny handful of hay.

On the other side of the street, you have Observational Studies (OS). These are massive piles of data collected from real-world hospitals and records. They have millions of people, so they have plenty of hay to find needles in. But there's a catch: the data is messy. People weren't randomly assigned; they chose their treatments based on their own habits, doctors' preferences, or hidden factors. It's like trying to solve a mystery using a stack of unsorted, handwritten notes where the writer might have been biased.

The Big Problem

Scientists have long wanted to combine these two worlds: use the massive size of the Observational Studies to help the small, trustworthy RCTs. But there's a big fear. If you just mash the messy notes into the strict experiment, you might ruin the experiment's trustworthiness. Most previous attempts tried to assume that the "rules" of how the medicine works are exactly the same in both the messy notes and the strict experiment. The authors of this paper argue that this assumption is often wrong. The "rules" might shift slightly between the real world and the lab.

The New Solution: R-OSCAR

The authors propose a new framework called R-OSCAR (Robust Observational Studies for CMO-Augmented RCT). Here is how it works, using a creative analogy:

Imagine the RCT is a master chef in a tiny, high-end kitchen. The chef knows exactly how to cook a perfect dish (the treatment effect) but only has a few ingredients (a small sample size). The Observational Study is a massive, chaotic food truck with a million customers and a huge pantry, but the cooking instructions are scribbled on napkins and might be slightly off.

Instead of just dumping the food truck's messy ingredients into the chef's kitchen (which would ruin the dish), R-OSCAR acts like a smart sous-chef.

The Taste Test: The sous-chef takes the food truck's recipes (the observational data) and tries them out in the chef's kitchen.
The Calibration: The sous-chef notices the difference between the food truck's taste and the chef's perfect taste. They don't throw away the food truck's data; instead, they calculate a "correction factor" (a discrepancy) to adjust the truck's recipe so it fits the chef's kitchen.
The Final Dish: The chef uses this corrected recipe to predict how the dish will taste for different types of eaters (patients).

The magic trick is that the final prediction is still anchored in the chef's (RCT's) truth. Even if the food truck's original recipe was totally wrong, the correction step ensures the final result remains unbiased.

What They Found (The Numbers)

The authors didn't just guess; they tested this with computer simulations and real-world data.

The Simulation Results: In their computer experiments, they found that using R-OSCAR could reduce the number of people needed in the RCT by up to 75% to detect the same treatment effects. For example, if a study usually needed 1,000 patients to get a clear answer, R-OSCAR could get the same precision with just 250 patients, provided they had access to a large observational dataset of 10,000 people.
The "Safety Net": The paper also introduces a "diagnostic" tool. Think of this as a smoke detector. Before the chef uses the food truck's data, the smoke detector checks if the data is safe to use. If the observational data is too messy or biased (like if the food truck is serving rotten food), the detector sounds the alarm, and the system automatically switches back to using only the chef's own data. This ensures that borrowing data never makes the results worse than if they hadn't borrowed at all.
Real-World Tests: They tested this on two real scenarios:
1. A semi-synthetic study of the Tennessee STAR class-size experiment, where they artificially created messy data to see if the method could recover the truth. It worked perfectly.
2. The Greenlight Plus pediatric obesity trial, which was linked with real electronic health records. Here, the method successfully improved the estimation for the control group (the kids who didn't get the digital intervention) by borrowing from the massive external records, but only when those records actually covered the same types of patients as the trial.

What They Explicitly Rule Out

The paper is very clear about what does not work:

Blindly assuming the rules are the same: They argue against the idea that the treatment effects (how the medicine works) are identical between the real world and the lab. R-OSCAR explicitly allows for these differences to exist and corrects for them.
Borrowing without checking: They show that if the observational data is too biased or the "correction" needed is too complex (like if the difference between the truck and the kitchen is huge and messy), the method naturally defaults to not borrowing, rather than forcing a bad fit.

How Sure Are They?

The authors are confident in their simulations, where they proved mathematically that their method reduces error under specific conditions (like when the differences between the datasets are "sparse," meaning only a few things are different). They demonstrated this with 100 different simulation runs for various scenarios.

In the real-world tests, they showed that the method can improve precision and that the "smoke detector" diagnostic works as intended. However, they present these as successful validations of the framework rather than a permanent solution to every medical mystery. The method suggests that by carefully calibrating big, messy data against small, clean data, we can make our medical experiments much more efficient without losing their trustworthiness.

In short, R-OSCAR is a clever way to let a small, perfect experiment borrow strength from a giant, messy one, but only if it double-checks the math first to make sure the borrowing doesn't introduce any new errors.

Technical Summary: Improving Precision of RCT-Based CATE Estimation using Data Borrowing with Double Calibration

1. Problem Statement

Randomized Controlled Trials (RCTs) are the gold standard for causal inference but are often underpowered to detect Heterogeneous Treatment Effects (HTEs) across diverse patient subgroups due to constraints on cost, time, and sample size. Conversely, large-scale Observational Studies (OS) offer richer data and broader population coverage but suffer from unmeasured confounding and potential outcome distribution shifts (concept shift) relative to the RCT population.

Existing methods for integrating RCT and OS data primarily focus on generalizing RCT findings to broader OS populations ( $R \to O$ ) or borrowing historical control arms to reduce variance in Average Treatment Effect (ATE) estimation. However, there is a lack of robust frameworks for improving the precision of Conditional Average Treatment Effect (CATE) estimation within the RCT population ( $O \to R$ ) by leveraging both treatment arms of OS data without assuming that CATEs or outcome means are exchangeable across populations. The challenge lies in utilizing the large sample size of OS data to reduce variance while correcting for confounding and outcome shifts to preserve the unbiasedness guarantees of the RCT.

2. Methodology

The authors propose a framework that frames CATE estimation as a supervised learning problem, utilizing a Counterfactual Mean Outcome (CMO) as an augmentation function. The core insight is that estimation variance is minimized when the augmentation function equals the true CMO.

2.1 Theoretical Foundation

Supervised Learning Formulation: The CATE $\tau(x)$ is estimated by minimizing the squared loss of a pseudo-outcome $\tau_m(X, A, Y) = \frac{A(Y - m(X))}{\pi_A(X)}$ .
Optimal Augmentation: The risk-minimizing choice for the augmentation function $m(X)$ is the CMO, defined as $\mu(x) = \sum_a \pi_{-a}(x)\mu_a(x)$ , where $\mu_a(x)$ is the potential outcome mean for treatment $a$ .
Error Bounds: The authors derive finite-sample error bounds showing that the Mean Squared Error (MSE) of the CATE estimator depends on the MSE between the estimated augmentation function and the true CMO. Specifically, improving the estimation of the CMO directly reduces both prediction risk and estimation error.

2.2 The R-OSCAR Framework

To leverage OS data for CMO estimation without introducing bias, the authors introduce R-OSCAR (Robust Observational Studies for CMO-Augmented RCT), a two-stage estimator that relaxes the assumption of outcome mean exchangeability.

Key Steps:

OS Outcome Modeling: Estimate outcome models $\hat{\mu}_o(x, a)$ using the large OS dataset.
Outcome Calibration (Stage 1): Estimate the discrepancy $\hat{\delta}_a(x)$ between the OS outcome model and the RCT outcome model using RCT data. This step treats the discrepancy as a structured function (e.g., sparse or low-dimensional) to be learned via regularized regression (e.g., LASSO). The calibrated outcome mean is $\hat{\mu}_o(x, a) + \hat{\delta}_a(x)$ .
CMO Construction: Construct the CMO $m(x)$ using the calibrated OS outcome means and RCT propensity scores.
CATE Calibration (Stage 2): To address regularization bias and potential misspecification in the first stage, a second calibration step estimates a CATE discrepancy function $\hat{\delta}(x)$ . This corrects residual biases in the preliminary CATE estimate derived from the calibrated CMO.

Algorithmic Implementation:

The method employs sample splitting (cross-fitting) to ensure theoretical validity, using independent folds of the RCT data for outcome calibration and CATE calibration.
It supports single-arm borrowing (e.g., borrowing only control data from OS when the intervention is unique to the RCT) by adapting the CMO construction.

2.3 Diagnostic for Borrowing

A cross-fitted RCT diagnostic is proposed to determine a priori whether borrowing from a specific OS is supported. This diagnostic performs a paired hypothesis test on held-out RCT data to compare the predictive accuracy of RCT-only outcome models versus OS-calibrated models. If the OS-calibrated models do not significantly outperform the RCT-only models, the framework defaults to the RCT-only estimator (RACER), ensuring robustness against severe model misspecification or unmeasured confounding.

3. Key Contributions

$O \to R$ Framework: Developed a framework that borrows both treatment arms from OS data to improve CATE estimation within the RCT population, moving beyond simple control-arm borrowing.
Relaxed Assumptions: Relaxed the strong assumption of CATE or outcome mean exchangeability across populations. Instead, the method models the discrepancy between populations as a structured, learnable function (e.g., sparse shifts).
Learning-Theoretic Bounds: Derived non-asymptotic error bounds demonstrating that CATE estimation efficiency depends on the accuracy of the CMO augmentation function.
R-OSCAR Estimator: Introduced a doubly-calibrated estimator that separates CMO estimation from CATE estimation, providing consistency guarantees even if the OS outcome models are misspecified, provided the CATE discrepancy function is correctly specified.
Diagnostic Tool: Provided a data-driven diagnostic to certify when borrowing is beneficial, preventing efficiency loss in scenarios where OS data is too biased or structurally different.
Empirical Validation: Validated the method through extensive simulations and two real-world case studies.

4. Results

4.1 Simulation Studies

Sample Size Reduction: In linear settings with sparse discrepancies, R-OSCAR achieved comparable or better performance than RCT-only methods (Naive and RACER) using only 250 RCT samples, whereas the RCT-only methods required 1,000 samples. This represents a potential 75% reduction in required RCT sample size for HTE detection.
Robustness to Misspecification: R-OSCAR maintained superior performance over baselines even when outcome models were misspecified (e.g., containing quadratic or sinusoidal terms) and when baselines used flexible methods like Random Forests.
Unmeasured Confounding: The method remained robust to unmeasured confounding. As the strength of unmeasured confounding increased, the diagnostic correctly reduced the probability of selecting R-OSCAR, defaulting to the unbiased RCT-only estimator when bias became too severe.
Dimensionality: The advantage of R-OSCAR increased with the dimensionality of covariates ( $p$ ), aligning with theoretical predictions regarding sparse discrepancies.

4.2 Real-World Case Studies

Tennessee STAR Study (Semi-synthetic): Using a constructed observational cohort from the STAR trial with induced confounding, R-OSCAR and OSCAR significantly outperformed the RCT-only baseline (RACER) in recovering the ground-truth CATE, demonstrating unbiasedness and efficiency gains.
Greenlight Plus Trial (External EHR): In a pediatric obesity trial linked with external Electronic Health Record (EHR) data (controls only), the one-arm adaptation (R-OSCAR-1arm) improved control-arm estimation. The diagnostic successfully certified borrowing only when the EHR data covered the trial population, failing to recommend borrowing when coverage was insufficient.

5. Significance and Claims

The paper claims to offer a principled approach to integrating observational and experimental data using tools from statistical learning and transfer learning. Its significance lies in:

Preserving Unbiasedness: Unlike methods that directly combine datasets or assume transportability, R-OSCAR preserves the unbiasedness of RCT estimates by anchoring the final CATE estimation in the RCT data through calibration and discrepancy correction.
Efficiency Gains: It demonstrates that large observational datasets can be leveraged to drastically reduce the sample size requirements for detecting heterogeneous treatment effects, a critical factor for personalized medicine.
Robustness: By explicitly modeling outcome shifts and employing a diagnostic to detect when borrowing is inappropriate, the framework avoids the pitfalls of naive data integration, such as bias inflation due to unmeasured confounding.
Theoretical Rigor: The work connects transfer learning theory to causal inference, providing finite-sample error bounds that justify the use of OS data for CMO augmentation under specific structural assumptions (e.g., sparse discrepancies).

The authors conclude that their framework enables more efficient use of available data for personalized treatment strategies while maintaining the rigorous causal guarantees inherent to RCTs.