Technical Summary: B-CALM (Bias-Limited Bayesian Borrowing)

1. Problem Statement

Randomized Controlled Trials (RCTs) provide internally valid estimates of treatment effects but are often underpowered for estimating Conditional Average Treatment Effects (CATEs) across patient subgroups. Large-scale Observational Studies (OS) offer substantial sample sizes but suffer from confounding, population mismatch, and partial covariate overlap (where RCTs and OSs share some covariates $W$ but have unique sets $U$ and $V$, respectively).

The core challenge is to leverage OS data to reduce uncertainty in RCT-defined CATE estimation without allowing the biased OS data to dominate the inference (i.e., preventing "negative transfer" where the OS shifts the estimand away from the true RCT target). Existing methods often rely on point estimates with separately propagated uncertainty or global borrowing weights that do not explicitly model the functional form of bias under covariate mismatch.

2. Methodology: B-CALM

The authors propose B-CALM (Bayesian Calibrated ALignment under covariate Mismatch), a Bayesian framework that treats the OS as a biased, partially mismatched measurement of the RCT outcome surfaces.

2.1 Model Architecture

• Latent Alignment: Source-specific encoders ($q_{\phi_r}$ for RCT, $q_{\phi_o}$ for OS) map distinct covariate blocks ($X_r = (W, U)$ and $X_o = (W, V)$) into a shared latent patient state $Z$. This creates a common coordinate system for comparison.
• Outcome Surfaces:
  • RCT Surface: $\eta_r(a, z) = \mu(z) + a\tau(z)$, where $\mu(z)$ is the control prognosis and $\tau(z)$ is the target CATE.
  • OS Surface: $\eta_o(a, z) = \mu(z) + a\tau(z) + b_0(z) + ab_\Delta(z)$.
• Bias Decomposition:
  • $b_0(z)$: Baseline Bias, representing the difference in control outcomes between OS and RCT at the same latent state (e.g., due to unmeasured comorbidities).
  • $b_\Delta(z)$: Comparative Bias, representing the difference in treatment contrasts ($\tau_{OS} - \tau_{RCT}$) after accounting for baseline shifts. This is the critical parameter governing borrowing.

2.2 Probabilistic Formulation

The method places Gaussian Process (GP) priors on the outcome functions ($\mu, \tau$) and bias functions ($b_0, b_\Delta$).

• Borrowing Mechanism: The prior variance of the comparative bias, $\sigma^2_\Delta$, acts as a "sensitivity knob."
  • If $\sigma^2_\Delta \to 0$, the OS and RCT contrasts are assumed commensurate, allowing full pooling.
  • If $\sigma^2_\Delta \to \infty$, the OS contrast is allowed to be arbitrarily biased, reverting inference to RCT-only.
  • Intermediate values define a sensitivity analysis where the OS informs the RCT CATE only to the extent that the bias function is constrained by the prior.

2.3 Alignment and Calibration

The framework incorporates an Integral Probability Metric (IPM) to quantify the residual mismatch between the latent distributions of the two sources. The model optimizes a variational lower bound (ELBO) that includes:

1. Likelihood terms for both RCT and OS outcomes.
2. KL divergence from priors.
3. An alignment penalty (e.g., MMD or adversarial loss) to ensure the latent spaces are sufficiently aligned.

3. Key Theoretical Contributions

The paper establishes four main theoretical results:

1. Bias-Limited Information Bound: The authors prove that the information the OS can contribute about the RCT treatment-effect function is capped by the prior precision of the comparative bias function $b_\Delta$. Even as the OS sample size ($n_o$) approaches infinity, the information gain does not diverge; it saturates.
2. Effective Sample Size (ESS) Saturation: In a scalar special case (e.g., Average Treatment Effect), the authors derive an ESS formula: $n_{eff} = n_o / (1 + n_o \sigma^2_b / \sigma^2_o)$. As $n_o \to \infty$, $n_{eff}$ converges to $\sigma^2_o / \sigma^2_b$. This demonstrates that increasing OS sample size cannot overcome prior uncertainty about the bias.
3. Function-Valued Extension: Theorem 1 extends this saturation result to finite-dimensional representations of the CATE function, showing that the Fisher information for the treatment effect is bounded by the prior precision of the bias function in every direction.
4. PAC-Bayes Risk Decomposition: The authors provide a target-risk decomposition that separates the RCT empirical risk, the latent alignment error (IPM), and the residual calibration error of the debiased OS surface. This clarifies that alignment alone is insufficient; the debiased OS surface must also be calibrated to the RCT surface.

4. Experimental Results

The method was evaluated on three datasets: a synthetic factorial study, a semi-synthetic benchmark based on the Tennessee STAR trial, and a real-world pediatric obesity external-control study.

• Synthetic Study:
  • Coverage: B-CALM maintained near-nominal 90% credible interval coverage (average 0.939) across varying bias regimes. In contrast, pooled methods and Causal Forests (CF) became overconfident (coverage dropped to 0.581 and 0.811, respectively) under comparative bias.
  • Negative Transfer: B-CALM exhibited low negative transfer (0.023), meaning it rarely performed worse than RCT-only analysis.
  • Saturation: As OS sample size increased, B-CALM's credible interval widths flattened to a floor determined by $\sigma_\Delta$, whereas pooled methods continued to shrink toward zero, ignoring bias.
• Semi-Synthetic (STAR): B-CALM achieved coverage of 0.936 compared to 0.425 for pooled methods, with comparable RMSE.
• Real-Data (Pediatric Obesity): Using EHR data as external controls, B-CALM reduced the 90% credible interval width by 9.4% relative to RCT-only while maintaining a "bias-susceptibility ratio" below 1.0. Pooled methods achieved larger width reductions (up to 70%) but implied bias risks exceeding their reported interval widths (ratio > 1.0).

5. Significance and Claims

The paper positions B-CALM as a shift from "borrowing" as a point-estimation problem to "borrowing" as a posterior sensitivity analysis.

• Explicit Bias Modeling: Unlike methods that use global discount factors or exchangeability indicators, B-CALM explicitly models the bias function $b_\Delta(z)$. This allows the analyst to control where and how much borrowing occurs based on the prior precision of the bias.
• Safety Mechanism: The "bias-limited" property ensures that no matter how large the observational dataset becomes, the inference cannot be driven entirely by potentially biased data. The prior on the bias function acts as a hard ceiling on information transfer.
• Practical Output: The authors claim the primary output is not a single estimate, but a calibrated analysis including a posterior over the RCT-defined estimand, a sensitivity curve over prior bias scales, and an effective sample size diagnostic. This allows researchers to report how conclusions change as assumptions about OS bias vary, rather than committing to a single, potentially fragile, borrowing weight.

The authors acknowledge limitations, including the non-identifiability of $b_\Delta$ without priors (necessitating sensitivity analysis), the reliance on shared covariates for latent alignment, and computational costs at large scales.