Technical Summary: Deconfounding via Profiled Transfer Learning

1. Problem Statement

In regression-based effect estimation and causal inference, unmeasured confounders are a primary source of bias, leading to spurious correlations and invalid causal conclusions. Traditional deconfounding methods typically operate within single-dataset frameworks and rely on either direct estimation of confounding structures (e.g., factor analysis) or the incorporation of auxiliary variables (e.g., instrumental variables or proxy variables). These approaches often face practical challenges, such as the difficulty of selecting suitable auxiliary variables or the reliance on stringent assumptions like the exclusion restriction or parallel trends.

Furthermore, while transfer learning is increasingly used to leverage source datasets with similar structures to improve target estimation, conventional transfer learning paradigms fail to address hidden confounding. Direct application of standard transfer learning to confounded models does not exploit the shared confounding structures between source and target data, resulting in biased estimates of model shifts and target parameters.

2. Methodology: ProTrans Framework

The paper proposes ProTrans (Profiled Transfer Learning), a framework designed to mitigate confounding effects in a target dataset by leveraging additional source datasets that share similar confounding structures. The core innovation is the concept of profiled residuals, which characterize and transfer shared confounding patterns from sources to the target.

2.1 Model Setup

The target model is defined as a structural equation model with hidden confounders $H_t$:
$$Y_{ti} = \beta_t^\top X_{ti} + \phi_t^\top H_{ti} + \varepsilon_{ti}$$
$$X_{ti} = f(H_{ti}) + E_{ti}$$
where $\beta_t$ is the parameter of interest, and $f(\cdot)$ is an unknown mapping describing the confounding structure. The goal is to estimate the target parameter $\beta_t$ or the model shift $\eta_0 = \beta_t - \beta_s$ (where $\beta_s$ is the average source parameter).

2.2 The ProTrans Algorithm

The method proceeds in two main stages:

1. Source Parameter Estimation and Residual Construction:

   • Source data is pooled to estimate an initial source parameter $\hat{\beta}_{init}$ (which need not be deconfounded).
   • Profiled Source Residuals are constructed: $\hat{Z}_s^{(k)} = Y_s^{(k)} - X_s^{(k)}\hat{\beta}_{init}$. These residuals encode information about the hidden confounders in the source data.
   • A deconfounded source estimator $\hat{\beta}_s$ is obtained (e.g., via trim transform or LASSO on pooled source data). Due to the large sample size of source data ($n_s \gg n_t$), deconfounding the source models is more feasible and precise than in the target.

2. Profiled Residual Transfer and Target Estimation:

   • A profiled target residual $\hat{Z}_t$ is constructed by aligning the target covariate-residual correlation with the source covariate-residual correlation:
     $$\hat{Z}_t = \arg \min_{Z_t} \left\| \frac{1}{n_t} X_t^\top Z_t - \frac{1}{n_s} \sum_{k=1}^K X_s^{(k)\top} \hat{Z}_s^{(k)} \right\|_\infty$$
   • This step effectively transfers the confounding effects from the sources to the target, ensuring that the term $(X_t^\top X_t)^\dagger X_t^\top (H_t \phi_t - \hat{Z}_t)$ approximates the bias induced by the initial estimator.
   • Model Shift Estimation: The model shift $\eta_0$ is estimated by regressing the adjusted target response $(Y_t - \hat{Z}_t - X_t \hat{\beta}_{init})$ against $X_t$ using a penalized least squares estimator (e.g., LASSO).
   • Target Parameter Estimation: The final target parameter is estimated as $\hat{\beta}_t = \hat{\beta}_s + \hat{\eta}$.

2.3 Source Selection

To handle scenarios where confounding effects vary across sources, the authors propose an informative source selection procedure based on a newly proposed quality score. This allows the framework to filter out non-informative sources while preserving theoretical guarantees.

3. Theoretical Results

The paper establishes non-asymptotic error bounds and minimax optimality for the ProTrans estimators under mild conditions.

• Model Shift Estimation ($\hat{\eta}$):

  • The estimator achieves the oracle convergence rate of $\sqrt{s_\eta \log p / n_t}$, matching the optimal rate for models without hidden confounders.
  • Crucially, this rate is achieved without imposing specific assumptions on the true confounding structure (e.g., linearity) or requiring auxiliary variables.
  • In contrast, conventional transfer learning estimators ($\tilde{\eta}$) suffer from error bounds dominated by confounding bias, specifically terms involving $\sqrt{p \|\phi_t\|_2^2 / (n_t \lambda_q(\Psi)^2)}$, which can dominate the error if the confounding structure is ill-conditioned.

• Target Parameter Estimation ($\hat{\beta}_t$):

  • The estimation error consists of two components: the confounding-free error from estimating the model shift and the source estimation error.
  • The confounding bias is reduced from order $n_t^{-1/2}$ (in classical methods) to order $n_s^{-1/2}$ because the confounding effects are mitigated using the large source sample size.
  • The ProTrans estimator achieves the minimax lower bound for the target parameter estimation under mild conditions.

• Robustness:

  • The framework is robust to non-informative sources via the proposed selection strategy.
  • Theoretical guarantees hold even when the confounding structure $f(\cdot)$ is non-linear and unknown, provided the confounding effects are similar across source and target (or at least one source is similar).

4. Key Contributions and Significance

The authors summarize the contributions of ProTrans as follows:

1. First Transfer Learning Approach for Deconfounding: ProTrans is the first attempt to leverage additional source information specifically to mitigate confounding effects. Unlike existing deconfounding methods, it does not require restrictive assumptions about hidden confounding structures or the availability of auxiliary variables (instrumental or proxy variables).
2. Correction of Conventional Transfer Learning: The paper demonstrates that direct application of conventional transfer learning to confounded models fails to eliminate confounding-induced bias. ProTrans complements the transfer learning literature by explicitly transferring the confounding structure via profiled residuals.
3. Treatment Effect Estimation: As a byproduct, the framework offers a novel approach for estimating treatment effects where the treatment group serves as the target and the control group as the source. In this setting, the model shift directly yields the treatment effect estimate without needing instrumental variables.
4. Theoretical Optimality: The paper proves that ProTrans achieves minimax optimal rates for both model shift and target parameter estimation, significantly outperforming classical transfer learning methods in the presence of strong or complex confounding.

5. Empirical Validation

The effectiveness of ProTrans is validated through both simulated experiments and real-world data applications. These experiments confirm that ProTrans effectively mitigates latent confounding effects and supports the theoretical findings regarding convergence rates and robustness to non-informative sources.