Causal Gaussian Processes for Robust Treatment Effect Evaluation with Unobserved Confounding

Technical Summary: Causal Gaussian Processes for Robust Treatment Effect Evaluation with Unobserved Confounding

Problem Statement

The central challenge addressed in this paper is the evaluation of causal treatment effects from observational data contaminated by unobserved confounding. In many practical scenarios, the target causal effects (e.g., $E_x[Y]$) are non-identifiable from observational data alone because unmeasured confounders introduce spurious correlations between treatments ($X$) and outcomes ($Y$).

While standard Gaussian Process (GP) regression excels at modeling complex functional mappings in continuous domains, it is vulnerable to learning and amplifying biases present in observational data. Consequently, standard GPs often fail to generalize to interventional distributions (what would happen if $X$ were set to a specific value $x$), leading to incorrect policy evaluations. Existing robust strategies often rely on strong parametric assumptions, detailed prior knowledge of the environment, or are restricted to discrete treatment and outcome spaces, limiting their applicability to complex, continuous domains.

Methodology

The authors propose a novel framework that bridges the gap between non-parametric causal inference and Gaussian process modeling through three main methodological steps:

1. Universal Discretization of Exogenous Domains
The paper introduces a universal discretization procedure for the exogenous variables ($U$) in any Structural Causal Model (SCM) with continuous treatment and outcome domains.

• The authors define a Causal Canonical Model (CCM) where the structural functions determining covariates ($Z$), actions ($X$), and rewards ($Y$) are simple functions (linear combinations of indicator functions over disjoint subsets of the exogenous domain).
• Theorem 3.3 establishes that the set of these Causal Canonical Models is dense in the space of all SCMs with respect to the $L_1$ norm. This means that for any arbitrary causal model and any desired accuracy $\epsilon$, there exists a CCM that approximates both the observational distribution $P(X, Y, Z)$ and the interventional treatment effects $E_x[Y|z]$ within $\epsilon$.

2. Reparameterization into Causal Generative Models (CGM)
To facilitate learning, the discrete simple functions of the CCM are reparameterized into Causal Generative Models (CGM) using continuous functions.

• In a CGM, the population is stratified into a finite number of latent states (strata) $u \in \{1, \dots, d\}$.
• Within each stratum, the reward function $h_u(x, z)$ is a continuous function mapping treatment and covariates to the outcome.
• Theorem 4.2 confirms that this family of CGMs retains the causal approximation property, effectively representing any SCM's observational and interventional distributions.

3. Causal Gaussian Processes (CGP) and Variational Inference
The authors instantiate the CGM family using Gaussian Processes to handle the continuous nature of the reward functions $h_u$.

• Model Structure: The exogenous distribution $P(u)$ is modeled via a truncated Dirichlet process (stick-breaking process). For each latent stratum $u$, the covariates $Z$ and treatment $X$ are drawn from conditional multivariate normal distributions. The outcome $Y$ is generated by a function $h_u(x, z)$ drawn from a Gaussian Process with a specific kernel (e.g., Radial Basis Function).
• Learning: Since exact posterior inference is intractable, the authors employ Variational Inference (VI) with mean-field approximation. The model learns a posterior distribution over the latent strata assignments and the parameters of the Gaussian processes.
• Inference: The target treatment effect $E_x[Y|z]$ is computed by averaging the learned reward functions $h_u(x, z)$ weighted by the posterior probability of the latent strata $P(u|z)$. This allows the model to "search" over potential partitions of the exogenous domain to find a robust explanation of the data that accounts for unobserved confounding.

Key Contributions

1. Universal Discretization: The paper provides a formal proof that exogenous domains in continuous causal models can be discretized without loss of generality to approximate any causal model's observational and interventional distributions with arbitrary accuracy.
2. Causal Gaussian Process (CGP) Family: The authors develop a new family of generative models that reparameterize the canonical discretization into a form compatible with standard Bayesian inference and deep learning frameworks.
3. Robust Partial Identification: The framework enables the derivation of robust posterior distributions over causal effects from confounded observational data, effectively performing partial identification in continuous domains without requiring strong parametric assumptions about the underlying structural functions.

Experimental Results

The authors evaluated the proposed CGP against standard Gaussian Process regression on synthetic datasets with various reward functions (Polynomial, Logistic, Phase, Linear) and a real-world dataset from the International Stroke Trial (IST).

• Metric: Performance was measured using the Negative Log Predictive Likelihood (NLPL) on interventional data (data from randomized trials or simulated interventions).
• Findings:
  • Standard GPs consistently overfit the confounded observational data, resulting in high NLPL scores and failure to generalize to interventional distributions.
  • Causal GPs consistently achieved significantly lower NLPL scores across all synthetic and real-world experiments.
  • The posterior distributions learned by CGP successfully captured the true treatment effects, whereas standard GPs deviated significantly due to unobserved confounding.

Significance and Claims

The paper claims that this work generalizes the canonical partitioning of structural causal models (previously established for discrete domains) to continuous domains. By introducing a universal discretization property, the authors demonstrate that it is possible to approximate complex, unknown causal mechanisms with a finite number of latent states.

The significance lies in providing a robust, data-driven approach to causal inference in settings where:

• Unobserved confounding is present.
• Variables are continuous.
• Detailed parametric knowledge of the generative process is unavailable.

The authors assert that their method allows for the "robust partial identification" of causal effects, enabling learners to infer treatment effects that generalize to interventional scenarios even when the observational data is biased. The work positions CGPs as a tool that automates the discretization of latent spaces, making robust causal inference feasible in complex, real-world continuous environments.