Estimating Joint Interventional Distributions from Marginal Interventional Data

1. Problem Statement

The paper addresses the Causal Marginal Problem, a scenario where researchers possess data from disjoint subsets of variables (marginal data) rather than a full joint dataset. Specifically, the authors tackle the challenge of estimating joint interventional distributions (e.g., $P(Y | do(X_1, X_2))$) when only:

• Observational marginal data (e.g., $P(X)$, $P(Y|X_i)$) is available.
• Single-variable interventional data (e.g., $P(Y | do(X_i))$) is available for subsets of variables.

Context: In fields like medicine or agriculture, conducting Randomised Controlled Trials (RCTs) for all possible combinations of treatments (joint interventions) is often prohibitively expensive due to combinatorial explosion. Researchers often have data from separate studies (e.g., the effect of Drug A alone, Drug B alone, or observational data) but lack the joint distribution required to determine causal parents or predict the outcome of combined interventions. Existing methods for causal discovery (like ICP or KCI) typically require full joint observational data, which is unavailable in this setting.

2. Methodology: Interventional Causal Maximum Entropy (i-CMAXENT)

The authors propose i-CMAXENT, an extension of the Causal Maximum Entropy (CMAXENT) principle.

Core Concept

The Maximum Entropy (MAXENT) principle selects the probability distribution that maximizes entropy (is closest to uniform) while satisfying given statistical constraints (empirical averages). CMAXENT adapts this to causal settings by ordering variables according to a causal graph.

The Extension:
The authors extend CMAXENT to incorporate interventional constraints.

1. Optimization Objective: Maximize the conditional entropy $H(Y|X)$ subject to:
   • Observational constraints (marginal and conditional expectations).
   • Interventional constraints: Expectations derived from single-variable interventions, e.g., $E[h_l(Y, x_{S^I_l}, x_{S^C_l}) | do(X_{S^I_l})]$.
2. Identifiability Assumption: The method assumes that the interventional expectations can be expressed as functionals of observational distributions (nonparametric identification). This relies on Assumption 5 (Weak Causal Sufficiency), which posits no unobserved confounders between the effect $Y$ and its potential causes $X$, though confounders among $X$ are allowed.
3. Theoretical Result (Theorem 8): Using Lagrange duality, the authors prove that the solution to the i-CMAXENT optimization problem lies in the exponential family. The resulting distribution takes the form:
   $$p_\lambda(y|x) = \exp\left( \sum \lambda_k f_k + \sum \lambda_j g_j + \sum \lambda_l h_l + \beta(x) \right)$$
   where $h_l$ represents the interventional functions. This confirms that incorporating interventional data does not break the mathematical structure of the MAXENT solution.

Parental Discovery Mechanism

The method infers causal parents by analyzing the Lagrange multipliers of the solution.

• Proposition 9: If a variable $X_j$ has no children other than potentially $Y$, the interventional distribution $P(Y | do(X_j))$ is identifiable using the rest of the variables as an adjustment set.
• By fitting the model, non-zero Lagrange multipliers associated with specific variables indicate a causal link, allowing for parental discovery even when variables are not jointly observed.

3. Key Contributions

1. Extension of CMAXENT: The paper formally extends the Causal Maximum Entropy principle to include data from experimental conditions (interventions), creating i-CMAXENT.
2. Theoretical Proof: It proves that the solution to the optimization problem with interventional constraints remains within the exponential family, generalizing previous MAXENT results to causal semantics.
3. Nonparametric Identification: It demonstrates conditions under which joint interventional distributions can be identified nonparametrically from a mixture of observational and single-variable interventional marginal data, even in the presence of confounding among causes.
4. Parental Discovery under Marginal Constraints: It provides a method to identify causal parents using only marginal data, overcoming the limitation of previous CMAXENT versions that could only use observational data.

4. Experimental Results

The authors evaluated i-CMAXENT on synthetic data using three different graph structures (varying degrees of confounding and connectivity).

Task 1: Parental Discovery

• Comparison: i-CMAXENT was compared against standard CMAXENT (observational only) and the Kernel Conditional Independence (KCI) test (which assumes access to full joint data).
• Findings:
  • i-CMAXENT consistently outperformed CMAXENT.
  • It achieved accuracy comparable to KCI, despite KCI having access to the full joint distribution $P(X, Y)$ while i-CMAXENT only had marginal data.
  • Performance improved as the proportion of variables with interventional data increased.

Task 2: Estimating Joint Interventional Distributions

• Goal: Estimate $P(Y | do(X_1, X_2))$ using only single-variable intervention data ($P(Y | do(X_i))$) and/or conditional data.
• Findings:
  • Adding constraints (whether interventional or conditional) significantly reduced the residual error compared to unconstrained MAXENT.
  • i-CMAXENT reduced residuals to approximately 5% even when relying solely on single-variable interventional or conditional information.
  • The method successfully reconstructed joint interventional distributions without requiring distributional assumptions (e.g., linearity or Gaussianity) or additive noise models.

5. Significance and Implications

• Bridging Data Gaps: The method provides a principled framework for merging data from disparate studies (e.g., separate clinical trials for different drugs) to infer the effects of combined treatments, a critical need in complex domains like medicine and policy.
• Efficiency: It reduces the need for expensive, high-dimensional RCTs by leveraging existing single-variable intervention data and observational data.
• Robustness: Unlike many parametric approaches, i-CMAXENT is nonparametric and does not require assumptions about the functional form of the causal mechanisms (e.g., linearity), making it applicable to a wider range of real-world systems.
• Limitations: The method requires knowing which variables were intervened upon to formulate the constraints. It also relies on the identifiability of the interventional distributions, which depends on the underlying causal graph structure (specifically the absence of backdoor paths from causes to the effect).

In summary, this paper establishes a rigorous theoretical and practical foundation for combining marginal interventional data to solve causal inference problems that were previously intractable without full joint observations.