Technical Summary: CAUSALGUARD

Problem Statement

Estimating individualized treatment effects (Conditional Average Treatment Effect, or CATE) from observational data requires selecting a valid adjustment set to block confounding. This selection is contingent upon the underlying causal Directed Acyclic Graph (DAG), which is rarely known in practice.

• Graph Misspecification: Relying on a single, potentially incorrect graph can lead to invalid adjustment sets (e.g., omitting confounders or adjusting for colliders), destroying identification or inducing bias.
• Limitations of Current Conformal Methods: While conformal inference offers finite-sample, distribution-free coverage guarantees, standard "graph-agnostic" wrappers often fail to provide valid coverage for the causal estimand if the underlying adjustment set is invalid. Furthermore, averaging intervals from multiple graphs after calibration introduces a Jensen gap, leading to unnecessarily wide intervals (large padding) to regain nominal coverage.

Methodology: CAUSALGUARD

The authors propose CAUSALGUARD, a structure-weighted conformal framework that integrates graph uncertainty directly into the conformal calibration process rather than treating it as a post-hoc error source. The pipeline proceeds in four stages:

1. Probabilistic Structural Prior Elicitation

Instead of assuming a fixed graph, the method constructs a proposal distribution over candidate DAGs:

• LLM-Derived Priors: A pre-trained autoregressive language model is queried to estimate edge probabilities ($P_{prior}(u \to v)$) based on domain descriptions. These serve as a proposal mechanism, not a source of guaranteed truth.
• DAG Sampling: Candidate DAGs are sampled from this prior, respecting temporal metadata (e.g., treatment precedes outcome) to ensure acyclicity.

2. Statistical Pruning and Bayesian Weighting

• Conditional Independence (CI) Pruning: Spurious edges in sampled graphs are pruned using Fisher's $z$-test for conditional independence on the training data ($D_{train}$).
• BIC Reweighting: The remaining candidate graphs are scored using the Bayesian Information Criterion (BIC) combined with the structural prior. Duplicate adjustment strategies are collapsed, and weights ($w_k$) are normalized to form a posterior distribution over unique adjustment sets.

3. Graph-Conditional Doubly Robust Estimation

For each candidate graph $G_k$ (and its implied adjustment set $Z_k$):

• Nuisance Estimation: Doubly robust nuisance models (propensity scores and outcome regressions) are fitted on $D_{train}$.
• Pseudo-Outcome Generation: Augmented Inverse Probability Weighting (AIPW) pseudo-outcomes ($\tilde{Y}^{(k)}$) and heuristic bounds are computed.

4. Structure-Weighted Conformal Calibration

This is the core innovation. Rather than calibrating individual graphs and averaging the results, CAUSALGUARD:

• Aggregates First: It computes a posterior-weighted pseudo-outcome ($\bar{\gamma}$) and aggregated bounds by combining the graph-conditional estimates using the BMA weights.
• Composite Score: A single nonconformity score is defined on this aggregated target: $E_i = \max(\bar{q}_{low} - \bar{\gamma}_i, \bar{\gamma}_i - \bar{q}_{high})$.
• Calibration: The conformal threshold $\hat{Q}$ is derived from the distribution of these composite scores on the calibration set ($D_{cal}$).
• Final Interval: The output interval is $\hat{C}(x) = [\bar{q}_{low}(x) - \hat{Q}, \bar{q}_{high}(x) + \hat{Q}]$.

Key Contributions

1. Structure-Weighted Conformal Inference: A novel framework that aggregates graph-conditional doubly robust pseudo-outcomes via a posterior-weighted composite score, transferring structural uncertainty directly into the calibration step.
2. Aggregate-Before-Calibrate Scoring: Theoretical proof (Proposition 1) that calibrating the aggregated target directly avoids the Jensen gap inherent in averaging graph-specific intervals, resulting in tighter intervals for the same coverage guarantee.
3. Graph-Ensemble Construction: A practical procedure combining LLM-derived edge proposals, CI-based pruning, and BIC reweighting. The LLM prior acts as a finite-sample proposal distribution, while the data likelihood (via BIC) dominates as sample size increases.
4. Theoretical Guarantees:
   • Finite-Sample Validity: Theorem 1 establishes distribution-free marginal coverage for the aggregated pseudo-outcome ($\bar{\gamma}$) under exchangeability, without assuming any candidate graph is correct.
   • Asymptotic Consistency: Corollary 2 demonstrates that under causal identification, overlap, nuisance stability, and score concentration on valid strategies, the conditional mean of the aggregated target converges to the true CATE.

Empirical Results

The method was evaluated across five benchmarks (IHDP, Sachs, ACIC2019, Jobs, Twins) with 10 random seeds per dataset.

• Coverage: CAUSALGUARD consistently achieved mean coverage above the nominal 90% level for the directly evaluable target (true CATE on IHDP/Twins; weighted pseudo-outcome on others).
• Efficiency: The method significantly reduced interval width compared to graph-agnostic conformal baselines (e.g., Conformal Causal Forest, Conformal Naive DR), particularly in datasets where adjustment-set uncertainty drives inefficiency (e.g., IHDP and Jobs).
• Robustness to Invalid Adjustment: Stress tests involving the injection of a collider variable showed that CAUSALGUARD successfully suppressed invalid collider adjustments (via temporal metadata and CI pruning), maintaining high coverage and low RMSE, whereas naive baselines suffered severe under-coverage.
• Prior Sensitivity: In low-data regimes, the LLM prior helps organize the search. However, as sample size increases, BIC weighting causes the prior to "wash out," making the final interval increasingly data-driven. The method remains stable even under adversarial or uninformative priors when data is informative.

Significance and Claims

The paper positions CAUSALGUARD as a solution to the tension between the need for valid causal adjustment and the uncertainty of the underlying graph structure.

• Shift in Paradigm: It moves away from "selecting one best graph" or "padding for worst-case uncertainty." Instead, it treats the adjustment set as uncertain and propagates this uncertainty through the interval construction.
• Practical Utility: The method is most effective when interval inflation is driven by uncertainty in adjustment choices rather than irreducible outcome noise. It provides a way to leverage semantic priors (LLMs) for finite-sample search while ensuring asymptotic validity through data-driven reweighting.
• Limitations: The authors note that the method assumes causal sufficiency and currently scales to moderate graph sizes due to the combinatorial nature of candidate generation. It guarantees coverage for the aggregated pseudo-outcome; convergence to the true CATE relies on additional assumptions regarding target alignment and nuisance stability.

In summary, CAUSALGUARD offers a distribution-free, finite-sample valid approach to treatment effect estimation under graph uncertainty, achieving tighter intervals than graph-agnostic methods by calibrating a structure-weighted target rather than averaging calibrated intervals.