Causal Machine Learning Is Not a Panacea: A Roadmap for Observational Causal Inference in Health

Problem
The increasing availability of large-scale, multi-modal observational health data has spurred interest in using causal machine learning (ML) to identify causal relationships, such as treatment effects, where randomized controlled trials (RCTs) are infeasible due to ethical or logistical constraints. However, the flexibility of ML models introduces significant risks: they are optimized for predictive performance and may leverage spurious associations rather than true causal mechanisms. Inappropriately drawing causal conclusions from observational data risks producing biased or misleading results, which can negatively impact medical science and patient safety. A critical gap exists in shared knowledge regarding the limitations of causal ML and the necessity of satisfying specific validity assumptions.

Methodology
The paper does not propose a new algorithm but rather presents a conceptual roadmap and framework for applying causal ML responsibly in observational settings. The methodology is structured around three core pillars:

1. Identification Challenges: The authors emphasize the necessity of defining a causal Directed Acyclic Graph (DAG) with domain experts to establish identifiability assumptions. They detail three critical assumptions and how ML can obscure their violations:

   • Overlap: Ensuring every patient has a non-zero probability of receiving the treatment. The paper notes that high discriminative performance in propensity score models may indicate low overlap, but poor performance could also signal underfitting.
   • Conditional Exchangeability (No Unobserved Confounding): Acknowledging that this is unverifiable in observational data. The authors advise using sensitivity analyses to quantify the magnitude of unobserved confounders required to alter conclusions.
   • SUTVA (Stable Unit Treatment Value Assumption): Ensuring consistency (no hidden treatment variations) and non-interference (one patient's treatment does not affect another's outcome). The paper warns that ML models may inadvertently use variables like room IDs or dose strengths as predictive features, mistaking interference or treatment heterogeneity for causal factors.

2. Modeling Choices: The authors discuss the selection of covariates and modeling approaches. They caution against including mediators (intermediate variables) or irrelevant variables, which can dilute treatment effect estimates. They focus on metalearners, which combine "base" models (e.g., random forests, neural networks) to estimate treatment effects. These are categorized into:

   • Indirect Estimators (e.g., S-learner, T-learner): Predict outcomes under each treatment separately and take the difference. These are simple but fragile, as errors in either prediction are magnified, and regularization may flatten treatment heterogeneity.
   • Direct Estimators (e.g., X-learner, R-learner, Double Machine Learning): Model the treatment effect explicitly in a second step using pseudo-outcomes or orthogonalization. These are generally more robust to base model errors than indirect methods.

3. Evaluation: The paper argues that standard ML evaluation (held-out test sets) is infeasible because the "true" treatment effect is unobserved. Instead, they propose evaluating causal hypotheses through:

   • Global Null Analysis: Using synthetic treatments with no true effect.
   • Negative Control Analysis: Testing treatments known to have no effect.
   • Stability Checks: Comparing results across different modeling approaches to ensure consistency in sign and magnitude.

Key Contributions

• A Roadmap for Rigor: The paper provides a structured template (Figure 3) to guide interdisciplinary teams through causal assumptions, modeling choices, and evaluation strategies.
• Clarification of ML Risks: It explicitly details how the flexibility of ML can obscure violations of causal assumptions (e.g., overfitting to spurious signals, misinterpreting interference) and offers mitigation strategies involving domain expertise.
• Distinction between Hypothesis and Finding: The authors reframe the output of causal ML on observational data not as definitive findings, but as causal hypotheses that must be prioritized for future testing via RCTs or biological validation.

Results
As a perspective and roadmap paper, the work does not present empirical experimental results or new datasets. Instead, its "results" are the synthesis of existing literature and the articulation of a practical framework. The paper demonstrates that without careful justification of assumptions and modeling choices, causal ML approaches risk producing biased estimates. It illustrates that direct estimators often offer greater robustness than indirect estimators and that domain expertise is non-negotiable for defining DAGs and interpreting model features.

Significance
The paper claims significance as a corrective to the enthusiasm surrounding causal ML in health. It argues that while causal ML is a powerful tool for generating hypotheses from rich, multi-modal data, it is not a panacea. The authors assert that their roadmap is essential for strengthening the rigor and interpretability of causal analyses, thereby preventing the dissemination of misleading results that could compromise clinical research and patient care. The work serves as a foundational guide for both clinical experts and ML practitioners to collaborate effectively, ensuring that causal assumptions are satisfied and modeling choices are justified before drawing conclusions.