A Step Towards Inherently Interpretable Causal Machine Learning Models For Decision Support

Technical Summary: A Step Towards Inherently Interpretable Causal Machine Learning Models for Decision Support

Problem Statement
The increasing reliance on machine learning (ML) for organizational decision-making has highlighted a critical gap: while models are effective at prediction, they often lack transparency and causal understanding. Current approaches fall into two categories: post-hoc explainability methods applied to complex "black-box" models, and inherently interpretable machine learning (IIML) models. However, both paradigms primarily reveal how models exploit correlations to maximize predictive performance, which does not equate to causal insight. Consequently, standard ML models struggle with "what-if" scenario evaluations (interventions) because they cannot distinguish between spurious correlations and genuine causal relationships. This limitation hinders their utility for decision support tasks that require understanding the effects of deliberate changes in a system.

Methodology
To address these limitations, the authors propose a framework that integrates Structural Causal Models (SCMs) with Inherently Interpretable Machine Learning (IIML) models for cross-sectional data. The framework is organized into four components:

1. Input Data and Causal Graph: The process begins with a dataset and a causal graph representing causal relationships among variables. This graph can be derived from domain expertise, causal discovery algorithms, or a combination of both. The graph serves as the backbone for causal reasoning, explicitly encoding assumptions about the data-generating process.
2. Modelling Approach: Unlike standard ML, which typically uses all available features to predict a target, the proposed approach models each variable in the causal graph as a function of its direct parents (and an exogenous noise term). This is implemented using IIML techniques, specifically Generalized Additive Models (GAMs) and Symbolic Regression (SR), alongside traditional black-box models (XGBoost, Random Forest, MLP) for comparison.
3. Task Execution:
   • Prediction: For pure prediction tasks, the model performs causal feature selection, predicting the target based only on observed values of its causal parents to avoid error accumulation.
   • What-if Analysis: For intervention scenarios (e.g., $do(A=x)$), the framework propagates the intervention through the causal graph. The effect of changing a variable is calculated by blocking incoming paths to that variable and updating the predictions of its children, thereby respecting the system's causal structure.
4. Interpretation: The framework distinguishes between four levels of interpretability: (1) association post-hoc (e.g., SHAP on black-box models), (2) association inherent (direct analysis of IIML without causal structure), (3) causal post-hoc, and (4) causal inherent interpretation. The highest level provides transparency regarding the system structure, causal relationships, and the functional forms connecting variables.

Key Contributions

• Integration Framework: The paper proposes a novel framework combining SCMs with IIML (GAMs and SR) to support decision-making that requires both prediction and causal analysis.
• Research Questions: It addresses how causal inference can be combined with IIML (RQ1) and how these models perform in prediction versus what-if analysis (RQ2).
• Empirical Evaluation: The study evaluates the proposed approach against non-causal black-box models using two experiments:
  1. A synthetic production dataset with a known data-generating process to demonstrate interpretability levels and the impact of spurious correlations.
  2. A benchmark evaluation using four datasets from Geffner et al. (2022) to assess performance across different causal structures.

Results

• Prediction Performance: In both experiments, the causal IIML models achieved competitive predictive performance on standard test sets, often matching or slightly outperforming black-box counterparts. The results suggest that for tabular data, high accuracy does not require sacrificing interpretability.
• What-if Analysis: The causal models significantly outperformed non-causal ML models in intervention scenarios. In the synthetic experiment, non-causal models failed to handle the intervention correctly because they relied on spurious correlations (e.g., attributing importance to "personnel" which was correlated with the target but not causally linked). In contrast, causal models correctly propagated the intervention through the graph, yielding robust results.
• Interpretability: The causal IIML approach provided superior transparency.
  • Symbolic Regression: Successfully recovered mathematical expressions closely matching the true data-generating process when restricted to causal parents.
  • GAMs: Produced simpler, more accurate partial dependence plots that reflected the true functional relationships, avoiding the complex, misleading shapes found in non-causal models that tried to fit spurious correlations.

Significance and Claims
The paper claims that integrating causal structure with inherently interpretable models offers a pathway to "informed, transparent, and causally grounded decisions." The authors argue that this approach:

• Enables What-if Analysis: It allows models to move beyond the first rung of the "Ladder of Causation" (association) to the second rung (intervention), which is essential for business decision-making.
• Enhances Transparency: By explicitly modeling causal relationships and functional forms, the approach avoids the "black box" nature of complex models and the potential misleading nature of post-hoc explanations on correlational data.
• Bridges Fields: It contributes to Causal Machine Learning (CML) by incorporating IIML to clarify functional forms, and to Interpretability research by integrating causal knowledge into variable selection and model structure.

Limitations and Future Directions
The authors modestly note several limitations:

• The study relies on ground-truth causal graphs; it does not empirically test the robustness of the framework when graphs are misspecified or inferred incorrectly.
• The evaluation focuses on the first two rungs of the Ladder of Causation (seeing and doing), not counterfactual reasoning (the third rung).
• The interpretability was demonstrated conceptually; no qualitative user study was conducted to assess how stakeholders actually perceive or trust these models.
• The framework is currently limited to cross-sectional data, with time-series applications left for future research.
• Training multiple models (one per node) can be computationally expensive, particularly for SR.