A Closed-Loop Framework for Knowledge-Based Causal Model Refinement Using Process Mining and Human-AI Governance

Technical Summary: A Closed-Loop Framework for Knowledge-Based Causal Model Refinement

Problem Statement
In epidemiological causal inference, Directed Acyclic Graphs (DAGs) are essential for encoding assumptions about causal relationships and guiding statistical analysis. However, traditional knowledge-based DAGs rely heavily on expert opinion and literature, often lacking empirical validation. This leads to uncertainty in causal structures, particularly in complex chronic diseases characterized by competing risks, temporal dynamics, and multiple interacting conditions. Such uncertainty can bias effect estimation regardless of the robustness of the statistical method used. Conversely, while real-world data (RWD) offers vast longitudinal evidence, applying automated causal discovery algorithms (e.g., PC, FCI) often fails due to unmet assumptions (faithfulness, causal sufficiency) and a lack of clinical interpretability. Furthermore, recent attempts to use Large Language Models (LLMs) for causal reasoning raise concerns regarding hallucinations and the inability to distinguish between associational and causal claims. There is a need for a framework that integrates empirical data patterns with expert knowledge without replacing human authority over causal interpretation.

Methodology
The authors propose a closed-loop, governed framework that treats knowledge-based DAGs as testable scientific hypotheses, iteratively refined through a human-AI collaboration loop based on process mining. The methodology consists of four distinct phases:

1. Phase 1: Knowledge-Driven Event Log Construction:

   • An initial knowledge-based DAG is generated using LLMs to synthesize literature and expert input. This serves as a syntactic hypothesis, not a causal truth.
   • The DAG guides the construction of a structured event log (patient-level, longitudinally ordered sequences of timestamped clinical events), defining populations, events, and covariates.
   • Governance: Human experts define clinically meaningful events and inclusion criteria; AI assists in literature synthesis.

2. Phase 2: Data-Driven Discovery:

   • Process mining algorithms (specifically Directly-Follows Graphs or DFGs) are applied to the event log to discover actual temporal transition patterns.
   • The output is a DFG representing observed transitions (e.g., CKD diagnosis $\to$ Cardiovascular event) annotated with frequency and timing.
   • Governance: The DFG serves as an empirical baseline, capturing observed temporal succession rather than inherent causality.

3. Phase 3: DAG Refinement (The Human-AI Loop):

   • The empirical DFG is systematically compared against the initial knowledge-based DAG.
   • Discrepancies (e.g., missing edges, unexpected reverse transitions, or high-frequency competing risks) trigger candidate refinements proposed by the AI.
   • Governance: A human expert adjudicates each proposal. The AI proposes changes (adding/removing edges/nodes, restructuring for competing risks), but the human decides based on biological plausibility and clinical insight. This phase explicitly rejects observation-induced artifacts (e.g., post-event coding shifts or surveillance bias) that do not reflect biological mechanisms.

4. Phase 4: Statistical Model Selection:

   • The refined, evidence-based DAG is mapped to appropriate statistical models using explicit decision rules (e.g., Fine-Gray models for competing risks, causal mediation analysis for mediators).
   • The framework recommends specific analytical strategies (e.g., propensity-score balancing, sensitivity analyses) aligned with the refined structure.
   • Governance: Human experts review model assumptions and interpret effect sizes; the AI provides structured recommendations.

The loop closes by feeding the results and the refined DAG back into the knowledge base for future iterations, ensuring transparency and traceability.

Key Contributions

• Governed Human-AI Collaboration: The paper introduces a framework where AI acts as a "checkpoint augmentation" tool rather than an autonomous decision-maker. It explicitly separates exploratory refinement (AI-driven pattern discovery) from confirmatory estimation (human-adjudicated causal interpretation).
• Process Mining as Diagnostic Appraisal: Unlike traditional causal discovery, this approach uses process mining not to prove causality but to diagnose and challenge pre-specified causal assumptions against empirical temporal patterns.
• Artifact Rejection Mechanism: The framework includes specific "plausibility gates" (e.g., directionality rules, activity-type rules) to reject artifacts such as reverse coding shifts or administrative feedback loops that mimic causal pathways in raw data.
• Open-Source Implementation: The authors developed an interactive web tool (HealthProcessAI) and a simulation environment to demonstrate the workflow.

Results
The framework was evaluated using two approaches:

1. Tutorial Demonstration (Synthetic Data): Using data calibrated to the SCREAM cohort (comparing Proton Pump Inhibitors vs. Histamine-2 Receptor Blockers), the framework successfully:

   • Identified Death as a critical competing-risk node, which was missing in the initial literature-based DAG.
   • Validated CKD progression as a candidate mediator on the PPI-to-Cardiovascular Adverse Event pathway, while rejecting a spurious reverse transition (CVAE $\to$ CKD) that did not differ significantly between groups.
   • Recommended appropriate statistical models (Fine-Gray, mediation analysis) consistent with epidemiological best practices.

2. Ground Truth Simulation Study: A controlled simulation with known causal structures and injected artifacts (a "drug-feedback decoy" and a "reverse coding shift") was tested across 500 replications.

   • Ungoverned Arm (AI-only): Accepted 100% of injected artifacts, degrading precision.
   • Governed Arm (Human-AI Loop): Rejected 100% of injected artifacts while recovering all high-support true transitions and the competing-risk node.
   • Estimand Recovery: The model selected by the governed framework recovered the true cause-specific hazard ratio with negligible bias and near-nominal confidence interval coverage, whereas unadjusted or adjusted models failed due to surveillance bias.

Significance and Claims
The paper claims to advance methodological discipline in real-world evidence generation by providing a transparent, reproducible, and governed approach for integrating AI into causal inference.

• It addresses the gap between theoretical causal epidemiology and empirical discovery by treating DAGs as falsifiable hypotheses.
• It preserves human authority over causal interpretation, ensuring that AI serves as a diagnostic tool for structural refinement rather than a replacement for domain expertise.
• The framework is modest in its scope: it does not claim to discover causality autonomously or eliminate unmeasured confounding. Instead, it offers a systematic mechanism to identify and reject observation-induced artifacts, thereby improving the robustness of the causal models upon which statistical estimation is based.
• By embedding empirical appraisal into DAG construction, the framework aims to bridge the divide between conceptual models and observational data, aligning with the iterative scientific philosophy of John Snow.