Counterfactual Analysis of Executable Clinical Decision Logic

Technical Summary: Counterfactual Analysis of Executable Clinical Decision Logic

Problem Statement
Clinical practice guidelines are frequently expressed in narrative formats, which limits their direct execution, auditability, and ability to provide patient-specific interpretations. While Decision Model and Notation (DMN) offers a structured, human-readable, and executable representation of decision logic, standard DMN tables evaluate current input states without automatically demonstrating how outputs would change under feasible modifications. There is a need for a framework that bridges data-driven pattern discovery with transparent, auditable decision logic capable of generating "what-if" explanations without relying on opaque prediction processes.

Methodology
The authors propose a hybrid decision-support framework validated on an NHANES-derived fasting cohort (2,582 participants) to classify documented diabetes status. The workflow is strictly separated into training, validation, and untouched test sets to prevent data leakage.

1. Data Preprocessing and Feature Screening:

   • The dataset was constructed from NHANES Cycle J, merging demographic, examination, laboratory, and questionnaire modules.
   • A survey-weighted L2-regularized logistic model was used for feature screening. HbA1c was identified as the dominant predictor, followed by total cholesterol and triglycerides.
   • The cohort was split 60/20/20, stratified by outcome and "Gate0" membership (a non-diagnostic subgroup defined by HbA1c < 6.5% and FPG < 126 mg/dL).

2. Rule-Ensemble Learning:

   • A RuleFit-style ensemble was trained on the training set using 400 shallow gradient-boosted trees. Terminal-node paths were extracted as binary rules and combined with linear terms in an L1-regularized weighted logistic model.
   • This "full model" achieved high predictive performance (ROC–AUC 0.959 in the full fasting cohort) but remained a complex, non-executable black box.

3. Clinical Candidate Rule Selection:

   • A library of 172 clinically constrained rules (involving three conditions each, including at least one glycemic threshold) was evaluated exclusively on the validation set.
   • Four candidate rules were selected based on survey-weighted support (≥2%), coverage, lift (>1), and confidence. These rules combined variables such as HbA1c, FPG, BMI, Age, and HDL.

4. DMN Construction and Weighting:

   • The selected rules were mapped to an executable DMN decision table using a "collect/sum" hit policy.
   • A nonnegative survey-weighted logistic model was fitted to the combined development cohort to assign weights to the rule activations. This ensured that activating a risk rule could only increase the score.
   • One candidate (C3) was removed due to a zero independent coefficient after accounting for overlap, resulting in a final three-rule DMN.

5. Counterfactual Sensitivity Analysis:

   • Instead of estimating causal effects, the framework performs policy sensitivity analysis. Modifiable inputs (specifically BMI) were perturbed, and the DMN was re-executed.
   • The analysis quantifies how the model-estimated probability changes only when a modified input crosses an explicit threshold used by an active rule.

Key Results

• Predictive Performance: The initial rule-ensemble model achieved an ROC–AUC of 0.959 and PR–AUC of 0.873 on the full fasting test set. In the more challenging Gate0 subgroup, it achieved 0.861 ROC–AUC and 0.499 PR–AUC.
• DMN Performance: The simplified, auditable three-rule DMN achieved an ROC–AUC of 0.769 and PR–AUC of 0.153 on the untouched Gate0 test set. While less discriminative than the full ensemble, it provided a compact policy representation.
• Counterfactual Findings: In small rule-defined subgroups, hypothetical five-unit BMI reductions lowered the mean model-estimated probability by 2.40 to 5.89 percentage points, but only when the reduction crossed specific BMI thresholds (e.g., from ≥30 to <30 kg/m²). The response was stepwise, reflecting the binary nature of the DMN rules.
• Rule Characteristics: The final DMN consisted of three rules (R1, R2, R3) involving combinations of HbA1c, FPG, BMI, Age, and HDL. For example, Rule R1 triggers for HbA1c [6.0, 6.5), FPG [100, 126), and BMI ≥30.

Significance and Claims
The paper claims to present a validation-separated framework that successfully integrates survey-weighted rule-ensemble learning with executable DMN logic and counterfactual sensitivity analysis.

• Transparency and Auditability: The framework converts complex data patterns into a finite set of inspectable rules with additive, nonnegative contributions. This allows clinicians to trace output changes directly to specific rule activations or deactivations rather than opaque model weights.
• Policy Sensitivity vs. Causal Inference: The authors explicitly state that the counterfactual analysis characterizes policy sensitivity rather than causal treatment effects. The findings demonstrate how a deployed decision table behaves under hypothetical changes, identifying the specific thresholds responsible for output shifts.
• Early-Alert Mechanism: The derived rules are not proposed as new diagnostic criteria or clinical guidelines. Instead, the framework is positioned as a potential "early-alert mechanism" for individuals who remain below diagnostic glycemic thresholds but possess risk factor combinations associated with a higher estimated probability of documented diabetes status. This could support closer monitoring or earlier clinical review.
• Limitations: The authors acknowledge that the study relies on a single NHANES cycle with a limited number of positive cases, particularly in the Gate0 test set, leading to wide confidence intervals. The rules describe associations with documented status, not future incidence, and require external validation and clinical expert review before any clinical deployment.