Prediction of delayed inpatient antidepressant initiation after the first 24 hours of hospital admission using early electronic health record data: a MIMIC-IV v3.1 landmark prediction study

Technical Summary: Prediction of Delayed Inpatient Antidepressant Initiation

Problem Statement
The study addresses the challenge of modeling "workflow-sensitive" operational outcomes within Electronic Health Records (EHRs). Specifically, it targets delayed inpatient antidepressant initiation, defined as the first order or administration of an antidepressant more than 24 hours after hospital admission but before discharge. This outcome is distinct from a direct biological measurement; it is a clinician-mediated prescribing event influenced by patient characteristics, medication reconciliation, off-label indications, and institutional workflows. The core technical problem is to determine whether structured EHR data available strictly within the first 24 hours of admission can predict this downstream event without introducing temporal data leakage, while accounting for the fact that EHR data often contains traces of care processes (e.g., order volume, monitoring intensity) rather than just patient pathology.

Methodology

• Data Source & Cohort: The study utilized MIMIC-IV version 3.1 (Beth Israel Deaconess Medical Center, 2008–2019) as a retrospective cohort study. The unit of analysis was the hospital admission.
  • Inclusion: Adult admissions (≥18 years) where the patient remained alive and hospitalized through the 24-hour landmark.
  • Exclusion: Admissions with antidepressant exposure before or within the first 24 hours, and admissions with documented prior antidepressant exposure within MIMIC-IV prior to the index admission.
  • Final Cohort: 253,529 admissions (19,810 events, 233,719 controls; prevalence ~7.8% in the full cohort, 9.9% in the test set).
• Landmark Design: A strict 24-hour landmark was enforced. Predictors were restricted to structured data available between admission and 24 hours post-admission, or data from prior admissions. This design prevents post-landmark information from inflating performance and distinguishes patients already treated from those at risk.
• Predictors: Features included demographics, admission context, prior psychiatric history (from prior admissions only), early laboratory results, ICU vital signs, early non-antidepressant medication exposures, and workflow/care-intensity measures (e.g., order counts, order diversity, CPOE activity). Diagnosis codes were excluded to avoid coding heterogeneity and leakage.
• Modeling Approach:
  • Algorithms: Prevalence-only baseline, regularized logistic regression, and boosted-tree classifiers.
  • Validation: Subject-disjoint temporal split (earlier years for development, later years for testing) to ensure no patient overlap across partitions.
  • Ablation & Sensitivity: Extensive analyses included removing workflow features, excluding target-proximal medications (psychiatric/sedatives), restricting to non-ICU patients, and testing narrower pharmacologic definitions (e.g., SSRI/SNRI only).
  • Metrics: Emphasis on AUPRC (Area Under the Precision-Recall Curve) and calibration metrics, acknowledging that AUROC alone can overstate utility for low-prevalence outcomes.

Key Results

• Model Performance:
  • The boosted-tree model outperformed regularized logistic regression, achieving an AUROC of 0.701 (95% CI 0.694–0.711) and AUPRC of 0.199 (95% CI 0.192–0.208) on the held-out test set.
  • The regularized logistic regression achieved an AUROC of 0.663 and AUPRC of 0.172.
  • The difference in AUPRC between the two models was 0.027.
• Feature Contribution:
  • Workflow Features: Removing workflow and care-intensity features reduced the boosted-tree AUPRC from 0.199 to 0.190, indicating these variables provide measurable, incremental predictive signal.
  • Domain Decomposition: The "Clinical-only" model (demographics, vitals, labs) achieved an AUPRC of 0.190, while the "Workflow-only" model achieved 0.165. The combined model (0.199) confirmed that no single domain fully accounts for the prediction, but the combination yields the best performance.
• Calibration & Utility:
  • The model showed reasonable calibration (Brier score 0.085) but was not perfect (slope 1.111).
  • At the top 1% risk threshold, the model flagged ~10 admissions per 1,000 with a Positive Predictive Value (PPV) of 30.1% (enrichment ~3x over baseline).
  • Sensitivity analyses confirmed robustness: performance persisted even when excluding target-proximal medications or restricting to non-ICU patients.

Significance and Claims
The paper positions this work as a methodological benchmark for clinical informatics rather than a deployable clinical tool.

• Operational Modeling: It demonstrates that early structured EHR data can meaningfully enrich risk for downstream clinician-mediated events, provided strict temporal constraints and leakage controls are applied.
• Workflow Signals: The study validates that "workflow" variables (order intensity, monitoring) encode operational concern and care complexity, serving as legitimate predictive features for operational outcomes, provided they are not interpreted causally.
• Research Tool: The authors explicitly state that the internally validated model remains a research tool. It is suitable for enriching cohorts for retrospective chart review, studying care-process variation, and benchmarking operational prediction methods.
• Limitations: The paper acknowledges that the outcome is an operational proxy, not psychiatric ground truth, and that the model relies on single-center data with internal temporal validation only. It emphasizes that independent-site validation and local recalibration are prerequisites for any clinical implementation.

In summary, the study successfully constructs a rigorous landmark prediction model for a complex, workflow-sensitive outcome, proving that early EHR signals can capture the operational dynamics of delayed antidepressant initiation while adhering to strict temporal and methodological standards.