From Data Lifting to Continuous Risk Estimation: A Process-Aware Pipeline for Predictive Monitoring of Clinical Pathways

Technical Summary: From Data Lifting to Continuous Risk Estimation

Problem Statement
Digital health systems face significant challenges in transforming heterogeneous clinical data (e.g., electronic health records, vital signs, laboratory results) into reliable decision-support indicators. Traditional process mining in healthcare is predominantly retrospective, analyzing completed patient trajectories to discover models or identify bottlenecks. However, modern healthcare scenarios require continuous and prospective monitoring, where decisions must be supported under partial and evolving information. Existing predictive monitoring approaches often focus on isolated predictive models, neglecting the critical integration of data transformation, process-aware representations, and reproducible experimental pipelines. Furthermore, the "data lifting" required to convert fragmented, temporally inconsistent clinical records into usable event logs is frequently treated as a preliminary step rather than a core component of the predictive architecture.

Methodology
The authors propose a reproducible, process-aware pipeline designed to transform raw clinical data into continuous risk estimates for partially observed patient trajectories. The framework is organized into four modular stages:

1. Data Integration and Temporal Validation:

   • Heterogeneous data sources (patient records, emergency data, diagnoses, medications, vital signs, lab tests) are aligned at the patient level using unique identifiers.
   • A deterministic "data lifting" procedure cleans data (handling missing values, standardizing zero/empty strings) and performs temporal reconstruction.
   • Rule-based transformations and an inference table (aggregating candidate timestamps from multiple sources) resolve inconsistencies, such as triage timestamps preceding admission times, ensuring a temporally consistent event sequence.

2. Event Log Construction and Enrichment:

   • Validated data are transformed into a standard event log (XES format) using modular event collectors.
   • Clinical activities are mapped to events with specific attributes (case ID, activity name, timestamp, and context-specific attributes like vital signs or diagnostic codes).
   • Trace-level attributes (e.g., demographics) are propagated to all events within a case.

3. Prefix-Based Process Representation and Feature Extraction:

   • To enable predictive monitoring, complete traces are decomposed into prefixes (partial sequences of events).
   • A feature engineering module extracts prefix-level vectors, including:
     • Temporal features: Elapsed time, time since admission, inter-event intervals.
     • Activity features: Counts, frequencies, and the last observed activity.
     • Transition features: Patterns of consecutive activities.
     • Clinical features: Aggregated statistics (min, max, average, latest) of time-varying signals.
     • Case attributes: Static demographic information.

4. Predictive Monitoring:

   • The framework employs standard machine learning models (Logistic Regression and Random Forest) trained on prefix-level feature vectors.
   • The prediction target (ICU admission) is defined at the case level and propagated to all corresponding prefixes, enabling supervised learning on partially observed trajectories.
   • The system simulates an online monitoring scenario where risk estimates are updated incrementally as new events arrive.

Case Study and Experimental Setup
The pipeline was evaluated using the COVID Data for Shared Learning (CDSL) dataset, comprising 4,479 patient cases and 46,804 prefixes.

• Task: Predicting the likelihood of ICU admission based on evolving patient trajectories.
• Split: A case-level split was used to prevent information leakage, with 896 patients (and their derived prefixes) reserved for the test set.
• Evaluation: Models were assessed using AUC, accuracy, precision, recall, and F1-score, with a specific focus on performance evolution across different prefix lengths.

Key Results

• Overall Performance: Logistic Regression achieved the best performance with an AUC of 0.906 and an F1-score of 0.835, outperforming the Random Forest model. This suggests that the prefix-based feature representation is sufficiently informative for accurate prediction using relatively simple, interpretable models.
• Progressive Signal Emergence: Predictive performance improves significantly as the clinical pathway evolves and more events become available:
  • At early stages (prefix length 1), AUC is 0.642.
  • At prefix length 20, AUC rises to 0.772.
  • At later stages (prefix length 50), AUC reaches 0.942.
• Reproducibility: The pipeline generates explicit intermediate artifacts (event logs, prefix datasets, feature tables, prediction outputs), ensuring the entire workflow is traceable and reproducible.

Significance and Claims
The paper argues for a paradigm shift from static, model-centric prediction to continuous, pipeline-centric monitoring. The authors claim that:

1. Predictive signals emerge progressively: Clinically relevant information is not concentrated at admission but is encoded in the sequence of events over time. Consequently, risk estimation must be a continuous process that updates as the patient journey evolves.
2. Data transformation is critical: The quality of temporal reconstruction and event log construction is a decisive factor in predictive performance. Data preprocessing (specifically temporal alignment) should be viewed as a core component of the predictive architecture, not merely a preliminary step.
3. Process-aware representations are essential: Modeling patient trajectories as evolving sequences provides predictive value beyond static snapshots. The ability to reason on running cases using prefix-based representations enables effective early risk estimation.
4. Reproducibility and Traceability: By decomposing the pipeline into modular, artifact-generating stages, the framework supports fine-grained inspection, validation, and debugging, which are crucial for clinical applications.

The study concludes that while advanced learning algorithms are useful, the integration of robust data lifting, process-aware representations, and reproducible pipelines is the primary enabler of reliable predictive monitoring in healthcare.