Temporal Data Requirement for Predicting Unplanned Hospital Readmissions

1. Problem Statement

The proliferation of Electronic Health Records (EHRs) has enabled the development of data-driven predictive models for healthcare outcomes, such as 30-day unplanned hospital readmissions. However, a critical gap exists in determining the optimal temporal observation window for historical data.

• The Assumption: There is a prevailing belief in machine learning that "more data is better," leading researchers to use long observation windows (often spanning years) to train models.
• The Challenge: It is unclear whether extending the historical window indefinitely improves predictive accuracy, or if there is a point of diminishing returns. Furthermore, it is unknown if the optimal window length differs between structured data (tabular, coded) and unstructured data (clinical notes).
• The Goal: To identify the specific time-window length that maximizes predictive performance for 30-day readmissions following hip and knee arthroplasties, comparing structured vs. unstructured data sources.

2. Methodology

Dataset

• Source: Retrospective data from Partners Healthcare (Epic EHR system).
• Cohort: 7,174 adult patients (aged 18–90) undergoing hip or knee arthroplasty between 2006 and 2016.
• Data Volume: Over 4 million encounter records and 80,000 clinical notes.
• Target Variable: Unplanned readmission within 30 days post-surgery (Positive Class = 1, Negative Class = 0).
• Class Imbalance: The dataset is highly imbalanced (approx. 9.5% for hip, 10.5% for knee readmissions). Strategies like class weighting, undersampling, and oversampling were employed.
• Split: Data was partitioned at the patient level: 70% Training, 15% Validation, 15% Testing.

Observation Windows

The study evaluated observation windows ranging from 0 months (surgery day only) to 36 months (3 years) prior to surgery.

Feature Engineering & Encoding

The study utilized three data modalities:

1. Structured Data: Categorical and numerical features across 8 domains (Demographics, Labs, Diagnosis, Medication, Health History, Procedures, Vitals, Admission Info).
   • Encoding: One-hot encoding, text representation of categorical variables, and handling of time-sensitive vs. non-time-sensitive data.
2. Unstructured Data (Clinical Notes): Free-text notes from physicians, nurses, and scripts.
   • Preprocessing: Tokenization, lowercasing, and stemming.
   • Non-Neural Encoders: Bag of Words (Binary, Count), TF-IDF, and Latent Dirichlet Allocation (LDA).
   • Neural Encoders:
     • Average: Linear projection + ReLU.
     • 1D-CNN: Convolutional layers with kernel sizes (3, 5, 7, 9).
     • BiLSTM: Bidirectional Long Short-Term Memory networks.
     • Attention Mechanisms: To weight hidden states based on relevance.
     • Hierarchical Models: BiLSTM for note-level, then BiLSTM for patient-level aggregation.
     • BERT: Clinical BERT (pre-trained on MIMIC-III) for contextualized word representations.

Model Architecture

• Independent Models: Separate models trained for Hip and Knee surgeries.
• Multitask Learning: Joint models predicting readmission for both surgery types, sharing encoder parameters to "borrow strength" across tasks.
• Combinations: Models were trained on (a) Clinical Notes only, (b) Structured Data only, and (c) A combination of both.
• Evaluation Metric: Area Under the Receiver Operating Characteristic Curve (AUROC).

3. Key Contributions

1. Temporal Window Optimization: The study empirically demonstrates that the optimal historical data window is not uniform across data types.
2. Unstructured vs. Structured Divergence: It reveals a fundamental difference in how temporal data impacts model performance:
   • Structured Data: Performance improves as the window extends, plateauing at 12 months.
   • Unstructured Data (Clinical Notes): Performance peaks at a shorter window (3–6 months) and does not necessarily improve (and may degrade) with older data.
3. Model Agnosticism: These temporal patterns hold true regardless of the model complexity, applying to both simple non-neural encoders (Logistic Regression + BoW) and complex neural architectures (BERT, BiLSTM).
4. Data Efficiency: The findings suggest that for unstructured data, storing and processing 3 years of history is inefficient; a 3–6 month window yields superior results, reducing computational and storage costs.

4. Key Results

• Performance Trends:

  • Clinical Notes Only: The highest AUROC was achieved using data from 3 to 6 months prior to surgery. Performance dropped when using data older than 6 months or data from the surgery day alone.
  • Structured Data Only: AUROC increased steadily as the window grew from 0 to 12 months, then plateaued. No significant gain was observed beyond 12 months.
  • Combined Data: The combined model followed the trend of the clinical notes, with the optimal window being 3–6 months. This indicates that the high predictive power of recent clinical notes dominates the contribution of older structured data.

• Quantitative Findings (AUROC):

  • For Knee Surgery using Clinical Notes + Structured Data, the model using 3 months of historical data achieved an AUROC of 0.930 (Multitask), significantly outperforming models using only surgery-day data (0.881) or 36 months of data.
  • Hip Surgery showed similar trends, with the 3-month window yielding the best results (e.g., 0.888 AUROC for combined data).

• Multitask Learning: While multitask learning showed slight performance improvements in some combined scenarios, the temporal data requirements remained consistent with independent models. The optimal window length was independent of the learning strategy (independent vs. multitask).

5. Significance and Implications

• Challenge to "More Data is Better": The study refutes the general heuristic that longer historical windows always improve ML performance in healthcare. For unstructured clinical notes, recency is more valuable than volume.
• Operational Efficiency: Hospitals and data engineers can optimize EHR storage and retrieval systems by focusing on the 3–6 month window for unstructured data, rather than ingesting years of historical notes, thereby reducing latency and storage costs.
• Clinical Relevance: The findings suggest that the clinical narrative surrounding a patient's recent pre-operative status (3–6 months) is the strongest predictor of immediate post-surgical complications, whereas older structured history (e.g., demographics, past diagnoses >1 year ago) provides diminishing returns.
• Future Directions: The authors suggest validating these findings on other surgical types (e.g., cardiovascular) and across different healthcare systems to ensure generalizability beyond the Epic EHR system used in this study.

Conclusion

This paper provides a critical guideline for building predictive models in healthcare. It establishes that optimal temporal data requirements are data-type specific: structured data benefits from a longer history (up to 12 months), while unstructured clinical notes are most predictive within a short, recent window (3–6 months). Ignoring these nuances can lead to suboptimal model performance and inefficient resource utilization.