Improving Clinical Applicability of Heart Failure Readmission Prediction via Automated Feature Engineering

This study demonstrates that automated feature engineering via Deep Feature Synthesis significantly enhances the discrimination, calibration, and clinical utility of gradient-boosted tree models for predicting heart failure readmissions from longitudinal electronic health records, outperforming traditional clinician-curated features.

The Gist

Imagine a hospital is like a busy airport, and patients with Heart Failure are travelers who often get stuck in a loop: they fly in, get treated, fly home, and then immediately have to fly back in again because they aren't fully recovered. This "readmission" is expensive, stressful, and dangerous.

Hospitals want a crystal ball to predict which travelers are likely to get stuck in this loop so they can give them extra help before they leave. For years, doctors have tried to build these crystal balls using manual checklists. They pick a few obvious clues (like age, blood pressure, or how many times they've been hospitalized before) and feed them into a computer.

But the computer models built on these manual checklists have been a bit like a weatherman using only a thermometer to predict a hurricane: they miss the big picture and aren't very accurate.

The New Idea: The "Feature Chef"

This paper introduces a new tool called Deep Feature Synthesis (DFS). Think of DFS not as a chef who picks the ingredients, but as a super-automated sous-chef who takes the raw ingredients (the patient's massive medical history) and chops, blends, mixes, and cooks them into thousands of new, complex recipes.

Instead of just looking at "Blood Pressure," this automated chef might create a new ingredient called: "The average drop in blood pressure during the last three visits, weighted by how many days it's been since the last visit." It finds hidden patterns and connections that a human doctor would never think to write down on a checklist.

The Experiment: Who Wins the Cooking Contest?

The researchers tested this "Super Sous-Chef" (DFS) against the "Human Chef" (the manual checklist) using data from over 350,000 heart failure patients. They tried to predict who would return to the hospital in 30, 60, or 90 days.

But here is the twist: The result depended entirely on the type of "Taste Tester" (the computer model) they used.

1. The Flexible Taster (Gradient-Boosted Trees)

Imagine a taste tester who is a master chef. They can handle complex, spicy, and weird flavor combinations.

• The Result: When this flexible taster tried the "DFS ingredients," the food was delicious. The predictions became much more accurate. The model got better at spotting the right patients and, crucially, became better at knowing how sure it was (calibration).
• The Benefit: It reduced the number of "false alarms." Before, the model might have flagged 100 people as "high risk," but only 20 were actually going to return. With DFS, it flagged 100 people, and 25 were actually going to return. This saves doctors from wasting time on patients who are fine.

2. The Rigid Taster (Logistic Regression)

Now, imagine a taste tester who only likes simple, plain food. They get confused if you mix too many spices or add complex textures.

• The Result: When this rigid taster tried the "DFS ingredients," the food tasted worse. The predictions got slightly less accurate. The sheer volume of complex new ingredients confused the simple model, making it stumble.

The Big Takeaway

The main lesson of this paper is: Just because you have a better tool (DFS) doesn't mean it works for everyone.

• If you use a smart, flexible computer model (like LightGBM or XGBoost): Let the automated chef do its thing. It will find hidden patterns, make the predictions sharper, and save the hospital from alert fatigue (too many false alarms).
• If you use a simple, linear model: Stick to the manual checklist. The fancy automated ingredients will just make things messy and confusing.

Why This Matters for Real Life

Hospitals are currently drowning in data but starving for insights. This study shows that we don't necessarily need to build massive, unexplainable "black box" AI systems to get better results. Instead, we can use automated feature engineering to feed our existing, trusted models better data.

It's like upgrading a car engine. If you put a high-performance fuel (DFS) into a race car (a tree-based model), it flies. If you put that same fuel into a bicycle (a simple linear model), the engine might just sputter and break.

In short: Automated tools can make heart failure predictions much better, but only if you pair them with the right kind of computer brain. When done right, it means fewer patients get readmitted, and doctors spend less time chasing false alarms.

Technical Summary

1. Problem Statement

Heart failure (HF) readmission is a critical clinical and economic burden, with 20–25% of patients returning within 30 days of discharge. Existing prediction models often suffer from limited discrimination and poor calibration. A primary cause is their reliance on manually curated, cross-sectional features that fail to capture the rich temporal dynamics (e.g., vital sign trajectories, prior utilization patterns) embedded in longitudinal Electronic Health Record (EHR) data. While deep learning approaches can exploit this data, they often require massive datasets and lack interpretability. The authors investigate whether automated feature engineering via Deep Feature Synthesis (DFS) can bridge this gap, improving clinical utility without sacrificing model interpretability or requiring end-to-end deep learning architectures.

2. Methodology

Study Design and Data

• Cohort: A retrospective study using data from a large U.S. urban safety-net health system (2010–2025).
• Sample Size: 355,217 eligible HF-related index hospitalizations.
• Outcomes: Prediction of 30-, 60-, and 90-day readmission.
• Data Handling: Strict protocols were used to prevent temporal leakage; features were generated using data available only up to the point of discharge. Missing data was handled via a hybrid strategy (MICEforest for key variables, median/mode imputation for others).

Feature Engineering Strategies

The study compared two distinct feature construction approaches:

1. Clinician-Curated Baseline: A feature set developed iteratively by three cardiologists. It included standard demographics, vitals, and summarized clinical measures, prioritizing clinical relevance and interpretability.
2. Automated Feature Construction (DFS): Using the Deep Feature Synthesis algorithm to automatically generate features from multi-domain EHR tables (labs, vitals, meds, procedures).
   • Configuration: Shallow depth (max depth = 1) to maintain interpretability.
   • Primitives: Aggregation (mean, max, min, count) and datetime transformations (month, weekday, etc.).
   • Scale: Generated up to 5,000 candidate features per horizon, filtered down to a usable set.

Model Development

To isolate the effect of feature engineering, the same model families were trained on both the Baseline and DFS-enhanced feature sets:

• Logistic Regression (LR): A transparent linear baseline.
• Gradient-Boosted Decision Trees (LightGBM/XGBoost): To capture non-linear interactions.
• Multilayer Perceptron (MLP): A neural network baseline.

Evaluation Metrics

• Discrimination: Area Under the ROC Curve (AUROC) and Area Under the Precision-Recall Curve (AUPRC).
• Calibration: Brier Score and Brier Skill Score (BSS).
• Operating Characteristics: Sensitivity, Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV).
• Clinical Relevance: Performance was evaluated at a fixed sensitivity target of 80% to simulate real-world workflows where health systems prioritize capturing high-risk patients while managing false-positive alerts.

3. Key Results

Discrimination and Precision-Recall

• Gradient-Boosted Trees (LightGBM): DFS consistently improved performance across all horizons.
  • AUROC: Increased by +0.015 to +0.016 across 30, 60, and 90 days.
  • AUPRC: Increased by +0.018 to +0.021.
• Logistic Regression: DFS resulted in neutral or negative performance changes.
  • AUROC: Decreased by -0.013 to -0.015.
  • AUPRC: Decreased by -0.012 to -0.020.

Operating Characteristics (at 80% Sensitivity)

• LightGBM: DFS significantly improved Specificity and PPV while maintaining the target sensitivity.
  • Example (30-day): Specificity improved from 0.470 to 0.502; PPV improved from 0.228 to 0.239.
  • Impact: This translates to fewer false-positive alerts per true readmission (reducing clinician alert fatigue).
• Logistic Regression: DFS degraded performance, lowering both specificity and PPV compared to the baseline.

Calibration

• LightGBM: DFS consistently improved calibration (lower Brier scores) across all horizons. The 95% confidence intervals for the difference in Brier scores did not cross zero.
• Logistic Regression: DFS showed no calibration benefit at 30/60 days and worsened calibration at 90 days (Brier score increased).

4. Key Contributions

1. Model-Class Dependence of Automated FE: The study provides robust evidence that the benefits of automated feature engineering are strongly dependent on the model class. DFS enhances non-linear, tree-based models but degrades linear models.
2. Clinical Applicability Beyond AUC: The research moves beyond aggregate discrimination metrics to demonstrate improvements in deployment-relevant metrics, specifically reducing false-positive workloads at clinically relevant sensitivity thresholds.
3. Calibration Improvements: It highlights that DFS not only improves ranking ability but also the reliability of probability estimates (calibration) for tree-based models, which is crucial for risk stratification.
4. Practical Middle Ground: The work suggests DFS as a viable "middle ground" between manual feature engineering and complex end-to-end deep learning, offering systematic temporal feature extraction while remaining compatible with interpretable, deployed models like LightGBM.

5. Significance and Implications

• For Clinical Deployment: Healthcare systems implementing readmission prediction should pair automated feature engineering (like DFS) with gradient-boosted tree models rather than linear models to maximize clinical utility and reduce alert fatigue.
• For Research: The findings challenge the assumption that "more features" or "automated features" are universally beneficial. The inductive bias of the model must align with the feature representation; linear models cannot effectively utilize the high-dimensional, non-linear interactions generated by DFS.
• Future Directions: The authors call for prospective studies to validate if these statistical improvements translate to better patient outcomes and clinical workflows, as well as further investigation into feature pruning and interpretability for DFS-generated features.

Conclusion: Automated feature engineering via DFS is a powerful tool for improving HF readmission prediction, but its success is contingent upon using non-linear learners (specifically gradient-boosted trees). When paired correctly, it yields meaningful gains in discrimination, calibration, and operational efficiency.