Electronic Health Record-Based Estimation of Kansas City Cardiomyopathy Questionnaire Scores in Heart Failure

This study demonstrates that machine learning models trained on electronic health record data can accurately estimate Kansas City Cardiomyopathy Questionnaire scores and classify heart failure severity, offering a viable solution to overcome the limitations of incomplete patient-reported outcome data collection.

The Gist

The Big Problem: The Missing Puzzle Piece

Imagine you are a doctor trying to understand how a heart failure patient is really feeling. You have their blood work, their X-rays, and their medication list. But there is one crucial piece of the puzzle missing: how the patient feels.

In the medical world, this "feeling" is measured by a survey called the KCCQ (Kansas City Cardiomyopathy Questionnaire). It asks things like, "Can you walk up a flight of stairs?" or "Do you feel tired?"

The Catch: Patients are busy, sick, or tired. They often forget to fill out the survey, or they don't fill it out completely. It's like trying to bake a cake but forgetting to add the sugar. You have all the other ingredients (the medical data), but you're missing the key ingredient (the patient's voice) that tells you if the cake tastes good.

The Solution: The "Sherlock Holmes" AI

The researchers in this paper asked a bold question: "If we can't get the survey, can we guess the score using the medical records we already have?"

They built a Machine Learning (AI) detective called a "Histogram-based Gradient Boosting" model. Think of this AI as a super-smart Sherlock Holmes. Instead of asking the patient, "How do you feel?", the AI looks at the clues left behind in the hospital's computer system (the Electronic Health Record or EHR).

The Clues the AI looked for:

• The "Symptom" Clues: Did the patient mention swelling in their feet? Did they have trouble sleeping sitting up?
• The "Lifestyle" Clues: Did they stop going shopping? Did they stop exercising? (The AI found that if a patient stops shopping, it's a huge red flag they aren't feeling well).
• The "Lab" Clues: Are their blood markers for heart stress high? Is their oxygen low?
• The "History" Clues: What happened in the last 240 days?

How They Trained the Detective

The researchers gathered data from 10,889 heart failure patients. They gave the AI a "lookback window" (a time machine) to see what happened in the patient's life before the survey was taken.

They tested different time windows:

• 15 days: Too short. It's like trying to judge a movie by watching only the first 15 minutes. The AI got confused.
• 240 days (8 months): This was the "Goldilocks" zone. It was long enough to see the patient's full story and patterns, but not so long that the data got messy.

The Result: The AI got really good at guessing. It could predict the survey score with about 52% accuracy (which is huge in the medical world) and made mistakes of only about 12 points on a scale of 0 to 100.

The "Calibration" Trick: Fixing the Blind Spot

Here is the clever part. The AI was great at telling the difference between "Okay" and "Great," but it was terrible at spotting the very sick patients (the ones with the lowest scores).

Why? Because in the real world, very sick patients are rare. The AI learned that "most people are okay," so it guessed "okay" for everyone to be safe.

The Fix: The researchers applied a "calibration" filter. Imagine a fishing net. Originally, the net had holes that were too big to catch the tiny, sick fish. They tightened the mesh specifically to catch the sick ones, even if it meant catching a few more "okay" fish by mistake.

The Payoff: After this fix, the AI became twice as good at identifying the most dangerous patients. This is critical because catching a patient who is about to crash allows doctors to step in early and save a life.

What Did the AI Learn? (The "Aha!" Moments)

The researchers peeked under the hood to see why the AI made its guesses. They found some fascinating things:

1. It uses the "Little Things": The AI realized that if a patient stops doing daily chores (like bathing or shopping), their heart health is likely plummeting.
2. It sees the "Whole Person": It didn't just look at the heart. It looked at lung issues, alcohol use, and even social factors (like legal status). It understood that heart failure isn't just a heart problem; it's a life problem.
3. It works without the Survey: Even though the AI could see the actual survey answers if they were there, it mostly relied on the medical records. This proves that we don't always need the patient to fill out a form to know how they are doing.

Why This Matters (The Bottom Line)

Think of this study as building a backup generator for patient care.

• Right now: If a patient doesn't fill out the survey, the doctor is flying blind regarding their quality of life.
• With this AI: The doctor can look at the computer, and the AI will say, "Hey, based on their lab results, their missed appointments, and their recent blood work, this patient is likely feeling terrible, even though they didn't fill out the form."

This allows doctors to catch sick patients earlier, help more people, and make sure that the patients who are too sick or too busy to speak up still get the help they need. It turns the "silent" data in the hospital computer into a loud, clear voice for the patient.

Technical Summary

1. Problem Statement

The Kansas City Cardiomyopathy Questionnaire (KCCQ) is a gold-standard, FDA-qualified patient-reported outcome (PRO) measure used to assess health status, symptom burden, and functional capacity in heart failure (HF) patients. Despite its prognostic value (predicting hospitalization and mortality), its clinical utility is severely limited by incomplete and inconsistent data collection in routine practice.

• Barriers: Patient fatigue, competing demands, literacy/language issues, and critical illness often lead to missing KCCQ data.
• Consequence: Sparse data hinders population-level monitoring, risk stratification, and research, creating a gap in understanding the true health status of the HF population.
• Objective: The study aims to develop and validate machine learning (ML) models to estimate KCCQ overall summary scores directly from routine Electronic Health Record (EHR) data, bypassing the need for direct survey administration.

2. Methodology

Data Source and Cohort

• Database: Truveta database (aggregating data from >900 US hospitals and 20,000 clinics; >120 million individuals).
• Cohort: 10,889 unique heart failure patients with at least one recorded KCCQ score between 2012–2025.
• Exclusion Criteria: Missing scores, invalid ranges (0–100), duplicates, and patients lacking prior EHR history.
• Outcome: KCCQ Overall Summary Score (0–100). Scores were categorized into four severity strata: Very Poor/Poor (0–25), Poor/Fair (25–50), Fair/Good (50–75), and Good/Excellent (75–100).

Feature Engineering

• Temporal Windows: Predictor features were extracted from 13 distinct historical time windows preceding the KCCQ assessment (15, 30, 60, 90, 120, 150, 180, 210, 240, 270, 300, 330, and 360 days).
• Feature Types: Structured EHR data including demographics, clinical conditions (frequency counts), and laboratory/vital sign summaries.
• Scale: The initial feature space was high-dimensional, ranging from ~11,800 to ~19,200 features depending on the time window.

Model Development and Validation

• Algorithm Selection: A three-stage workflow was employed:
  1. Screening: Comparison of regression algorithms. Histogram-based Gradient Boosting (HGB) consistently outperformed others (e.g., RF, XGBoost, SVR).
  2. Feature Reduction: Comparison of strategies including Lasso, Elastic Net, and SHapley Additive exPlanations (SHAP). HGB-derived SHAP rankings were selected for stability and efficiency.
  3. Optimization: Hyperparameter tuning via nested cross-validation (CV) to prevent data leakage.
• Calibration: To address class imbalance in categorical severity prediction, a quantile-based calibration approach was applied. This adjusted category thresholds based on the empirical distribution of the training set to improve sensitivity for severe cases without altering the underlying continuous predictions.

3. Key Results

Predictive Performance (Continuous Scores)

• Best Window: The 240-day window yielded the optimal performance, achieving an $R^2$ of 0.522 and a Mean Absolute Error (MAE) of 12.485.
• Window Comparison: While the 15-day window (matching the KCCQ recall period) performed worst, performance improved with longer windows, stabilizing between 60–360 days. This suggests that capturing longitudinal patterns is more critical than short-term symptom fluctuations.
• Feature Reduction: The SHAP-based selection reduced the feature set by >94% (from ~18,000 down to ~700 features) while maintaining predictive accuracy comparable to the full model.

Categorical Severity Classification

• Discrimination: The model demonstrated strong ordinal discrimination with a mean Ordinal AUROC of 0.850.
• Calibration Impact:
  • Before Calibration: The model struggled to identify the most severe group (KCCQ < 25), with an F1-score of only 0.180.
  • After Calibration: The F1-score for the "Very Poor to Poor" category improved significantly to 0.428.
  • Overall Metrics: Macro F1 increased from 0.490 to 0.548, and Quadratic Weighted Kappa (QWK) improved from 0.601 to 0.640, indicating substantial agreement across ordered categories.

Feature Importance (SHAP Analysis)

• Top Predictors: The model relied on a diverse mix of features:
  • Sparse KCCQ Items: When available, specific item responses (e.g., swelling, sleep difficulty) were top predictors, though they were missing ~80% of the time.
  • Functional/Lifestyle: Alcohol use (AUDIT-C), shopping/bathing limitations, and exercise duration.
  • Clinical/Physiological: Respiratory comorbidities (COPD, dyspnea), lab values (BUN, natriuretic peptides, troponin I, EDR), and vital signs.
  • Demographics: Age, sex, and legal status.
• Subgroup Analysis: In the bottom 25% of predicted scores (poorest health), functional limitations and alcohol use became relatively more important compared to the global average.

4. Key Contributions

1. First Rigorous EHR-to-KCCQ Framework: Demonstrates that routine, non-survey EHR data can estimate PRO scores with clinically meaningful accuracy ($R^2 \approx 0.52$), outperforming typical benchmarks for complex clinical outcome prediction.
2. Optimal Temporal Window Identification: Challenges the assumption that short-term data is sufficient, showing that a 240-day longitudinal window provides the best predictive signal for health status.
3. Calibration Strategy: Introduces a post-hoc quantile calibration method that significantly improves the identification of high-risk patients (severe impairment) without sacrificing overall model discrimination.
4. High-Dimensional Efficiency: Proves that >94% of EHR features can be pruned using SHAP without losing predictive power, making the model computationally efficient and interpretable.

5. Significance and Clinical Implications

• Bridging Data Gaps: This approach allows for the assessment of patient-reported health status in populations where surveys are incomplete or unavailable (e.g., critically ill, low literacy, or non-English speaking patients).
• Population Health Management: Enables scalable, automated risk stratification and monitoring of heart failure populations at the health system level, potentially triggering proactive interventions for deteriorating patients.
• Equity: By relying on structured EHR data rather than self-reported surveys, the model may help reduce disparities in health status measurement for populations historically underrepresented in PRO data collection.
• Future Directions: The authors suggest this framework could be adapted for other PROs and emphasize the need for external validation across diverse demographics to ensure equity and generalizability.

Limitations: The study is retrospective, relies on structured data (excluding unstructured notes), and lacks item-level KCCQ granularity. External validation in independent cohorts is required before clinical deployment.