Feature engineering of digital health data for machine learning prediction of COPD exacerbations in a real-world cohort

Technical Summary: Feature Engineering for COPD Exacerbation Prediction

Problem Statement
Chronic Obstructive Pulmonary Disease (COPD) exacerbations are a primary driver of morbidity, mortality, and hospital admissions, yet effective short-term prediction models remain limited. While digital self-management platforms generate longitudinal data, previous attempts to predict exacerbations using limited sets of self-reported features have yielded only modest precision. Furthermore, it is unclear whether integrating routinely captured physical activity data (e.g., wrist-worn step counts) provides additional predictive value beyond symptom and health-status data. This study addresses the need for scalable, early warning systems that can distinguish between stable periods and impending exacerbations using real-world digital health data.

Methodology
The study utilized a retrospective cohort of 5,865 users from the myCOPD digital self-management platform. The analysis focused on two cohorts: a primary analytic cohort of 4,223 users with at least one labelled app entry, and a wearable subgroup of 459 users with linked wrist-worn step-count data.

• Prediction Task: The model predicted whether an exacerbation (defined as a symptom score of 3 or 4) would occur within the patient's next 1–3 app entries, rather than at fixed calendar intervals. This approach reflects a pragmatic, engagement-driven use case.
• Feature Engineering: The authors moved beyond raw variables to a rich, engineered feature set.
  • Dynamic Features: Included current symptom state, lagged values (1, 2, 3, 7 entries), rolling summaries (means, standard deviations), and deviations from personalized baselines (calculated from historically stable entries).
  • Exacerbation History: Features captured the recency and frequency of prior events (e.g., counts over 28/90 entries, time since last event).
  • Clinical & Engagement: Included COPD Assessment Test (CAT) scores, medication use, pulmonary rehabilitation load, and app engagement metrics.
  • Wearable Data: In the subgroup, step-count features were engineered similarly to app data, including lagged values, rolling statistics, and deviations from a personal step baseline.
• Modeling Strategy: XGBoost gradient boosting classifiers were trained using grouped nested cross-validation to ensure patient-level separation between training and testing sets (preventing data leakage).
  • Primary Cohort: Trained on 547,465 labelled entries.
  • Ablation Study: In the wearable subgroup, two identical models were compared: one with app-derived features only, and one with app features plus step-count features.
• Evaluation: Performance was evaluated using Precision-Recall AUC (PR-AUC), ROC-AUC, precision, and recall. Thresholds were optimized to maximize the F1 score within training folds to balance sensitivity and positive predictive value in a low-prevalence setting (4.8% positive class).

Key Results

• Primary Model Performance: In the main cohort, the model achieved a mean PR-AUC of 0.284 and a mean ROC-AUC of 0.834. At the optimized threshold, the model achieved a mean precision of 29.8% and recall of 41.2%. This represents a significant improvement in precision compared to previous myCOPD modeling efforts (approx. 7% precision) under a more stringent validation framework.
• Predictive Horizons: Positive alerts were followed by a self-reported exacerbation in nearly 30% of cases. The median time from a positive alert to the first exacerbation entry was 5 days, with 39.5% occurring within 3 days and 56.0% within 7 days.
• Feature Importance: The strongest predictors were symptom-derived variables (current state, variability, and lagged values) and recent exacerbation history. CAT-derived features and engagement patterns contributed meaningfully, while static clinical characteristics (e.g., age, smoking history) were less influential.
• Wearable Ablation: In the wearable subgroup, adding step-count features did not materially improve performance. The app-only model and app-plus-steps model yielded nearly identical results (e.g., PR-AUC 0.162 vs. 0.163; Precision 0.214 vs. 0.214). Descriptive trajectory analysis confirmed that step-count deviations showed only modest and inconsistent differences prior to exacerbations compared to the clearer separation seen in CAT and symptom trajectories.

Significance and Claims
The authors claim that this study demonstrates the feasibility of scaling exacerbation prediction using routinely collected digital self-management data. The primary contribution is the validation that feature-engineered representations of symptom and health-status trajectories significantly improve prediction precision compared to raw or limited feature sets, even under strict patient-level validation.

The study concludes that self-reported symptom signals captured through digital platforms provide useful early warnings of impending exacerbations. In contrast, the authors claim that wrist-worn step counts, as captured in this specific cohort and context, offer limited incremental value beyond the rich symptom and CAT data. The findings suggest that for this specific prediction task, the "signal" of an impending exacerbation is more strongly contained in the patient's reported symptom burden and history than in their physical activity levels. The authors emphasize that the model functions as a near-term early warning system triggered by user engagement, rather than a fixed-day calendar predictor, and that future utility depends on translating these predictive signals into actionable clinical or self-management interventions.