Bidirectional Prediction of Diabetes-Hypertension Comorbidity Transitions from Multimodal Outpatient EMRs: A Reference-Deviation-Aware Multimodal Stacking Framework (RD-MMS)

Technical Summary: Bidirectional Prediction of Diabetes-Hypertension Comorbidity Transitions via RD-MMS

Problem Statement
The management of chronic diseases like Diabetes Mellitus (DM) and Hypertension (HTN) is highly specialized, creating systematic blind spots in cross-disease risk identification. Endocrinologists typically manage DM while cardiologists manage HTN, often failing to screen for the bidirectional transition risks between these conditions. Existing predictive models suffer from three primary limitations when applied to real-world clinical settings:

1. Data Source Mismatch: Most models rely on inpatient data or population surveys, which differ significantly from outpatient data in information density and quality.
2. Unidirectional Modeling: Current approaches typically model disease progression in a single direction (e.g., DM to HTN), failing to capture the task asymmetry inherent in bidirectional comorbidity transitions.
3. Technical Challenges in Outpatient Data: Real-world outpatient Electronic Medical Records (EMRs) present challenges including cross-instrument heterogeneity in laboratory data, high sparsity (low laboratory test coverage), and hidden information leakage risks that inflate internal validation performance.

Methodology: The RD-MMS Framework
To address these challenges, the authors propose the Reference-Deviation-Aware Multimodal Stacking (RD-MMS) framework. This architecture is designed to perform bidirectional prediction of DM-to-HTN and HTN-to-DM transitions using a large-scale, real-world outpatient multimodal dataset (2023–2024) from Beijing Pinggu Hospital.

The framework consists of four core components:

1. Reference-Deviation (RD) Encoding:

   • Instead of conventional Z-score or min-max normalization, which rely on global statistics and require mean imputation for missing values, RD encoding utilizes patient-specific reference intervals maintained by the Laboratory Information System (LIS).
   • Laboratory values are encoded as relative deviations: values within the normal range are set to zero; abnormal values are encoded as directional continuous deviations; and missing values (NaN) are preserved intact.
   • This approach resolves cross-instrument heterogeneity and allows tree-based models to utilize "not tested" as a weak negative signal, rather than treating missingness as noise.

2. Multimodal Feature Fusion:

   • The system integrates five heterogeneous feature types:
     • RD-encoded numerical features: Derived from 107 core laboratory indicators.
     • Text embeddings: BERT semantic embeddings (768 dimensions) and character-level TF-IDF (500 dimensions) extracted from clinical narratives.
     • Visit behavior statistics: Metrics such as visit density, intervals, and counts.
     • Guideline-based clinical thresholds: 20 features encoding abnormal states (e.g., blood pressure grading, eGFR staging) based on ESC/ESH, ADA, and CKD-EPI guidelines.
   • These features are concatenated to form a high-dimensional input vector (~2,357 dimensions).

3. Four-Dimensional Leakage Control:

   • To prevent data leakage and ensure the model uses only information available prior to the index visit, the pipeline enforces:
     • Patient-level splitting: GroupKFold cross-validation ensures no patient overlap between folds.
     • Temporal cutoff: Features are restricted to records dated before the baseline.
     • Semantic masking: Disease names and targeted drugs in text are replaced with [UNK] tokens; LIS year fields are masked.
     • Within-fold preprocessing: Scaling and feature selection are fitted exclusively within the training fold.

4. Stacking Ensemble:

   • Six base models (MLP, RF, LR, XGBoost, LightGBM, CatBoost) are trained in parallel.
   • An L2-regularized logistic regression meta-learner integrates the out-of-fold (OOF) predictions from the top-performing base models to generate the final risk score, reducing overfitting.

Key Results
The framework was evaluated on a development cohort (2023–2024) and an independent 2025 holdout cohort.

• Predictive Performance:

  • DM-to-HTN: AUROC of 0.767 (Internal) and 0.748 (External).
  • HTN-to-DM: AUROC of 0.735 (Internal) and 0.715 (External).
  • The model outperformed all baseline methods (including single-model XGBoost and logistic regression) and existing benchmarks, despite laboratory test coverage in the outpatient setting being only 13–18%.
  • Temporal drift on the external cohort was minimal (<1.5%), whereas a standard Random Forest model without OOF mechanisms degraded significantly (AUROC drop of 0.122).

• Ablation and Perturbation Studies:

  • RD Encoding: Semantic perturbation (randomly mismatching reference intervals) caused AUROC drops of 0.028–0.030, confirming the model relies on clinical semantics rather than just numerical scaling.
  • Sparsity Robustness: As laboratory missingness increased to 90%, the full multimodal model's performance degraded by only 0.059, compared to 0.149 for a numerical-only model. Text features provided a compensatory gain of up to +0.110 AUROC in high-sparsity scenarios.
  • Leakage Control: Masking LIS year fields improved AUROC by ~0.030, highlighting the impact of implicit temporal leakage.
  • SMOTE Sensitivity: Applying SMOTE to tree-based models disrupted "not tested" signals, causing a performance drop of 0.041; restricting SMOTE to non-tree models restored performance.

• Clinical Utility:

  • Risk Stratification: The top 10% high-risk group showed a 2.9-fold enrichment in event rates compared to the baseline.
  • Decision Curve Analysis (DCA): The model demonstrated superior net benefit over "treat-all" and "treat-none" strategies across threshold probabilities of 2%–50%.
  • Feature Importance: SHAP analysis revealed task asymmetry. DM-to-HTN was driven by vascular aging and blood pressure, while HTN-to-DM was heavily influenced by visit density and text-extracted glucose anomalies, underscoring the value of multimodal data.

Significance and Claims
The paper claims that RD-MMS provides a complete methodological paradigm for constructing reliable chronic comorbidity prediction models from real-world outpatient EMRs. Its primary contributions include:

1. Addressing Outpatient Specifics: It offers a systematic solution for the specific technical challenges of outpatient data, including low laboratory coverage and cross-instrument heterogeneity, achieving performance comparable to inpatient-based studies despite lower data quality.
2. Bidirectional Asymmetry: It demonstrates that bidirectional modeling is necessary to reveal task-level information asymmetry (e.g., the differing predictive value of text features in DM-to-HTN vs. HTN-to-DM), which unidirectional models obscure.
3. Robustness via Leakage Control: It establishes that rigorous four-dimensional leakage control and OOF stacking are critical for temporal generalizability, preventing the inflated internal validation metrics common in longitudinal EMR studies.
4. Clinical Deployability: The framework relies solely on data sources universally available in primary care (EMR, LIS, visit logs) and does not require specialist interpretation, making it feasible for embedded deployment in Hospital Information Systems to address cross-specialty diagnostic blind spots.
5. Generalizability: The modular, configuration-driven design allows the framework to be decoupled from specific disease types, supporting potential extension to other chronic comorbidity pairs (e.g., CKD-DM) with minimal reconfiguration.

The authors conclude that the framework successfully bridges the gap between specialized disease management and proactive cross-disease prevention, offering a deployable technical pathway for precision chronic disease management.