📄 medicine

Evaluation of risk prediction models for coronary artery disease: a systematic review, meta- analysis, and machine learning validation

This systematic review and meta-analysis of 68 studies reveals that while existing coronary artery disease risk prediction models show moderate discriminative ability, their high risk of bias and heterogeneity limit clinical applicability, suggesting a need for future research focused on specific subgroups and machine learning-based models.

Original authors: Tanglin Ouyang, Daoxi Qi, Xinxin Miao, Xinai Xie, Xinyu Deng, Fan Wang, Liang Cao, Chang Zhao, Ruiyang Zhu, Shuyang Sheng, Casimir Dewanou Akpovi, Antoine Abel Missihoun, Augustin Gaétan Julien Sègbo

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tanglin Ouyang, Daoxi Qi, Xinxin Miao, Xinai Xie, Xinyu Deng, Fan Wang, Liang Cao, Chang Zhao, Ruiyang Zhu, Shuyang Sheng, Casimir Dewanou Akpovi, Antoine Abel Missihoun, Augustin Gaétan Julien Sègbo, Clément Agbangla, Fang Zheng

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict which cars on a busy highway are about to break down. For years, mechanics (doctors) have been using various "risk checklists" to guess which vehicles might stall. This paper is like a massive inspection of all those checklists, combined with a brand-new, high-tech diagnostic tool built by the authors.

Here is the story of their findings, broken down simply:

1. The Big Cleanup (The Systematic Review)

The researchers went on a digital treasure hunt, searching through over 15,000 scientific papers about Coronary Artery Disease (CAD)—the clogging of heart arteries. After throwing out duplicates, non-English papers, and studies that didn't fit their rules, they were left with 68 solid studies.

Think of these 68 studies as 68 different "weather forecasts" for heart attacks. The researchers wanted to know: Do these forecasts actually work?

2. The "Goldilocks" Problem (The Meta-Analysis)

They combined the results of 58 of these models to get an average score.

The Score: The average model got a score of 0.805. In the world of predictions, this is like getting a "B" grade. It's decent—it can tell the difference between a healthy heart and a sick one better than flipping a coin—but it's not perfect.
The Catch: While the scores looked okay, the researchers found that almost every single study was "rigged" or flawed. They used a tool called PROBAST (think of it as a strict quality inspector) and found that 54 out of 68 studies had a "High Risk of Bias."

Why were they flawed?
Imagine if one mechanic checked a Ferrari, another checked a rusty truck, and a third checked a bicycle, but they all claimed to be testing "cars." The studies used different types of patients, different lab tests, and different math methods. Because the ingredients were so mixed up, the results were all over the place (high "heterogeneity"). It's hard to trust a recipe if the chefs are using different ingredients and different ovens.

3. The New High-Tech Tool (Machine Learning Validation)

Since the old checklists were messy, the authors decided to build their own "super-predictor" using Machine Learning (AI). They gathered data from 1,083 real patients at their hospital in Wuhan, China. They split them into three groups:

Healthy Controls (The "Good" cars).
Stable Angina (The "Squeaky" cars—chronic issues).
Acute Coronary Syndrome (The "Smoking" cars—immediate emergencies).

They fed this data into 13 different AI algorithms (like CatBoost, LightGBM, and Random Forest) to see which one could best spot the sick hearts.

The Results:
The AI models were incredibly sharp, almost scary so.

For the Acute (Emergency) group, the AI got a score of 0.993. That's nearly a perfect "A+."
For the Stable group, it got a 0.980.
For the General CAD group, it got a 0.984.

Why was the AI so good?
The authors explain that patients having an acute heart attack have very obvious, dramatic changes in their blood and body (like a car engine that is literally on fire). The AI could easily spot these loud signals. However, patients with stable, long-term issues are more like a car with a slow leak; the signals are quieter and harder to distinguish, which is why the score was slightly lower (though still excellent).

4. The Reality Check (The Conclusion)

The paper ends with a very important warning.

The authors admit that while their new AI tool is amazing in their specific hospital, it might not work as well everywhere else.

The "Local" vs. "Global" Gap: Their AI scored nearly perfect (0.99), but the global average of all other studies was only moderate (0.80). Why? Because their study was very controlled (like a test track), while the global studies were messy (like real-world traffic).
The Bias Issue: Because so many existing studies were poorly designed, we can't fully trust the "average" score of 0.80. It might be too high or too low because the data was messy.

The Bottom Line

The paper tells us two main things:

Current Tools are Flawed: Most existing heart disease prediction models are built on shaky ground. They are inconsistent and often biased, making them hard to rely on for every patient.
AI Shows Promise: When you use modern AI on clean, well-defined data, it can spot heart disease with incredible accuracy, especially for acute emergencies.

However, the authors are careful not to say "This AI is ready for every hospital tomorrow." They say we need to build better, more consistent models and test them in many different places before we can fully trust them to replace the old checklists. They are essentially saying, "We found a Ferrari engine, but we need to make sure it fits in every car before we sell it."

Technical Summary: Evaluation of Risk Prediction Models for Coronary Artery Disease

Problem Statement
Coronary artery disease (CAD) remains a leading cause of global mortality, necessitating early diagnosis and prevention. While numerous risk prediction models exist, their practical applicability in clinical settings is hindered by inconsistencies in study designs, subject selection, parameter definitions, and analytical methodologies. There is a critical need to systematically evaluate the diagnostic performance and quality of existing CAD risk prediction models to determine their reliability and identify directions for future clinical integration and research.

Methodology
This study employed a three-pronged approach: a systematic review, a meta-analysis, and a local machine learning validation.

Systematic Review and Meta-Analysis:
- Search Strategy: A comprehensive search was conducted across PubMed, Embase, and Web of Science up to February 22, 2025.
- Inclusion/Exclusion: The review included prospective, retrospective, cohort, case-control, and cross-sectional studies involving individuals aged 18+ (suspected CAD, general population, or confirmed CAD vs. controls). Studies were excluded if they were non-English, reviews, lacked valid outcome data (e.g., AUC), or focused exclusively on specific subgroups like confirmed CAD severity or mortality.
- Quality Assessment: The Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to evaluate bias across four domains: participants, predictors, outcomes, and statistical analysis.
- Statistical Analysis: Meta-analysis of Area Under the Receiver Operating Characteristic Curve (AUC) was performed using STATA 17. Heterogeneity was assessed via $I^2$ and Cochran's Q test. Random-effects models were applied when $I^2 \geq 50\%$ . Sensitivity and subgroup analyses were conducted based on patient type (e.g., Chronic Ischemic Syndrome vs. Acute Coronary Syndrome), model development method (machine learning vs. traditional), and study design.
Machine Learning Validation (Local Cohort):
- Data Source: Data were collected from 1,083 patients at Zhongnan Hospital of Wuhan University (January 1, 2025 – December 31, 2025), categorized into Stable Angina (SA, n=393), Acute Coronary Syndrome (ACS, n=302), and Controls (n=388).
- Feature Selection: Three methods (Lasso regression, SVM-RFE, and Random Forest) were applied to 51 baseline features. The intersection of these methods yielded 11 core features (including Hyperlipidemia, T2DM, HGB, LYMPH Percentage, DBIL, UBIL, TP, TG, HDL, NHDL, and Urea).
- Model Training: Thirteen machine learning algorithms (including CatBoost, LightGBM, SVM, RF, Neural Networks, etc.) were trained and compared. Hyperparameter optimization was performed via grid search with 5-fold repeated cross-validation.
- Validation: Models were tested on binary classification tasks: Control vs. CAD, Control vs. SA, and Control vs. ACS.

Key Contributions and Results

Systematic Review Scope: From 15,232 retrieved articles, 68 studies met the inclusion criteria. The majority utilized logistic regression (25 studies) or Cox models (9 studies). Common predictors included age, gender, smoking, diabetes, and hypertension.
Quality Assessment: A high risk of bias was identified in 54 of the 68 studies, primarily due to issues in study population selection (35 studies) and statistical analysis methods (41 studies), such as unclear handling of missing data or lack of calibration reporting.
Meta-Analysis Performance:
- The pooled AUC for 58 validated models was 0.805 (95% CI: 0.784–0.827), indicating moderate discriminative ability.
- Heterogeneity was extremely high ( $I^2 = 98.90\%$ ).
- Subgroup analyses revealed that machine learning-based models (8 studies) had a pooled AUC of 0.842, while traditional methods (25 studies) had a pooled AUC of 0.799.
- Studies focusing on Chronic Ischemic Syndrome (CIS) showed moderate heterogeneity ( $I^2 = 67.12\%$ ) with a pooled AUC of 0.812.
- Egger's test suggested potential publication bias in the overall meta-analysis ( $P = 0.0126$ ).
Local Machine Learning Validation:
- The local validation demonstrated superior performance compared to the meta-analysis averages.
- ACS Group: The LightGBM model achieved an AUC of 0.993 (95% CI: 0.986–1.000).
- General CAD Group: The CatBoost model achieved an AUC of 0.984 (95% CI: 0.975–0.993).
- SA Group: The Random Forest (RF) model achieved an AUC of 0.980 (95% CI: 0.967–0.993).
- The study noted that the high performance in the ACS group may be attributed to the acute pathophysiological changes and more pronounced traditional risk factors in this population.

Significance and Claims
The paper claims to be the first to perform a hierarchical analysis on CAD risk prediction models, systematically summarizing diagnostic thresholds and identifying sources of inconsistency. The study highlights that while existing models possess moderate discriminatory power, their clinical utility is limited by a high risk of bias and lack of external validation.

The authors assert that their local machine learning validation supports the meta-analysis conclusion that subgroup-specific modeling (e.g., distinguishing ACS from SA) and the use of advanced algorithms can significantly enhance predictive performance. They propose that future research should focus on:

Developing models for specific CAD subgroups.
Utilizing machine learning to capture non-linear interactions.
Adhering strictly to PROBAST guidelines to reduce bias, ensure consistent predictor definitions, and conduct rigorous internal and external validation.

The study concludes that while the developed models show promise, prospective multi-center external validation is essential before they can be considered ready for broad clinical application.