Machine learning-based advanced coronary artery disease pretest probability model: Comparison with conventional pretest probability models

This study developed and validated a machine learning-based pretest probability model (K-CAD) using ridge-penalized logistic regression on extensive Korean datasets, demonstrating that it significantly outperforms conventional models (UDF and CAD2) in predicting coronary artery disease risk within the Korean population.

The Gist

Imagine you are a doctor trying to figure out if a patient has a clogged pipe in their heart (coronary artery disease). To decide whether to send them for a risky, expensive, and invasive test (like a camera inside the heart), you first need to guess the odds. This guess is called the Pretest Probability.

For decades, doctors have used two main "rulebooks" (models) to make this guess: the Updated Diamond–Forrester (UDF) and the CAD Consortium (CAD2) models. Think of these rulebooks as weather forecasts created by meteorologists in London. They are great for predicting rain in London, but if you take them to Seoul, they might say it's going to pour when it's actually sunny, or vice versa.

Here is the story of how this paper fixed that problem for people in Korea.

1. The Problem: The "London Weather Forecast" in Seoul

The old rulebooks were built using data mostly from Western populations (Europe and the US). However, people in Korea (and East Asia generally) have different body types, different diets, and different patterns of heart disease.

When Korean doctors used the old "London" rulebooks, the results were messy:

• False Alarms: The models kept screaming "DANGER!" for people who were actually fine.
• The Consequence: Because the models said "High Risk," too many healthy people were sent for unnecessary, scary, and expensive heart scans. It was like calling a fire truck for a burnt piece of toast.

2. The Solution: Building a "Seoul-Specific" Weather App

The researchers in this paper decided to build a brand-new model specifically for Koreans, which they named K-CAD.

Instead of just looking at the basics (Age, Gender, and "Do you have chest pain?"), they built a smarter system that also looked at routine blood test results that almost everyone gets at the doctor's office anyway.

• The Ingredients: They took data from nearly 5,000 Korean patients. They fed the computer information about cholesterol, blood sugar, kidney function, and blood pressure, along with the usual symptoms.
• The Engine: They used a special type of math (Ridge-Penalized Logistic Regression). Think of this as a "smart filter" that prevents the computer from getting confused by too much information or memorizing the wrong answers. It finds the true signal in the noise.

3. The Test Drive: How did it perform?

The team put their new K-CAD model to the test against the old rulebooks using two different groups of people:

1. High-Risk Patients: People already in the hospital with chest pain.
2. The General Public: Over 117,000 people from a national health checkup database.

The Results:

• The Old Models (UDF & CAD2): They were okay, but they kept misclassifying people. They were like a security guard who stops everyone at the airport, even the people just visiting their grandma.
• The New Model (K-CAD): It was much sharper.
  • It correctly identified more people who actually had heart blockages.
  • Crucially, it stopped screaming "High Risk" for people who were actually low-risk. It successfully reclassified nearly 80% of the people the old models wrongly labeled as "High Risk" into "Low Risk."

4. The Analogy: The Tailored Suit vs. The Off-the-Rack Jacket

Imagine the old models are like buying a suit off the rack from a store in New York. It might fit a tall, broad-shouldered American, but for a Korean man, the sleeves might be too long, and the shoulders too tight. You can wear it, but it's uncomfortable and doesn't look right.

The K-CAD model is like a custom-tailored suit made specifically for the Korean body type. It uses measurements (blood tests and symptoms) that fit the local population perfectly. It fits better, looks better, and tells you exactly how you need to dress (or in this case, how much medical attention you need).

5. Why Does This Matter?

This isn't just about math; it's about saving people from unnecessary stress and money.

• Less Unnecessary Testing: By accurately identifying low-risk patients, fewer people will be sent for invasive heart procedures they don't need.
• Better Care: Doctors can focus their resources on the people who actually need them.
• Transparency: Unlike some "black box" AI models that no one understands, this model is open. The authors even built a free online calculator so any doctor can use it right now.

The Bottom Line

The researchers took a tool that was designed for the West, realized it didn't fit the East, and built a new, smarter tool using local data and routine blood tests. It's a "Seoul-specific" weather forecast that finally tells the truth about the rain in Korea, helping doctors make better decisions without overreacting.

Technical Summary

1. Problem Statement

• Limitations of Existing Models: Current guidelines recommend the Updated Diamond–Forrester (UDF) and CAD Consortium (CAD2) models for estimating the pretest probability (PTP) of Coronary Artery Disease (CAD). However, these models were developed primarily using Western population data.
• Ethnic Discrepancy: When applied to Asian (specifically Korean) populations, these models exhibit suboptimal predictive efficacy. This is attributed to ethnic differences in atherosclerosis patterns, risk factor profiles, and disease prevalence.
• Calibration Issues: Western models tend to overestimate CAD risk in Korean populations due to poor "calibration-in-the-large" (intercept) and calibration slopes, leading to the over-classification of patients into high-risk categories and potentially unnecessary invasive testing.
• Data Gaps: Existing models often rely solely on demographics and symptoms, excluding readily available routine laboratory data (e.g., lipid profiles, HbA1c) that could improve prediction accuracy.

2. Methodology

Study Design & Data Sources:

• Training Dataset: Aggregated data from 4,696 Korean patients with suspected CAD from three randomized controlled trials (CONSERVE, CREDENCE, 3V FFR-FRIENDS) and two retrospective cohorts (PARADIGM registry, Severance CCTA registry).
  • Final Analysis Set: 4,156 patients with complete data (3,396 non-obstructive, 760 obstructive CAD).
  • Reference Standard: Obstructive CAD defined as >50% diameter stenosis via Invasive Coronary Angiography (ICA) or Coronary Computed Tomography Angiography (CCTA).
• External Validation Cohorts:
  1. Cohort 1 (High-Risk Symptomatic): 428 patients from Seoul National University Bundang Hospital undergoing ICA.
  2. Cohort 2 (General Population): 117,294 individuals from the National Health Insurance Service-Health Screening Cohort (NHIS-HEALS). Used a surrogate endpoint (ICD-10 code I20 for clinical angina diagnosis) due to ethical/practical constraints of anatomical testing in asymptomatic populations.

Model Development (K-CAD):

• Algorithm: Ridge-penalized logistic regression (L2 regularization) was employed to prevent overfitting while handling multicollinearity among predictors.
• Predictors: Integrated a broader set of Clinical Risk Factors (CRFs) than UDF/CAD2, including:
  • Demographics: Age (discretized), Sex.
  • Symptoms: Typical/Atypical Angina, Non-cardiac chest pain.
  • Medical History: Hypertension, Diabetes, Dyslipidemia, Smoking.
  • Routine Laboratory Results: Lipid profiles (LDL, HDL, Triglycerides), Creatinine, and Glycated Hemoglobin (HbA1c).
• Feature Engineering:
  • Log-transformation of HDL and Triglycerides.
  • Discretization of Age and HbA1c levels.
  • One-hot encoding for symptom categories.
• Comparison: The new K-CAD model was benchmarked against UDF and CAD2 using continuous Receiver Operating Characteristic (ROC) analysis and Ternary Net Reclassification Improvement (NRI).

3. Key Contributions

• Population-Specific Calibration: Developed the first PTP model specifically calibrated for the Korean population, addressing the "calibration hierarchy" failure of Western models in Asian cohorts.
• Integration of Routine Labs: Successfully demonstrated that incorporating standard, low-cost laboratory results (lipids, HbA1c, creatinine) significantly enhances predictive power over models relying only on history and symptoms.
• Transparency and Reproducibility: Unlike many "black box" machine learning models, K-CAD uses interpretable ridge regression. The authors provided the full model parameters and an online calculator (https://metaeyes.io/med_scores/k_cad) to ensure clinical utility and reproducibility.
• Dual-Track Validation: Validated the model across two distinct populations: a high-risk symptomatic cohort (anatomical confirmation) and a massive nationwide asymptomatic screening cohort (clinical diagnosis).

4. Key Results

Performance Metrics (External Validation Cohort 1 - High Risk):

• Area Under the Curve (AUC):
  • K-CAD: 0.76 (95% CI 0.71–0.80)
  • CAD2: 0.71 (95% CI 0.67–0.76)
  • UDF: 0.68 (95% CI 0.63–0.73)
  • Significance: K-CAD significantly outperformed both UDF ($p < 0.001$) and CAD2 ($p < 0.05$).
• Risk Reclassification (NRI Analysis):
  • K-CAD significantly improved risk stratification by reclassifying 79.9% of non-obstructive patients (who were misclassified as high-risk by UDF) into lower-risk categories.
  • This suggests a reduction in unnecessary downstream invasive testing.
  • Risk Distribution: K-CAD classified 36.4% of patients as low-risk (vs. 1.4% for UDF and 24.8% for CAD2) and 33.6% as high-risk (vs. 85.7% for UDF).

Performance Metrics (External Validation Cohort 2 - Nationwide Screening):

• Using the surrogate endpoint (clinical angina diagnosis), K-CAD (without HbA1c) achieved an AUC of 0.67, outperforming UDF (0.61) and slightly exceeding CAD2 (0.66).

Key Predictors Identified:

• Positive Associations: Male sex, Hypertension, Dyslipidemia, Typical Angina, increasing Age, and higher HbA1c.
• Negative Associations: Higher BMI and higher HDL cholesterol.
• Note: Lower LDL cholesterol was observed in the obstructive group, likely due to confounding by indication (patients on statins).

5. Significance and Implications

• Clinical Decision Support: K-CAD offers a more accurate tool for Korean clinicians to stratify CAD risk, potentially reducing the overuse of invasive angiography in low-to-intermediate risk patients.
• Addressing Ethnic Bias: The study provides strong evidence that Western-derived risk models are not directly transferable to East Asian populations and require local recalibration or re-development.
• Cost-Effectiveness: By leveraging routine blood tests already available in standard checkups, K-CAD improves diagnostic precision without requiring expensive additional imaging or genetic testing.
• Future Directions: The authors suggest that future work should focus on formal recalibration of UDF/CAD2 to local prevalence to isolate discriminative value, and further validation in broader multi-ethnic Asian cohorts.

Limitations Noted:

• Applicability may be limited to Korean/East Asian populations.
• The training set used a mix of ICA and CCTA as reference standards (CCTA can overestimate stenosis).
• The nationwide validation used a clinical diagnosis surrogate rather than anatomical confirmation.
• Competitor models (UDF/CAD2) were not recalibrated to local prevalence in the comparison, which may partially inflate K-CAD's apparent superiority.