📄 medicine

External validation and comparison of PREVENT and SCORE2 atherosclerotic cardiovascular risk scores in the MASHAD cohort study

This study externally validated the PREVENT and SCORE2 cardiovascular risk models in the Iranian MASHAD cohort, finding that while both models demonstrated acceptable discrimination, they significantly underestimated absolute risk and required local recalibration to provide accurate predictions for this Middle Eastern population.

Original authors: Ali Tajik, Hamidreza Rahimi, Sara Saffar Soflaei, Mohsen Moohebati, Ashkan Bahrami, Gordon A. Ferns, Habibollah Esmaily, Majid Ghayour-Mobarhan, Hedieh Alimi

Published 2026-06-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ali Tajik, Hamidreza Rahimi, Sara Saffar Soflaei, Mohsen Moohebati, Ashkan Bahrami, Gordon A. Ferns, Habibollah Esmaily, Majid Ghayour-Mobarhan, Hedieh Alimi

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two very popular weather forecasters, PREVENT and SCORE2. These forecasters are famous in Europe and the United States for predicting the chance of a "storm" (a heart attack or stroke) hitting a person over the next 10 years. They use a standard recipe: they look at your age, blood pressure, cholesterol, and whether you smoke to give you a percentage chance of getting hit by the storm.

The researchers in this study wanted to see if these famous forecasters would work correctly in Mashhad, Iran. They treated the local population like a new, untested weather zone.

Here is what they found, explained simply:

1. The Problem: The "Foreign" Forecasters Were Too Optimistic

When the researchers used the original European and American formulas on the Iranian people, the forecasters made a big mistake: they were too optimistic.

The Analogy: Imagine a weatherman in London telling a farmer in a desert, "There is a 1% chance of rain today." But in the desert, it actually rains 30% of the time. The farmer would get soaked because the forecast was wrong for his specific location.
The Reality: Both PREVENT and SCORE2 told the Iranian people their risk of a heart event was very low. However, when the researchers looked at the actual data over 10 years, many more people had heart events than the models predicted. The models were "underestimating" the danger.

2. The Solution: "Local Tuning" (Recalibration)

The researchers didn't throw the models away. Instead, they performed a "local tune-up."

The Analogy: Think of the models as a radio station. The music (the way they rank people from low risk to high risk) was good, but the volume was too quiet. They didn't need to change the song; they just needed to turn up the volume knob to match the local audience.
The Action: They adjusted the math slightly to match the actual number of heart events happening in Mashhad. They kept the "ranking" system the same but shifted the numbers so the percentages reflected reality.

3. The Results: Who Was Better?

After the "tune-up," both models worked much better, but one had a slight edge.

The Ranking Ability (Discrimination): Both models were good at sorting people. They could tell who was more likely to have an event than someone else. PREVENT was slightly better at this sorting than SCORE2, like a slightly sharper pair of glasses.
The Accuracy (Calibration): Before the tune-up, the models were wrong about the amount of risk. After the tune-up, the predicted risk matched the actual risk almost perfectly.
- PREVENT was a bit more accurate overall. This is likely because PREVENT looks at more details, like kidney function and diabetes, which are very common in this specific group of people.
- SCORE2 is a simpler model (it doesn't look at diabetes), but it still worked well once it was tuned to the local "volume."

4. Why This Matters (The "Net Benefit")

The study used a concept called "Net Benefit" to see if the tuned models helped doctors make better decisions.

The Analogy: Imagine a security guard checking bags.
- Original Model: The guard was too relaxed, letting many dangerous bags through (missing high-risk people).
- Tuned Model: The guard became more alert. He stopped more dangerous bags (catching more high-risk people), but he also stopped a few harmless bags by mistake (lowering "specificity").
The Finding: The researchers found that the "tuned" models were worth it. Even though they flagged a few extra people who turned out to be safe, they successfully caught many more people who were actually at risk. In a health context, it is better to catch a potential heart attack early than to miss it.

Summary

The study concludes that you cannot simply copy-paste a risk calculator from Europe or America and expect it to work perfectly in Iran. The "recipe" for risk is different in different places.

However, the good news is that these tools can work if you "calibrate" them to the local population. Once the researchers adjusted the numbers to fit the reality of Mashhad, both PREVENT and SCORE2 became reliable tools for predicting heart risks, with PREVENT having a slight advantage because it considers more health factors.

Technical Summary: External Validation and Comparison of PREVENT and SCORE2 in the MASHAD Cohort

Problem Statement
Contemporary atherosclerotic cardiovascular disease (ASCVD) risk prediction models, such as the American Heart Association's PREVENT and the European Society of Cardiology's SCORE2, require external validation before implementation in populations distinct from their derivation cohorts. Prediction models often exhibit poor calibration when applied to populations with differing baseline event rates, risk-factor distributions, or treatment access, even if they retain acceptable discrimination. While previous studies have validated these tools in other regions (e.g., the Tehran Lipid and Glucose Study), there is a critical gap in evidence regarding their performance in northeastern Iran. Specifically, it remains unclear whether PREVENT and SCORE2 provide reliable absolute risk estimation for the MASHAD cohort, where cardiometabolic burdens, smoking patterns, and healthcare access differ from Western cohorts. Uncalibrated tools risk under-identifying high-risk individuals or exposing low-risk individuals to unnecessary treatment.

Methodology
This prospective external validation study utilized data from the Mashhad Stroke and Heart Atherosclerotic Disorder (MASHAD) cohort, following 9,069 adults (aged 30–65) over a 10-year period (2010–2020).

Cohort Selection: The full eligible cohort ( $n=9,069$ ) was used for PREVENT validation. For SCORE2, which is designed for non-diabetic adults, participants with baseline diabetes ( $n=1,233$ ) were excluded, resulting in an analytical sample of 7,836 non-diabetic participants.
Outcome Definition: The primary outcome was a composite of fatal or non-fatal myocardial infarction and fatal or non-fatal stroke, adjudicated by an independent committee using ICD-10 codes.
Model Application: Both models were applied using their published equations and coefficients without refitting. PREVENT utilized its logistic approximation of the competing-risks model, while SCORE2 utilized its standard 10-year risk equations.
Recalibration Strategy: To correct for miscalibration while preserving the original model structure, local recalibration was performed:
- PREVENT: Sex-specific intercept updating (logistic recalibration with the original linear predictor as an offset).
- SCORE2: Sex-specific baseline-hazard updating (Cox proportional hazards regression with the original linear predictor as an offset).
Performance Metrics: Models were evaluated using discrimination (C-statistics), calibration (calibration slopes, observed-to-expected [O/E] ratios, Brier scores), clinical performance at predefined thresholds (sensitivity, specificity, net benefit), categorical net reclassification improvement (NRI), Kaplan–Meier survival analysis, and decision curve analysis (DCA).

Key Results

Event Rates: Within the 10-year horizon, 678 ASCVD events occurred in the PREVENT sample and 470 in the SCORE2 sample.
Discrimination: Both models demonstrated acceptable ability to rank individuals by risk. PREVENT showed modestly superior discrimination compared to SCORE2.
- Men: PREVENT C-statistic = 0.714 vs. SCORE2 = 0.687.
- Women: PREVENT C-statistic = 0.731 vs. SCORE2 = 0.704.
- Recalibration did not alter C-statistics, as the original predictor coefficients were preserved.
Calibration: Both original models substantially underestimated absolute risk.
- Original O/E Ratios: PREVENT (2.904 overall; 3.004 men, 2.817 women) and SCORE2 (4.002 overall; 3.393 men, 4.902 women).
- Recalibration Impact: Sex-specific recalibration corrected calibration-in-the-large, bringing O/E ratios to approximately 1.00 for both models. Calibration slopes remained below 1.0, indicating imperfect spread of predicted risks, but Brier scores improved significantly post-recalibration.
Clinical Performance and Reclassification:
- Recalibration significantly increased sensitivity and the proportion of participants classified into higher-risk categories, though at the cost of reduced specificity.
- Example (PREVENT, Men, 5% threshold): Sensitivity increased from 46.2% to 90.1%; the proportion classified as high-risk rose from 21.3% to 67.0%. Net benefit increased from 0.0340 to 0.0529.
- Net Reclassification Improvement (NRI): Positive total NRI was observed for both models (PREVENT: 0.296; SCORE2: 0.236), driven largely by the upward reclassification of event cases (Event NRI > 0.80).
Decision Curve Analysis: Recalibrated models provided greater net benefit than original models across lower and intermediate threshold probabilities, aligning better with clinical decision-making scenarios.

Significance and Claims
The study concludes that while PREVENT and SCORE2 preserve acceptable ASCVD risk-ranking ability in this Iranian cohort, their original absolute risk estimates are inaccurate due to significant underestimation. The authors claim that local recalibration is essential for accurate absolute risk estimation in this population.

Comparative Performance: PREVENT demonstrated modestly better discrimination and calibration slopes than SCORE2, likely due to its inclusion of cardiometabolic and kidney-related predictors (e.g., diabetes, eGFR, medication use) which are highly relevant in the MASHAD population.
Clinical Utility: The primary contribution of this work is demonstrating that recalibration improves clinically meaningful performance metrics—specifically sensitivity, risk stratification, and decision-analytic net benefit—without altering the underlying prognostic signal.
Implementation Guidance: The authors assert that locally calibrated risk tools are necessary for ASCVD prevention in underrepresented regional populations. They recommend that PREVENT may be preferable when comprehensive data are available, while SCORE2 remains a viable, simpler option for non-diabetic adults if locally recalibrated.
Caveats: The authors emphasize that these recalibrated equations should not be directly adopted for national implementation without further independent validation in other Iranian and Middle Eastern populations, noting that the study is limited to a regional urban cohort and that recalibration trades specificity for sensitivity.