Comparative Validation of ASCVD, Framingham Risk Score, Globorisk, and PREVENT for 10-Year Cardiovascular Risk Prediction in the UK Biobank

Technical Summary: Comparative Validation of ASCVD, Framingham Risk Score, Globorisk, and PREVENT in the UK Biobank

Problem Statement
Cardiovascular disease (CVD) remains the leading cause of global mortality, necessitating accurate risk prediction models for effective primary prevention. While several models exist—including the Atherosclerotic Cardiovascular Disease (ASCVD) pooled cohort equations, the Framingham Risk Score (FRS), Globorisk, and the recently introduced PREVENT model—their performance varies significantly across different populations. Traditional models often rely on static baseline variables and may not generalize well to contemporary, diverse cohorts. Specifically, discrepancies in predictive performance across demographic groups (such as women and older adults) and the lack of a systematic, four-way comparison including the newest PREVENT model within a large UK cohort represent a gap in current evidence. This study addresses the need to evaluate the discrimination, calibration, and clinical utility of these four models within the same contemporary UK population to inform clinical decision-making.

Methodology
The study utilized data from the UK Biobank, a large population-based cohort. The analytical cohort consisted of 183,124 participants aged 40–69 years who were free of prior cardiovascular disease at baseline. Participants were followed for up to 10 years to identify incident cardiovascular events (fatal and non-fatal myocardial infarction and stroke), defined by ICD-10 codes.

Four risk prediction models were applied using baseline clinical variables:

1. ASCVD (2013 ACC/AHA): Uses sex- and race-specific equations including age, sex, systolic blood pressure, total and HDL cholesterol, diabetes, smoking, and blood pressure treatment status.
2. FRS (2008 General CVD): Uses sex-specific Weibull survival models with similar variables to ASCVD but without race adjustment.
3. Globorisk: Applied with UK-specific calibration coefficients to adjust for national disease incidence.
4. PREVENT (2023 AHA): Incorporates additional metabolic parameters, including kidney function (eGFR) and statin use.

Model performance was rigorously evaluated using:

• Discrimination: Receiver Operating Characteristic (ROC) analysis and Area Under the Curve (AUC) with 95% confidence intervals.
• Calibration: Comparison of mean predicted probabilities against observed event rates across risk deciles.
• Classification Metrics: Sensitivity, specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV) at optimal thresholds determined by the Youden Index.
• Clinical Utility: Decision Curve Analysis (DCA) to assess net clinical benefit across decision thresholds (1%–50%).
• Reclassification: Net Reclassification Improvement (NRI) at a 7.5% threshold.
• Stratification: Analyses were stratified by sex and age group (mid-aged ≤59 years vs. older adults >59 years).

Key Results

• Overall Performance: Over the 10-year follow-up, 10,560 incident events (5.8% event rate) were recorded. ASCVD demonstrated the best overall discrimination (AUC = 0.717), closely followed by FRS (AUC = 0.715). PREVENT showed intermediate performance (AUC = 0.668), while Globorisk exhibited the lowest discrimination (AUC = 0.605).
• Calibration: ASCVD showed the best calibration across risk strata. FRS systematically underestimated risk, particularly at moderate to high levels. PREVENT overestimated risk at intermediate levels, while Globorisk diverged progressively at higher risk thresholds.
• Sex-Stratified Findings: Among women, ASCVD (AUC = 0.709) and PREVENT (AUC = 0.701) performed comparably, with PREVENT showing higher sensitivity (66.5% vs. 38.2% for ASCVD). Among men, all models showed lower AUCs (0.647–0.677), with FRS exhibiting the highest sensitivity but lowest specificity (32.4%).
• Age-Stratified Findings: Discriminative performance declined significantly in older adults (>59 years) across all models. ASCVD and FRS dropped to AUCs of ~0.65, while Globorisk approached near-random discrimination (AUC = 0.527).
• Clinical Utility (DCA): ASCVD provided the highest net clinical benefit across the widest range of clinically relevant thresholds (approximately 5–20%). FRS showed negative net benefit at intermediate thresholds (8–25%), suggesting potential net harm from its application in this range.
• Reclassification (NRI): No single model demonstrated a consistent positive overall NRI over the others at the 7.5% threshold. However, PREVENT showed a positive Nonevent NRI relative to FRS, indicating better reclassification of low-risk individuals downward.

Key Contributions
This study provides a systematic, four-way comparative validation of major CVD risk models within a single, large, contemporary UK cohort. Its primary contributions include:

1. Direct Comparison: It is the first study to simultaneously compare ASCVD, FRS, Globorisk, and the newly introduced PREVENT model using identical data and outcome definitions.
2. Comprehensive Evaluation: The analysis extends beyond standard discrimination metrics (AUC) to include Decision Curve Analysis and Net Reclassification Improvement, offering a more nuanced view of clinical utility and reclassification capabilities.
3. Subgroup Insights: The study highlights critical performance disparities, specifically the decline in model accuracy for older adults and the varying performance profiles between sexes, challenging the "one-size-fits-all" application of these models.
4. Evidence for Model Selection: The findings provide empirical evidence supporting the continued use of ASCVD and FRS for 10-year risk stratification in the UK, while identifying specific limitations of Globorisk and the intermediate performance of PREVENT in this specific population.

Significance and Claims
The authors claim that ASCVD and FRS remain the most reliable tools for 10-year CVD risk prediction in the contemporary UK cohort, with ASCVD demonstrating superior calibration and net clinical benefit. The study underscores that while PREVENT shows promise, particularly for identifying high-risk women and reclassifying low-risk individuals, it does not yet outperform established models in overall discrimination within this population.

Crucially, the paper emphasizes that the substantial decline in model performance for adults over 59 years highlights the limitations of static, baseline-variable models in older populations. The authors conclude that future risk frameworks must incorporate dynamic risk factors, longitudinal data, and population-specific calibration to improve accuracy across the full age spectrum. The study advocates for the inclusion of Decision Curve Analysis as a standard component in the reporting of clinical prediction models to better assess real-world clinical utility.