Trade-offs in Cardiovascular Risk Prediction Using Race and Social Determinants of Health

This study demonstrates that while removing race from cardiovascular risk prediction models improves statistical parity metrics, it inadvertently redistributes clinical harms by causing systematic underprediction and concentrated overtreatment or undertreatment among Black individuals, highlighting the need for comprehensive empirical evaluation beyond population-average performance before adopting race-neutral tools.

The Gist

Imagine you are a doctor trying to decide who needs a "preventive shield" (like a daily statin pill) to stop a heart attack or stroke before it happens. You have a Risk Calculator, a digital tool that looks at a patient's data and gives them a score. If the score is above a certain line (7.5%), the doctor says, "You need the shield."

This study asks a big, complicated question: How do we build the best calculator?

Specifically, the researchers are debating three different ways to fill in the calculator's formula:

1. The Old Way: Include "Race" as a direct ingredient.
2. The Social Way: Remove "Race" but add "Social Determinants of Health" (SDoH) instead. This means adding details about a person's life: Do they have money? Do they have food? Do they feel safe? Can they afford a doctor?
3. The Pure Way: Remove "Race" and remove the social details. Just use medical numbers like blood pressure and cholesterol.

The researchers tested these three calculators on a group of 3,000 people over 10 years to see which one made the fairest and most helpful decisions.

Here is what they found, explained with some simple analogies:

1. The "Overall Score" Was a Tie

If you just looked at the overall accuracy (like a batting average in baseball), all three calculators performed almost exactly the same. They were all about 76% accurate at predicting who would get heart disease.

• The Trap: If you only looked at this "batting average," you might think, "Great! They are all the same, so let's pick the one that feels most modern." But the study shows that the average hides the real story.

2. The "Social Way" (SDoH) = The Over-Generous Shopper

When the researchers swapped "Race" for "Social Factors" (Model 2), the calculator started acting like a very generous shopkeeper who is afraid of missing a sale.

• What happened: It started flagging more people as "high risk" to make sure they didn't miss anyone.
• The Good: It caught a few more Black patients who were actually at risk (fewer missed cases).
• The Bad: It also flagged a lot of healthy Black patients who didn't need the pill. It was like giving a life jacket to people who are already standing on a boat, not in the water. This is called overtreatment.
• The Metaphor: It's like a smoke detector that is so sensitive it goes off when you just toast a piece of bread. It's better at catching real fires, but it creates a lot of false alarms that annoy people and waste resources.

3. The "Pure Way" (No Race, No Social Data) = The Strict Gatekeeper

When they removed everything except medical numbers (Model 3), the calculator acted like a strict gatekeeper who only lets in people with a perfect ticket.

• What happened: It became much stricter. It stopped flagging people unless their medical numbers were screaming "Danger!"
• The Good: It stopped the false alarms. Fewer healthy people got the pill unnecessarily.
• The Bad: It missed the people who actually needed help. Because it ignored the social context (like stress, poverty, or discrimination) that makes heart disease more likely for Black patients, it underestimated their risk.
• The Result: In this scenario, four Black patients who eventually had a heart attack were told, "You are low risk, you don't need the pill," and they didn't get the treatment that could have saved them. This is undertreatment.

4. The "Fairness" Illusion

The study found that when you remove "Race" from the equation, the numbers look fairer. The gap between how often Black and White people get flagged shrinks.

• The Analogy: Imagine two runners on a track. One is running on a flat, smooth track (White patients). The other is running on a muddy, uphill path (Black patients).
  • If you just look at the finish line times, the runner on the mud looks slower.
  • If you remove the "mud" from the equation (by ignoring the social factors), the gap in times looks smaller. But you haven't actually helped the runner on the mud; you've just stopped acknowledging the obstacle they are facing.
  • The study shows that making the "numbers" look equal doesn't mean the outcome is actually fair.

The Big Takeaway

The researchers conclude that there is no "perfect" calculator.

• Including Race helps catch more high-risk people but feels uncomfortable because it treats race as a biological fact rather than a social reality.
• Using Social Data tries to be more precise about why people are at risk, but in this study, it led to too many healthy people getting unnecessary treatment.
• Removing Everything makes the numbers look clean and equal, but it accidentally leaves the most vulnerable people behind.

The Final Lesson:
You can't just pick a model based on which one looks the "fairest" on a spreadsheet. You have to look at the real-world consequences.

• Do you want to risk giving a pill to a healthy person (overtreatment)?
• Or do you want to risk not giving a pill to a sick person (undertreatment)?

The study argues that before hospitals switch to "race-neutral" tools, they need to understand that changing the ingredients changes who gets the medicine. It's not just a math problem; it's a life-and-death trade-off that requires careful, human judgment.

Technical Summary

1. Problem Statement

Health systems are increasingly transitioning from race-inclusive cardiovascular risk prediction models to race-neutral alternatives. This shift is driven by ethical concerns that race-based models may reinforce structural inequities and normalize race-based clinical decision-making. A common proposed solution is to either:

1. Substitute race with Social Determinants of Health (SDoH) to capture the structural conditions driving disparities.
2. Exclude race entirely to create strictly race-neutral models.

However, the empirical consequences of these design choices on clinical utility (specifically, who receives preventive treatment) remain poorly understood. While previous studies have focused on overall predictive performance (e.g., AUC) and calibration parity, they often fail to quantify how these choices redistribute clinical benefits and harms (overtreatment vs. undertreatment) at the specific decision thresholds used in practice. This study addresses the gap by evaluating how model design choices reshape treatment eligibility for Black and White populations, particularly given that Black individuals bear a disproportionate cardiovascular burden.

2. Methodology

Study Design & Data

• Cohort: Retrospective cohort study using data from the Coronary Artery Risk Development in Young Adults (CARDIA) study.
• Sample: 3,241 Black and White adults (46% Black) without known cardiovascular disease (CVD) at baseline (Year 25, 2010).
• Outcome: 10-year incident CVD (myocardial infarction, stroke, heart failure hospitalization, or coronary revascularization) measured in 2021.
• Follow-up: ~10 years.

Model Development
Three distinct prediction models were developed using LASSO-regularized logistic regression with 10-fold cross-validation and upsampling to address class imbalance (6.9% CVD incidence). The models differed only in their predictor sets:

• Model 1 (Race): Clinical predictors + Race.
• Model 2 (SDoH): Clinical predictors + Detailed SDoH measures (replacing race). SDoH included socioeconomic status, housing, food insecurity, discrimination, etc.
• Model 3 (Clinical Only): Clinical predictors only (excluding both race and SDoH).

Evaluation Framework
The study employed an integrated evaluation approach assessing four dimensions:

1. Predictive Performance: AUC, Log Loss, Sensitivity, Specificity.
2. Calibration: Assessment of whether predicted probabilities match observed risks overall and by race.
3. Fairness Metrics: Seven group-based metrics (e.g., demographic parity, equal opportunity, disparate impact) to measure relative differences in error rates between Black and White participants.
4. Realized Clinical Utility: An adapted utility framework quantifying overtreatment (treating those who won't develop CVD) and undertreatment (failing to treat those who will) at the ACC/AHA 7.5% risk threshold. This threshold is critical as it triggers statin therapy recommendations.

3. Key Contributions

• Holistic Evaluation: Moves beyond standard performance metrics (AUC) to analyze the distribution of clinical harm and benefit across racial groups.
• Mechanism of "Parity": Demonstrates that improvements in fairness metrics (parity) when removing race are often mechanical (narrowing gaps by constraining the model from stratifying by group) rather than indicative of genuinely improved equitable performance.
• SDoH Limitations: Provides empirical evidence that even with rich, longitudinal SDoH data, substituting race does not fully recover the sensitivity achieved by race-inclusive models for Black populations.
• Utility-Centric Analysis: Quantifies the specific number of individuals who would be misclassified (untreated or overtreated) under different model designs, revealing concentrated harms invisible in population averages.

4. Key Results

Overall Performance

• AUC: All three models showed similar overall predictive performance (AUC range: 0.762 – 0.768).
• Calibration: Model 2 (SDoH) showed systematic underprediction of absolute risk across all groups. Model 3 (Clinical Only) showed modest overprediction among Black participants.

Impact on Clinical Decisions (The 7.5% Threshold)

• Model 2 (SDoH vs. Race):
  • Effect: Increased sensitivity overall but significantly increased false positives (overtreatment).
  • Black Subgroup: While undertreatment decreased slightly, there was a substantial increase in overtreatment concentrated among Black participants.
  • Trade-off: Gains in identifying true cases came at the cost of treating many more individuals who would not develop CVD.
• Model 3 (Clinical Only vs. Race):
  • Effect: Reduced false positives but decreased sensitivity.
  • Black Subgroup: Generated four new cases of untreated CVD (undertreatment) with no offsetting reduction in undertreatment elsewhere.
  • Trade-off: Removing race entirely led to missed diagnoses for Black patients who would have been identified by the race-inclusive model.

Fairness Metrics

• Parity metrics (e.g., disparate impact) generally improved as race was removed or replaced.
• However, the authors argue this improvement is partly an artifact of the model's inability to stratify by race, rather than a true reduction in inequitable error distribution. Residual disparities persisted in all models.

Predictor Importance

• Traditional clinical factors (kidney function, blood pressure, lipids, diabetes) remained the dominant predictors.
• Including race or SDoH did not alter the ranking of core clinical predictors but shifted the risk estimates enough to change treatment eligibility near the threshold.

5. Significance and Conclusion

Clinical Implications
The study reveals that no single model simultaneously optimizes accuracy, calibration, fairness, and clinical outcomes.

• Race-Inclusive Models: Identify more high-risk Black individuals but may result in higher false-positive rates.
• SDoH Substitution: Improves some parity metrics but leads to systematic underprediction and concentrated overtreatment among Black patients.
• Race-Neutral Models: Improve parity metrics mechanically but increase the risk of undertreatment for Black patients, potentially missing preventable CVD events.

Policy and Research Impact

• Beyond Conceptual Arguments: The case for removing race has largely rested on conceptual grounds. This study provides the necessary empirical evidence showing that race removal or substitution carries tangible clinical trade-offs.
• Need for Integrated Evaluation: Health systems cannot rely solely on AUC or calibration parity. Decisions on model design must consider the realized utility and the specific distribution of harms (overtreatment vs. undertreatment) across racial groups.
• Data Gaps: The finding that SDoH did not fully substitute for race suggests that current SDoH measures may not fully capture the cumulative structural disadvantage that race proxies in clinical settings.

Conclusion
The authors conclude that while the conceptual argument for race-neutral models is strong, the empirical reality involves significant trade-offs. Health systems must engage in holistic, multi-dimensional evaluations before adopting new risk tools to ensure they do not inadvertently harm the populations they aim to protect.