The fundamental problem of risk prediction for individuals: health AI, uncertainty, and personalized medicine

Technical Summary: The Fundamental Problem of Risk Prediction for Individuals

Problem Statement
Clinical prediction models, increasingly driven by artificial intelligence (AI), are routinely evaluated based on their performance at the population level (e.g., discrimination, calibration, and clinical utility). However, clinical decisions are made for individuals. A critical gap exists in the quantification and understanding of uncertainty regarding risk estimates for single patients. While traditional statistical approaches focus on estimation uncertainty (variability due to sample size), they often overlook model uncertainty (variability arising from modeling choices) and applicability uncertainty (variability due to differences in measurement procedures and population characteristics). The paper argues that individual risk estimates are far more uncertain than commonly assumed, and that relying on a single model from a single study creates a false sense of precision.

Methodology
To empirically illustrate the magnitude of these uncertainties, the authors conducted a case study on predicting the risk of malignant ovarian cancer in patients presenting with an ovarian tumor.

• Data Sources: The study utilized real-world retrospective data from the University Hospitals Leuven (1,122 patients, 1999–2015) and synthetic data generated to explore sample size effects. A fixed test set of 100 real patients from Leuven was used to evaluate variability in risk estimates.
• Modeling Framework: The authors decomposed epistemic uncertainty into three categories:
  1. Estimation Uncertainty: Variability in parameter estimates due to different training samples.
  2. Model Uncertainty: Variability due to choices in model class (logistic regression, random forests, XGBoost), handling of continuous variables (splines vs. others), variable selection, penalization, and hyperparameter tuning.
  3. Applicability Uncertainty: Variability due to data definitions (e.g., tumor size measurement), missing data handling, and population differences (data from Leuven, Rome, and Malmö).
• Experimental Design: The authors trained a massive ensemble of 59,400 models. This was achieved by:
  • Creating 594 distinct modeling scenarios (variations in the three uncertainty categories).
  • Repeating each scenario with 100 randomly drawn training samples (bootstrapping) to capture estimation uncertainty.
  • Testing sample sizes of $N=400$, $N=2,000$, and $N=10,000$.
• Metrics:
  • Population Level: Area Under the Receiver Operating Characteristic Curve (AUROC), Estimated Calibration Index (ECI), and Relative Utility (RU).
  • Individual Level: The 95% range of risk estimates (difference between the 97.5th and 2.5th percentiles) and Decision Uncertainty (DU). DU measures the proportion of models that would yield a different clinical decision (e.g., surgery recommendation) based on a threshold of 0.1 risk.

Key Results

• Dominance of Non-Estimation Uncertainty: While estimation uncertainty decreased significantly as training sample size increased (the mean 95% range dropped from 0.22 at $N=400$ to 0.04 at $N=10,000$), model and applicability uncertainty remained large and largely unaffected by sample size.
• Individual Risk Variability: Even with a large training set of 10,000 patients, the average 95% range of risk estimates for a single patient across all 59,400 models was 0.39 (ranging from 0.05 to 0.99). For some patients, the range covered nearly the entire probability spectrum (0 to 1).
• Decision Uncertainty: When considering only estimation uncertainty, increasing the sample size to 10,000 reduced decision uncertainty (models suggesting a different action than the majority) to 1%. However, when model and applicability uncertainties were included, decision uncertainty remained high at 11%, even with $N=10,000$.
• Population vs. Individual Performance: The models maintained high population-level performance (AUROC 0.92–0.93, good calibration, positive relative utility) across all scenarios. This demonstrates that a model can be highly useful for guiding population-level strategies while providing highly uncertain and potentially conflicting risk estimates for individuals.

Key Contributions

1. Framework for Uncertainty: The paper introduces a structured framework distinguishing between estimation, model, and applicability uncertainty, highlighting that traditional evaluations often hide the latter two.
2. Empirical Evidence: It provides empirical evidence that simply increasing dataset size does not resolve the uncertainty in individual risk predictions, as model specification and data/population differences dominate the error variance.
3. Quantification of Decision Uncertainty: The study quantifies how often different valid modeling choices lead to contradictory clinical decisions for the same patient, challenging the notion of a single "true" risk estimate.

Significance and Claims
The authors conclude that individual risk estimates are inherently uncertain, often to a degree that exceeds what is suggested by classic confidence intervals or effective sample size metrics. They argue that this uncertainty does not invalidate the clinical utility of prediction models at the population level but necessitates a shift in how these tools are applied in practice:

• AI as Informant, Not Dictator: Predictive algorithms should inform, not dictate, care. The uncertainty implies that AI outputs should not be blindly trusted as definitive answers for individual patients.
• Shared Decision-Making: True personalized medicine requires human interaction. Clinicians must acknowledge the uncertainty in model outputs and integrate these estimates into a shared decision-making process with the patient, rather than relying on a single algorithmic output.
• Modesty in Personalization: The paper urges humility regarding the promises of personalized medicine. While AI can provide group-level estimations based on training data, the "personalized" nature of the risk estimate is limited by epistemic uncertainty. The value lies in the shared decision-making process supported by these estimates, not in the estimates themselves being perfectly precise.