A Statistical Framework for Data-Driven Discovery of Differential Performance in Clinical Risk Prediction Models

Technical Summary: A Statistical Framework for Data-Driven Discovery of Differential Performance in Clinical Risk Prediction Models

Problem Definition
The paper addresses the challenge of detecting and characterizing differential performance in clinical risk assessment instruments (RAIs) across population subgroups. While predictive models based on AI and machine learning are increasingly used for clinical decision support, they may exhibit disparities in performance across groups defined by race, age, sex, or other intersecting characteristics. Existing fairness assessment methods typically require the end-user to pre-specify the demographic groups for evaluation. This approach risks overlooking disparities in less obvious or complex subgroups defined by higher-order interactions of covariates (e.g., specific combinations of age, sex, and clinical history). Furthermore, standard fairness criteria (e.g., equalized odds, sufficiency) are often mutually incompatible, and a model appearing fair under one metric or grouping may still exhibit disparities under others. The authors define "fairness" in this context as the absence of statistical evidence of differential model performance, specifically focusing on miscalibration (discrepancies between predicted risk and observed outcomes).

Methodology: The Unfairness Tree (utree)
The authors propose the unfairness tree (utree), a data-driven recursive partitioning framework designed to identify subgroups with heterogeneous predictive discrepancies without pre-specifying group definitions.

• Core Algorithm: The utree adapts the conditional inference tree (ctree) framework. Unlike standard CART algorithms, ctrees separate variable selection from split-point selection to avoid bias toward continuous variables.
• Discrepancy Statistic: At each node, the algorithm tests for differential performance using a statistic derived from the Kolmogorov–Smirnov (KS) test. This statistic compares two weighted empirical cumulative distribution functions (CDFs) of a candidate covariate $Z_j$: one weighted by observed outcomes (or potential outcomes) and the other by predicted risks.
  • The test statistic is $T_j = \sup_z | \hat{F}_j(z; w_Y) - \hat{F}_j(z; w_{\hat{s}}) |$, where $w_Y$ and $w_{\hat{s}}$ are weights defined by outcomes and predictions, respectively.
  • Under the null hypothesis of conditional calibration, the weighted distributions are identical. Rejection indicates that the distribution of outcomes differs from predictions across values of $Z_j$.
• Counterfactual Framework: To address treatment-induced bias (where model predictions influence treatment decisions, thereby altering observed outcomes), the method employs a counterfactual evaluation framework. It estimates potential outcomes under a reference condition ($Y^0$) using doubly robust estimators (combining outcome regression and propensity score modeling). The discrepancy is then calculated between predicted risks and these counterfactual outcomes.
• Recursive Partitioning: The algorithm proceeds in two steps per node:
  1. Variable Selection: Computes test statistics for all candidate covariates and selects the one with the smallest empirical p-value (adjusted for multiplicity).
  2. Split-Point Selection: Identifies the split point for the selected variable that maximizes the test statistic magnitude.
• Flexibility: The framework can be extended to other performance metrics (e.g., true positive rate, precision) by modifying the weighting functions $w(a)$ and $w(b)$ in the KS statistic formulation.

Key Contributions

1. Data-Driven Subgroup Discovery: The utree eliminates the need for pre-specified subgroups, allowing for the discovery of disparities in complex, intersecting populations defined by multiple covariates.
2. Counterfactual Calibration: It integrates causal inference methods to distinguish between true model miscalibration and treatment-induced bias, a critical distinction in clinical settings where models inform treatment.
3. Statistical Validity: The method utilizes conditional inference trees and permutation-based p-values to control Type I error rates despite the adaptive nature of recursive partitioning.
4. Interpretability: The resulting tree structure provides an interpretable map of how combinations of clinical characteristics relate to differential model performance.

Results

• Simulation Studies:

  • Type I Error Control: Under the null scenario (no differential performance), the probability of selecting a split remained close to the nominal level (e.g., 0.048 at $N=10^3$), demonstrating effective error control.
  • Power and Recovery: The method showed increasing power to detect discrepancies as sample size and severity increased. It successfully recovered the true variables and interactions driving unfairness, with recovery rates rising from 0.06 to 0.92 under large miscalibration scenarios at $N=10^3$.
  • Estimation Accuracy: Leaf-wise estimates of discrepancy exhibited small bias that decreased with sample size, indicating approximate unbiasedness.
  • Computational Efficiency: Fitting times scaled predictably with sample size, remaining feasible even for $N=10^5$ (under 4 minutes).

• Real Data Application (GUSTO-I Trial):

  • The framework was applied to six mortality risk models (including logistic regression, GAM, and random forest) using data from 30,510 patients with acute myocardial infarction.
  • Consistent Drivers: Across models and metrics (calibration, true positive rate, precision), age, sex, systolic blood pressure, and Killip class were consistently identified as key variables defining subgroups with differential performance.
  • Metric Dependence: The specific subgroup structures identified varied depending on the chosen performance metric. For instance, calibration-based trees identified subgroups with significant underprediction (e.g., non-U.S. patients with prolonged time to treatment relief), while TPR and precision trees highlighted different structural patterns.
  • Validation: Out-of-sample validation on a held-out test set confirmed that many subgroup performance patterns were reproducible, particularly when a minimum terminal leaf size of $n \ge 50$ was enforced.

Significance and Claims
The paper claims that the utree provides a principled, statistically valid, and interpretable approach to detecting subgroup-specific miscalibration in clinical risk models. By moving beyond pre-defined demographic categories, it addresses a key limitation in current AI fairness assessments, which may miss disparities in complex, intersecting populations. The authors emphasize that the method identifies differential model performance rather than definitive algorithmic unfairness, noting that the latter depends on broader ethical and clinical considerations. The framework is presented as a tool for exploratory discovery and characterization of performance disparities, with the caveat that counterfactual estimates rely on standard causal assumptions (positivity, consistency, no unmeasured confounding) and are most compelling in randomized settings or where these assumptions are plausible. The authors acknowledge that while the method controls error rates at each node, formal theoretical guarantees for the stability of the final tree structure and selective inference for the discovered subgroups remain open questions for future research.