📊 statistics

Towards Fair Predictions: Group Conditional Concordance Index to Quantify Fairness in Time-to-Event Prognostication

This paper introduces the group-conditional Concordance Index (xCI), a novel fairness metric for time-to-event analyses that extends Harrell's CI to quantify and detect demographic biases in survival models through within-group and cross-group ranking accuracy, validated via theoretical proofs and real-world cardiovascular case studies.

Original authors: Haoyuan Wang, Riddhiman Bhattacharya, Richardo Henao, Daniel Wojdyla, Chuan Hong, Matthew Engelhard

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoyuan Wang, Riddhiman Bhattacharya, Richardo Henao, Daniel Wojdyla, Chuan Hong, Matthew Engelhard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to decide which patients need the most urgent care first. You have a computer program (a predictive model) that gives every patient a "risk score" to help you sort them. The goal is to make sure the sickest people get help first, no matter who they are.

However, there's a problem: sometimes these programs are biased. They might be great at spotting sick people in one group (say, Group A) but terrible at spotting sick people in another group (Group B). Or, they might be good at sorting people within Group A, but when they compare a person from Group A against a person from Group B, they consistently get the order wrong.

This paper introduces a new tool called the Group Conditional Concordance Index (xCI) to catch these hidden biases. Here is how it works, explained simply:

1. The Problem with the Old Ruler

Traditionally, doctors use a tool called the Concordance Index (CI) to check if a model is working. Think of the CI as a single grade for the whole class. It asks: "Out of all the pairs of patients we looked at, how often did the model correctly guess who was sicker?"

The problem is that this single grade can hide unfairness.

The Analogy: Imagine a teacher grading a math test. If the teacher only looks at the average score of the whole class, they might miss the fact that the teacher gave an easy test to Group A and a impossible test to Group B. The average might look "okay," but the experience for Group B was terrible.
The Paper's Point: The old CI is like that average. It mixes up comparisons between people of the same group and comparisons between people of different groups. If a model is unfair, the old CI might still look high because it's weighted heavily by the groups where the model works well.

2. The New Tool: xCI (The "Fairness Magnifying Glass")

The authors propose xCI. Instead of one big grade, xCI breaks the grading down into specific matchups. It asks two distinct questions:

Question A (Within-Group): "How well does the model sort two people from the same group?" (e.g., Group A vs. Group A).
Question B (Between-Group): "How well does the model sort a person from Group A against a person from Group B?"

The Creative Metaphor:
Imagine a race.

Old CI: "Did the runners finish in the right order overall?"
New xCI: "Did the runners finish in the right order within their own team? AND, when a runner from Team Red races a runner from Team Blue, does the system correctly predict who wins?"

The paper proves mathematically that the old CI is just a weighted average of all these specific xCI matchups. If the model is bad at comparing Team Red vs. Team Blue, but good at Team Red vs. Team Red, the old CI might hide that specific failure. The xCI shines a light on it.

3. Why This Matters for "Fairness"

The paper argues that in healthcare, fairness means that if two people have the same level of medical need, they should have the same chance of being prioritized correctly, regardless of their background.

The "Separation" Concept: The authors focus on a type of fairness called "separation." This means the model should be equally good at identifying the sick people, no matter which group they belong to.
The Discovery: The paper shows that by using xCI, they can find biases that other tools miss. For example, a model might look fair when looking at Group A alone and fair when looking at Group B alone, but it might systematically rank Group A patients ahead of Group B patients even when the Group B patients are actually sicker. The xCI catches this "cross-group" unfairness.

4. Handling the "Missing Data" Problem (Censoring)

In medical studies, we often don't know exactly when a patient gets sick or dies because they leave the study early or the study ends before the event happens. This is called "censoring."

The Analogy: Imagine a race where some runners drop out before the finish line. If you only count the runners who finished, you might get a wrong idea of who was actually faster.
The Solution: The paper provides a special mathematical method (called IPCW) to adjust the xCI calculation. It essentially gives extra weight to the data we do have, so the missing data doesn't trick the fairness score. This ensures the tool works even when the data is incomplete.

5. Real-World Tests

The authors tested this new tool on two real-world scenarios:

Heart Disease Studies: They looked at data from three major heart studies (Framingham, MESA, ARIC) to see if their models treated different groups fairly.
Electronic Health Records: They tested existing heart disease risk models using a massive database of real patient records (Truveta).

The Result: In both cases, the new xCI tool found biases and unfair rankings that the old, standard tools completely missed.

Summary

This paper doesn't just say "we need fairness." It builds a new ruler (xCI) specifically designed to measure fairness in time-based medical predictions. It proves that looking at the "average" performance isn't enough; you have to look at how the model treats people within their groups and between different groups. If you want to make sure a medical AI doesn't accidentally ignore the sickest people just because of their demographic group, this new tool is the way to check.

Technical Summary: Towards Fair Predictions: Group Conditional Concordance Index to Quantify Fairness in Time-to-Event Prognostication

Problem Statement
Predictive models in healthcare are increasingly used to allocate limited resources, such as preventive care or specialized referrals. While fairness metrics are essential for identifying biases in these models, existing approaches are predominantly designed for binary classification tasks. In the context of time-to-event (survival) analysis, fairness evaluation remains underdeveloped. Current methods often rely on comparing within-group performance metrics (e.g., Concordance Index or AUC calculated separately for each demographic group). However, this approach fails to capture between-group fairness: the ability of a model to correctly rank individuals from different groups relative to one another. In resource allocation scenarios, decisions are inherently comparative across groups; a model may appear fair within each subgroup independently while systematically favoring one group over another in cross-group comparisons, leading to inequitable resource distribution.

Methodology
The authors propose a novel group fairness metric called the Group Conditional Concordance Index (xCI). This metric extends Harrell's Concordance Index (CI) by conditioning on the group membership of both individuals in a pairwise comparison.

Definition of xCI: For two groups $a$ and $b$ , the xCI is defined as the probability that the model correctly ranks the risk of an individual from group $a$ against an individual from group $b$ , given that the individual from group $a$ experiences the event earlier than the individual from group $b$ :
$xCI_{a,b} = P(R_i > R_j \mid T_i < T_j, G_i = a, G_j = b)$
where $R$ represents the predicted risk score and $T$ represents the true event time.
Decomposition of CI: The paper formally proves that the standard CI is a weighted average of all possible xCIs across group pairs. The weights depend on the frequency of comparable pairs within each group combination. This mathematical relationship demonstrates that a high overall CI can mask poor performance in specific cross-group rankings if those groups are underrepresented in the comparable pairs.
Estimation under Censoring: To handle right-censored data, the authors develop a consistent estimator using Inverse Probability of Censoring Weights (IPCW). This estimator corrects for bias introduced when the set of observed comparable pairs is not a random sample of all comparable pairs due to differential censoring rates across groups.
Fairness Diagnostics: The authors define specific diagnostic metrics derived from xCI:
- $\Delta_{within} xCI$ : The difference in within-group concordance ( $xCI_{a,a} - xCI_{b,b}$ ), measuring disparities in ranking accuracy within groups.
- $\Delta_{between} xCI$ : The difference in cross-group concordance ( $xCI_{a,b} - xCI_{b,a}$ ), measuring directional asymmetry in how the model prioritizes one group over another.
- Min xCI: The minimum xCI value across all group pairs, serving as a worst-case summary of model performance.

Key Contributions

Novel Metric: Introduction of xCI as a rigorous, group-conditional decomposition of the Concordance Index specifically designed for time-to-event data.
Theoretical Foundation: Formal proof that the standard CI is a weighted average of xCIs, highlighting the limitations of aggregate metrics in detecting between-group bias.
Statistical Consistency: Development and proof of consistency for an IPCW-based estimator of xCI under right-censoring, ensuring the metric is applicable to real-world survival data.
Alignment with Ethical Principles: The metric is explicitly aligned with Rawlsian principles of justice, specifically "Fair Equality of Opportunity" (ensuring similar clinical needs yield similar prioritization chances regardless of group) and the "Difference Principle" (prioritizing those with the greatest need). The approach focuses on separation, ensuring predictions are conditionally independent of group status given the true outcome.
Empirical Validation: Application of the metric to two distinct case studies:
- Assessing survival models trained on harmonized data from the Framingham Offspring, MESA, and ARIC studies.
- Evaluating existing cardiovascular disease (CVD) risk prediction models using the Truveta electronic health record (EHR) database.

Results
The application of xCI in the case studies revealed that the metric effectively detects biases across demographic groups that are overlooked by existing fairness metrics. Specifically, the analysis showed that models could exhibit high overall concordance while simultaneously demonstrating significant between-group unfairness (asymmetry in cross-group ranking). The IPCW estimator successfully accounted for censoring, providing robust estimates of these disparities. The results indicate that standard within-group evaluations are insufficient for ensuring fair resource allocation, as they miss the systematic favoring of one group over another in comparative decision-making.

Significance and Claims
The paper claims that xCI provides a necessary and valuable tool for fairness assessment in survival analysis, particularly in constrained resource allocation settings. By explicitly quantifying between-group ranking accuracy, xCI complements existing evaluation approaches and addresses a critical gap in algorithmic fairness. The authors assert that without such metrics, interventions cannot be made to correct unfairness, potentially exacerbating healthcare disparities. The framework offers a statistically rigorous method to evaluate whether model-driven resource allocation aligns with ethical principles of equity, ensuring that individuals with comparable clinical needs have comparable chances of being correctly prioritized, regardless of demographic background.