Integrating Group and Individual Fairness Auditing in Clinical AI: A Post-Hoc, Model-Agnostic Approach

This paper introduces EquiLense, a practical, post-hoc, and model-agnostic auditing tool that bridges the gap between group and individual fairness assessments in clinical AI by utilizing a novel metric called Mean Predicted Probability Difference (MPPD) to identify systematic prediction inconsistencies across demographic groups.

The Gist

Imagine you have a very smart, automated assistant that helps doctors predict how a patient might do after surgery. This assistant is great at its job overall, but there's a nagging worry: Is it treating everyone fairly?

Sometimes, these assistants might be unfair in two different ways:

1. Group Unfairness: It consistently gives worse predictions for one entire group of people (like a specific race or gender) compared to another.
2. Individual Unfairness: It treats two patients who are medically identical (same age, same health issues, same surgery) differently just because they belong to different groups.

The problem is that most tools used to check for fairness only look at one of these angles. They might check if Group A gets worse scores than Group B, but they miss the fact that two specific, identical patients are being treated differently. Or they check if identical patients are treated the same, but miss the bigger picture of systemic bias against a whole group.

Enter "EquiLense": The Fairness Glasses

The authors of this paper created a new tool called EquiLense. Think of it as a pair of "fairness glasses" that a doctor or developer can put on after the AI model is already built and working. You don't have to rebuild the engine; you just look through the glasses to see what's really happening.

EquiLense does three main things to give a complete picture:

1. The Group Check: It looks at the big picture to see if certain demographic groups are getting systematically worse predictions than others.
2. The Individual Check: It finds pairs of patients who are medically twins (same age, same health history) and checks if the AI gives them the same prediction. If it gives one a "high risk" score and the other a "low risk" score just because of their race or insurance, that's a red flag.
3. The "Mean Predicted Probability Difference" (MPPD): This is the paper's secret sauce. It's a new way of measuring the gap between those "medical twins."

Here is a simple analogy for MPPD:
Imagine you are a judge sentencing two people who committed the exact same crime with the exact same history.

• Fairness: Both get 5 years.
• Unfairness: One gets 5 years, and the other gets 10 years just because they are from a different neighborhood.

MPPD is like a ruler that measures exactly how much extra time the second person got compared to the first, on average, across the whole courtroom. It quantifies the "unfair gap" between people who should be treated the same.

What Did They Find?

The team tested EquiLense on real hospital data involving over 59,000 surgical patients. They looked at models predicting two things: delirium (confusion after surgery) and readmission (coming back to the hospital within 30 days).

• The Surprise: The AI models were actually quite good at predicting outcomes overall (they were accurate). However, when they put on the EquiLense glasses, they found that the models were still treating "medical twins" differently based on race.
• The Specific Example: For patients who were medically identical to White patients, Asian patients were getting systematically different (and less fair) predictions. The "gap" in their scores was measurable and significant.
• The Fix Test: They tried a simple experiment: they told the AI to ignore race and insurance type when making its predictions. When they did this, the "unfair gap" (the MPPD score) shrank significantly. This suggests that simply removing those specific data points from the model's "brain" made it treat similar patients more equally, without making the model worse at its job.

Did It Work on Other Problems?

To make sure their new ruler (MPPD) actually worked, they tested it on two famous, non-medical datasets where bias was already known to exist:

1. COMPAS: A tool used to predict if criminals will re-offend. (We know this tool has historically been biased against Black defendants).
2. UCI Adult Income: A dataset predicting if someone earns over $50k. (We know this has historical gender bias).

The Result: EquiLense's MPPD metric successfully flagged the exact groups we already knew were being treated unfairly (Black defendants in the COMPAS data and women in the income data). This proved the tool works.

Why Does This Matter?

The paper argues that we need a tool that doesn't require us to throw away our current AI models and start over (which is expensive and hard). Instead, we need a way to audit them after they are built.

EquiLense is like a quality control inspector for AI in healthcare. It doesn't fix the machine for you, but it gives you a clear, easy-to-understand report card that says: "Hey, your machine is good at math, but it's treating these two identical patients differently just because of their background."

This allows doctors and developers to make informed choices, like deciding whether to remove certain data points (like race) from the model to make it fairer, without needing to be math wizards or rebuild the entire system from scratch.

Technical Summary

1. Problem Statement

The integration of AI into clinical decision-making raises critical ethical concerns regarding algorithmic fairness. Current fairness evaluation frameworks suffer from fragmentation:

• Group Fairness: Metrics (e.g., Demographic Parity, Equal Opportunity) assess disparities across demographic groups but often obscure patient-level variations. A model can appear fair at the group level while treating clinically similar individuals inconsistently.
• Individual Fairness: Metrics ensure that similar individuals receive similar predictions. However, these are rarely adopted in healthcare due to the difficulty of defining "clinical similarity" and the lack of operational tools. Existing approaches often require model retraining, specific architectures (e.g., graph neural networks), or complex causal graph specifications (Counterfactual Fairness), making them impractical for auditing already-deployed models.

The Gap: There is a lack of practical, post-hoc, model-agnostic tools that simultaneously evaluate group-level disparities and individual-level prediction consistency in clinical settings.

2. Methodology: The EquiLense Framework

The authors introduce EquiLense, a post-hoc, model-agnostic auditing tool designed to bridge group and individual fairness. It consists of three core components:

A. Group Fairness Evaluation

The tool calculates standard disparity metrics across sensitive attributes (e.g., race, sex, insurance type):

• Demographic Parity Difference (DPD)
• Equal Opportunity Difference (EOD)
• Equalized Odds (EO)
• Error Distribution Disparity Index (EDDI): A novel metric designed for clinical settings with diverse group sizes and class imbalance.

B. Individual Fairness Assessment

The tool measures prediction consistency among clinically similar patients.

• Similarity Definition: Users can define clinical similarity based on specific features (e.g., age, BMI, Charlson Comorbidity Index) using distance-based metrics (e.g., cosine similarity).
• Flexibility: The framework allows users to adjust similarity thresholds and feature sets to match specific clinical contexts.

C. Mean Predicted Probability Difference (MPPD)

This is the paper's novel methodological contribution, designed to operationalize individual fairness while capturing group-level effects.

• Concept: MPPD quantifies the average absolute difference in predicted probabilities between clinically similar patients who belong to different demographic groups.
• Implementation:
  1. Reference-Based: Identifies a reference group (e.g., White patients) and matches them with clinically similar patients from other groups. It calculates the prediction gap.
  2. Pairwise: Performs bi-directional matching between all group combinations (e.g., Black-White, Asian-Hispanic) to create a symmetric matrix of disparity scores. This avoids assuming a single "fair" reference group and highlights disparities between minority groups.
• Advantage: Unlike Counterfactual Fairness, MPPD relies on observed, real-world data rather than synthetic counterparts or unverified causal graphs, making it computationally efficient and interpretable for clinicians.

3. Study Design and Data

• Clinical Cohort: 59,047 surgical patients from a large academic medical center (2012–2022).
• Outcomes:
  1. Post-surgical delirium (identified via ICD codes, CAM scores, and NLP of clinical notes).
  2. 30-day hospital readmission.
• Models: Logistic Regression and XGBoost classifiers trained on clinical features (age, BMI, CCI, mental health history, etc.). Sensitive attributes (race, sex, insurance) were included or excluded to test their impact.
• External Benchmarks: Validated MPPD on two datasets with known historical biases:
  1. COMPAS: Recidivism prediction (n=6,172).
  2. UCI Adult Income: Income prediction (n=32,561).

4. Key Results

Clinical Application (Surgical Models)

• Performance: The XGBoost model achieved an AUROC of 0.78 for delirium and 0.61 for readmission. Removing sensitive attributes had negligible impact on predictive performance.
• Group Fairness: Standard metrics showed moderate disparities, particularly in Error Distribution (EDDI).
• Individual Fairness (MPPD):
  • MPPD revealed systematic inconsistencies: Among patients clinically similar to White patients, Asian patients exhibited the largest prediction disparity (MPPD difference = 0.045).
  • Feature Sensitivity: Excluding sensitive attributes (race/ethnicity) from model training significantly reduced MPPD scores, indicating improved consistency without sacrificing accuracy.
  • Pairwise Analysis: The pairwise matrix highlighted disparities between minority groups that reference-based methods might miss.

External Benchmark Validation

• COMPAS: MPPD correctly identified the highest disparity for African American defendants, consistent with known biases in recidivism scoring.
• UCI Adult Income: MPPD assigned the highest disparity score to female workers, aligning with documented gender bias in income prediction.
• Conclusion: MPPD successfully detected known ground-truth biases in non-clinical datasets, validating its utility as a screening metric.

5. Key Contributions

1. EquiLense Tool: A unified, post-hoc framework that integrates group and individual fairness metrics without requiring model retraining.
2. MPPD Metric: A novel, interpretable metric that bridges the gap between group disparity and individual consistency by comparing real, observed patients. It is computationally lightweight and does not require causal graph specification.
3. Practical Applicability: The tool is model-agnostic (works with any classifier) and allows clinicians to define "similarity" based on domain knowledge, making it directly applicable to real-world healthcare auditing.
4. Empirical Evidence: Demonstrated that high overall model performance does not guarantee fairness; models can systematically treat clinically similar patients differently based on demographic attributes.

6. Significance and Limitations

Significance:

• Bridging the Gap: EquiLense addresses the "blind spot" where group metrics miss individual inconsistencies and individual metrics lack systemic context.
• Actionable Insights: By allowing sensitivity analyses (e.g., removing sensitive features), the tool helps practitioners make informed decisions about model deployment and feature selection to mitigate bias.
• Post-Hoc Utility: It fills a critical need for auditing models already in production, a stage where training-time mitigation strategies are no longer applicable.

Limitations:

• Causality: MPPD is a correlation-based metric; it flags inconsistencies but cannot determine if they are due to algorithmic bias or clinically appropriate distinctions.
• Similarity Definition: The accuracy of individual fairness depends on the user's subjective definition of "clinical similarity." Poor feature selection or noisy data can distort results.
• Downstream Impact: The tool audits model outputs but does not assess downstream clinical consequences (e.g., actual changes in patient care or health outcomes).
• Sample Size: Estimates for small subgroups (e.g., Native American patients in the study) should be interpreted with caution.

Conclusion

The paper presents EquiLense as a vital step toward operationalizing fairness in clinical AI. By combining established group metrics with the novel MPPD, it provides a comprehensive, interpretable, and practical approach to auditing deployed models, ensuring that AI systems advance rather than undermine health equity.