People-Centred Medical Image Analysis

1. Problem Statement

Despite the high accuracy of recent data-centric medical AI models, their clinical adoption remains limited. The authors identify two primary bottlenecks:

1. Performance Bias: Existing models often fail to perform equitably across diverse patient populations (e.g., different sexes, ages, or races), creating regulatory and ethical barriers.
2. Workflow Integration: Current approaches to human-AI collaboration, specifically Learning to Defer (L2D) and Learning to Complement (L2C), are typically studied in isolation from fairness concerns.
   • L2D decides whether to let an AI predict or defer to a human.
   • L2C combines AI and human predictions.
   • The Gap: Prior work assumes prior knowledge of patient group membership (sensitive attributes) during testing (which is often unavailable in real-world settings) and fails to jointly optimize for fairness, accuracy, and clinician workload constraints.

2. Methodology: The PecMan Framework

The authors propose PecMan (People-Centred Medical Image Analysis), a unified framework that jointly optimizes AI fairness, diagnostic accuracy, and workflow effectiveness. It operates without assuming prior knowledge of patient group membership during the testing phase.

Core Architecture

The framework utilizes a dynamic gating mechanism to assign cases to:

• AI models (specifically cohort-specific models).
• Clinicians.
• A collaborative combination of both.

The system consists of four main components:

1. Backbone Model ($f_\theta$): Extracts features from input images.
2. Cohort-Specific Models ($h_{\phi_j}$): A set of classifiers, each specialized for a specific demographic group (e.g., male, female).
3. Gating Model ($g_\zeta$): A dynamic router that determines the optimal strategy for a given case (AI-only, Human-only, or Hybrid) based on the input image and workload constraints.
4. Consolidator Model ($m_\gamma$): Aggregates the weighted predictions from the selected AI models and/or the clinician to produce the final diagnosis.

Training Process (Three Steps)

The training is sequential and utilizes the Fair Identity Scaling (FIS) loss function, which balances individual sample difficulty with group-level fairness.

• Step 0: Backbone Training: The backbone model is trained using the FIS loss to ensure the feature extractor is fair and accurate across all groups initially.
• Step 1: Group-Specific Model Training: The backbone is frozen, and separate classification heads are trained for each demographic cohort. This step focuses on maximizing accuracy within specific groups using the FIS loss (with fairness weighting set to prioritize individual accuracy).
• Step 2: Unbiased L2D + L2C Training: The gating and consolidator models are trained.
  • The objective is to minimize the FIS loss on the final aggregated prediction.
  • Constraint: A "collaboration budget" ($\epsilon$) is enforced, ensuring a minimum proportion of cases are handled by AI models to respect clinician workload limits.
  • Crucially, this step does not use sensitive attributes as input during inference; it learns to route cases based on image features alone, while fairness is evaluated post-hoc.

3. Key Contributions

1. PecMan Framework: The first unified framework to jointly optimize AI fairness, L2D (deferral), and L2C (complementation) without requiring sensitive attribute labels at test time. It dynamically allocates cases to AI, humans, or both under workload constraints.
2. FairHAI Benchmark: A new benchmark designed to evaluate the trade-offs between Accuracy, Fairness, and Clinician Workload.
   • It utilizes four public medical imaging datasets: HAM10000 (dermatology), CMMD (mammography), CheXpert, and MIMIC-CXR (chest X-rays).
   • It simulates clinician annotations based on empirical performance gaps (AI vs. Human) to create a realistic evaluation environment.
   • It introduces metrics like AUACC (Area Under AUC-Coverage Curve) and AUESACC (Area Under Equity-Scaled AUC-Coverage Curve).

4. Experimental Results

Experiments were conducted on the FairHAI benchmark across four datasets, comparing PecMan against state-of-the-art baselines including ERM, GroupDRO, SWAD, Fair L2D, and LECODU.

• Performance: PecMan consistently outperformed all baselines across all datasets.
  • It achieved the highest AUC (Accuracy) and ES-AUC (Fairness-adjusted Accuracy) scores.
  • In the HAM10000 dataset, PecMan achieved an AUAUC of 97.63% and AUESACC of 97.21%, significantly higher than the next best method (LECODU: 94.60% / 94.44%).
• Robustness: Even at 0% coverage (clinician-only), PecMan outperformed other methods because the clinician's decision was effectively complemented by the cohort-specific models.
• Deferral Behavior: Analysis showed the gating model effectively adjusts the deferral rate to clinicians based on the workload budget ($\epsilon$), ensuring efficient human-AI collaboration.
• Ablation Study: Replacing the multiple cohort-specific models with a single fair model (L2D-FairDi) resulted in lower performance, proving the necessity of specialized models for different demographics.

5. Significance and Impact

• Clinical Viability: By explicitly modeling clinician workload constraints and integrating fairness into the decision-making loop, PecMan addresses the "last mile" problem of AI adoption in healthcare.
• Regulatory Compliance: The framework directly addresses the need for equitable performance across diverse populations, a key requirement for regulatory approval (e.g., FDA, EU AI Act).
• Paradigm Shift: The paper argues against treating fairness and human-AI collaboration as separate objectives. It demonstrates that interdependence between these factors is critical; ignoring it leads to suboptimal trade-offs.
• Future Direction: The authors suggest that while training multiple cohort models is currently resource-intensive, future work could explore "hidden cohorts" to reduce computational overhead while maintaining fairness.

In summary, PecMan represents a significant step toward trustworthy, equitable, and clinically deployable medical AI by unifying fairness, accuracy, and human workflow constraints into a single, dynamic learning framework.