Technical Summary: Contrast-Induced Class Overlap as a Fairness Bottleneck in Dermatological AI

1. Problem Statement

The paper addresses a critical fairness bottleneck in AI-assisted skin cancer triage: the systematic over-prediction of malignancy on darker skin tones. While prior work often attributes performance disparities to data scarcity (i.e., a lack of dark-skin training images), this study argues that the issue is mechanistic and structural.

The core problem is contrast-induced class overlap. Melanin concentration in darker skin reduces the optical contrast between a lesion and the surrounding tissue. Since convolutional neural networks (CNNs) rely on these contrast-dependent features, the effective signal-to-noise ratio (SNR) degrades significantly on darker skin. This physical limitation leads to a specific failure mode: when models are calibrated for high sensitivity (to avoid missing cancers), they systematically over-predict on darker skin, resulting in disproportionately high false-positive rates (low specificity) and unnecessary referrals, even when sensitivity remains comparable across skin tones.

Additionally, the paper identifies a methodological confound: a specific benign lesion class (vascular lesions) is over-represented in lighter skin within the HAM10000 dataset. If these are mislabeled as positive during binary classification, it artificially inflates the apparent sensitivity/AUC gap between light and dark skin, potentially masking the true nature of the disparity.

2. Methodology

Dataset and Preprocessing

• Dataset: The study utilizes the HAM10000 dataset (10,015 dermoscopic images).
• Splitting: A patient-level split (based on lesion_id) was used to prevent data leakage, resulting in a training set ($n=6,959$), validation set ($n=1,529$), and a locked test set ($n=1,527$).
• Task Definition: A referral triage task where the positive class includes melanoma, BCC, and AKIEC, while the negative class includes benign lesions (nevi, keratoses, etc.).
• Confound Correction: The authors corrected a labeling error where vascular lesions (VASC), which are benign, were previously prone to being grouped as positive. They noted that 62% of VASC images in the dataset belong to lighter skin tones; mislabeling them as positive artificially depressed light-skin sensitivity.

Skin Tone Estimation

• Proxy: Since HAM10000 lacks ground-truth Fitzpatrick labels, the authors estimated skin tone using the Individual Typology Angle (ITA) derived from CIE $L^*a^*b^*$ color space via Otsu background segmentation.
• Groups: Skin tones were categorized as Light (ITA > 41°), Medium (10° < ITA ≤ 41°), and Dark (ITA ≤ 10°). A high-confidence (HC) subset was used for primary analysis to minimize proxy error.

Signal-to-Noise Ratio (SNR) Framework

The authors formalized the physical mechanism using an SNR framework:
$$SNR(ITA) = \frac{|\mu_{lesion} - \mu_{skin}|}{\sigma_{skin}}$$
Analysis of pixel statistics revealed an SNR reduction of 5.2× from light to dark skin tones. Under a Gaussian signal-detection model, this reduction theoretically lowers the Bayes-optimal AUC ceiling from ~0.998 to ~0.709, explaining why models struggle even with balanced training data.

Experimental Design

The study employed a five-variant ablation using an EfficientNet-B2 backbone with a locked test set:

1. Baseline: Standard augmentation, no ITA conditioning.
2. Aug-only: Added dark-skin specific augmentation (Mixup, oversampling) without ITA conditioning.
3. Tone-only: Standard augmentation with ITA conditioning (concatenating a tone embedding).
4. Full: Combination of Tone-only and Aug-only.
5. Balanced Baseline: A control using equalized class exposure (pos_weight=1.0) to isolate the effect of class balancing.

Statistical significance was assessed via bootstrap 95% confidence intervals and pre-specified permutation tests.

3. Key Results

Robust Specificity–Sensitivity Asymmetry

On the high-confidence subset, the study found that while sensitivity was comparable across skin tones (Light: 0.813 vs. Dark: 0.803), specificity was significantly lower for darker skin (Light: 0.826 vs. Dark: 0.715).

• Disparity: A specificity deficit of −11.0 percentage points (pp) for darker skin ($p < 0.0002$).
• Clinical Impact: This translates to approximately 89 excess unnecessary referrals per 1,000 darker-skin patients compared to lighter-skin patients, driven by over-prediction rather than missed cancers.

The Labelling Confound

The authors demonstrated that a naive analysis (mislabeling vascular lesions as positive) overstates the AUC/sensitivity gap (appearing as −5.5 pp) but leaves the specificity gap largely intact (−11.1 pp vs. −11.0 pp after correction). This confirms that the specificity deficit is a robust physical signal, while the AUC gap can be partially an artifact of tone-correlated class composition.

Ablation Findings

• Tone Conditioning: Adding ITA conditioning improved dark-skin sensitivity (0.871) but failed to significantly recover specificity (0.673).
• Class Balancing: The Balanced Baseline (equalized class exposure) recovered the most dark-skin specificity, lifting it from 0.668 to 0.799. This narrowed the specificity gap to −6.3 pp.
• Conclusion: The asymmetry has a mixed causal structure: a physical component (contrast loss) and a decision-rule component (sensitivity calibration) that is addressable via class balancing and threshold adjustment, rather than feature conditioning alone.

External Validation

Zero-shot transfer to the DDI dataset (clinical photography) confirmed the mechanism. Dark-skin images received systematically lower model confidence scores regardless of the label, manifesting as score suppression under severe distribution shift.

4. Contributions and Significance

The paper claims three primary contributions:

1. Mechanistic Lens: It provides an SNR-based diagnostic framework explaining why pigmentation-driven contrast loss creates a specific "specificity–sensitivity asymmetry" in dermatological AI, applicable to other imaging domains with varying tissue pigmentation.
2. Robustness of the Signal: It demonstrates that the specificity deficit is a real clinical signal that survives the correction of a tone-correlated labeling confound, distinguishing physical failure modes from analytical artifacts.
3. Intervention Strategy: It shows that per-tone class balancing is more effective than tone conditioning for recovering dark-skin specificity, suggesting that the problem is partly data-addressable (via decision rule adjustment) rather than purely a feature-learning limitation.

Significance

The authors argue that fairness audits in dermatology must shift focus from aggregate AUC to subgroup specificity. Because sensitivity and specificity gaps can cancel each other out in AUC metrics, relying on AUC alone conceals the disproportionate burden of false positives placed on darker-skinned populations. Furthermore, the paper highlights the necessity of auditing for tone-correlated class composition to ensure that observed disparities are not artifacts of dataset labeling biases.

The study concludes that while the physical contrast loss is a hard constraint, the resulting over-prediction can be mitigated through data balancing and threshold calibration, offering a practical pathway to reduce the referral burden on underserved populations.