🤖 AI

Exploring the Impact of Skin Color on Skin Lesion Segmentation

This study reveals that while global skin tone metrics show weak association with segmentation performance, low lesion-skin contrast is a consistent driver of segmentation errors, suggesting that fairness improvements in AI-driven dermatology should prioritize handling low-contrast boundaries over discrete skin-tone categories.

Original authors: Kuniko Paxton, Medina Kapo, Amila Akagić, Koorosh Aslansefat, Dhavalkumar Thakker, Yiannis Papadopoulos

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kuniko Paxton, Medina Kapo, Amila Akagić, Koorosh Aslansefat, Dhavalkumar Thakker, Yiannis Papadopoulos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🎨 The Big Picture: The "Highlighter" Problem

Imagine you are a doctor trying to find a tiny, dangerous weed (a skin cancer lesion) growing in a vast garden (the patient's skin). To help you, you have a team of AI robots.

Before the robots can tell you if the weed is dangerous, they first have to outline the weed perfectly so they can zoom in on it. This outlining process is called Segmentation.

The big question this paper asks is: Does the color of the garden (the patient's skin) make it harder for the robots to outline the weed?

🚫 The Old Way: Sorting People into Boxes

For a long time, researchers tried to solve fairness issues by sorting people into a few "boxes" based on their skin tone (like the Fitzpatrick scale, which groups skin from "very fair" to "very dark").

The Analogy: Imagine trying to describe the color of a sunset by only saying "Blue," "Orange," or "Red." You miss all the beautiful purples, pinks, and gradients in between.

The paper argues that this "box" method is too blunt. It ignores the subtle differences within a single image. A person might have a mix of light and dark spots, or the lighting might change the look of the skin. By forcing everyone into a box, we lose the nuance needed to see exactly why an AI might fail.

🔍 The New Way: Measuring the "Contrast"

Instead of asking, "What color is the skin?", the researchers asked, "How different does the weed look from the grass?"

They introduced a new way of looking at the data:

Don't just look at the skin: Look at the difference between the skin and the lesion.
Think in gradients: Instead of "Dark Skin" vs. "Light Skin," they measured the contrast (how much the colors clash or blend).

The Analogy: Imagine trying to find a white cat in a room.

High Contrast: The cat is on a black rug. Easy to see! The AI can draw the outline perfectly.
Low Contrast: The cat is on a white rug. It's hard to tell where the cat ends and the rug begins. The AI gets confused and draws a messy line.

The paper found that this "blending" (low contrast) is the real culprit, not the skin color itself. Whether the skin is dark or light, if the lesion blends in too much with the surrounding skin, the AI struggles.

🤖 The Robot Team

The researchers tested three different types of AI "robots" (models) to see if they all suffered from the same problem:

UNet: The classic, reliable workhorse.
DeepLabV3: A more advanced, modern worker.
DINOv2: A brand-new, super-smart robot trained on millions of natural images.

The Result: All three robots failed in the exact same way. When the lesion and the skin looked too similar (low contrast), all the robots made mistakes. This proves the problem isn't that one specific robot is biased; it's that the task itself is harder when the colors blend together.

📉 What They Found (The "Aha!" Moment)

The researchers used a special mathematical tool (called Wasserstein Distance) to measure these color differences precisely. Here is what they discovered:

Myth Busted: Simply knowing a patient has "Dark Skin" doesn't automatically mean the AI will fail.
The Real Issue: The AI fails when the lesion is hard to see against the skin.
The Danger: When the AI can't draw a clear line, it might miss part of the cancer or cut off too much healthy skin. This error then gets passed down to the next step (diagnosis), potentially leading to a wrong medical conclusion.

💡 The Takeaway: How to Fix It

The paper suggests we stop trying to fix AI by just adding more "Dark Skin" photos to the database. Instead, we need to focus on Low Contrast cases.

The Solution Analogy:
If you are trying to teach a child to find a needle in a haystack, don't just give them more haystacks. Give them a magnet (better contrast detection) or teach them to look for the difference in texture, not just the color.

Practical Steps for the Future:

Audit by Contrast: When testing AI, don't just check "Fair Skin" vs. "Dark Skin." Check "High Contrast Lesions" vs. "Low Contrast Lesions."
Special Attention: If an AI sees a lesion that blends in with the skin, it should flag it as "Uncertain" and ask a human doctor to double-check, rather than guessing.
Better Training: Train the AI specifically on those tricky, low-contrast images so it learns to spot the subtle boundaries.

🏁 Summary in One Sentence

The paper proves that AI struggles to outline skin lesions not because of the patient's skin color, but because the lesion sometimes blends in too well with the skin; therefore, we need to teach AI to handle "blending" better, rather than just sorting people into skin-tone boxes.

1. Problem Statement

Skin cancer, particularly melanoma, requires early detection for improved survival rates. AI-driven dermatology systems often use skin lesion segmentation as a critical preprocessing step to isolate lesions from surrounding skin before classification. While fairness issues regarding skin tone in classification are well-documented, the impact of skin tone on the segmentation stage remains under-quantified.

Previous research has relied on discrete skin-tone categories (e.g., Fitzpatrick scale) or coarse metrics (mean ITA) to assess bias. The authors argue that these methods fail to capture:

Pixel-level variations and unevenness within a single image.
The nuance of lesion-skin contrast, which is the primary visual cue for boundary detection.
The possibility that segmentation models themselves introduce bias before the classification stage even begins.

2. Methodology

A. Datasets and Models

The study evaluated three distinct segmentation architectures on two public dermoscopic datasets (HAM10000 and ISIC2017):

UNet: A standard CNN-based architecture (ResNet-50 backbone).
DeepLabV3: A strong CNN-based alternative (ResNet-50 backbone).
DINOv2: A Vision Transformer (ViT-B14) backbone, leveraging self-supervised pretraining.

B. Preprocessing

To ensure accurate color analysis, a standard hair removal pipeline was applied to all images prior to tone computation. This involved noise reduction, hair/shadow emphasis using the Black Hat method, contrast enhancement, and binary masking.

C. Novel Metric: Distribution-Based Skin Tone Analysis

Instead of assigning a single scalar value to an image, the authors introduced a continuous, distribution-based approach:

Metric: They calculated the Individual Typology Angle (ITA) at the pixel level using CIELab color space.
Reference: A uniform reference distribution anchored at $-30^\circ$ (darkest group) was used to ensure stability.
Distance Measure: The Wasserstein Distance (WD) was used to compare cumulative distribution functions (CDFs) of ITA values. WD is preferred over L1/L2 distances as it better captures human-perceived color and texture similarities.

D. Six Comparison Patterns

The study defined six specific patterns to disentangle global tone from local contrast:

Global Tone: Distance between reference and the full image distribution.
Skin-Only Tone: Distance between reference and skin-only pixels.
Lesion-Only Tone: Distance between reference and lesion-only pixels.
Lesion-Skin Contrast: WD between skin-only and lesion-only distributions.
Image-Lesion Contrast: WD between full image and lesion-only distributions.
Image-Skin Contrast: WD between full image and skin-only distributions.

E. Evaluation

Performance was measured using nine standard metrics (IoU, Dice Coefficient, Sensitivity, Specificity, Pixel Accuracy, AUC, Cohen's Kappa, FPR, FNR). Correlations between the six color patterns and segmentation performance were analyzed using Spearman's rank correlation, validated with 95% confidence intervals via bootstrapping.

3. Key Contributions

Shift from Discrete to Continuous Analysis: The paper moves beyond the Fitzpatrick scale and mean ITA, proposing a distribution-based representation that preserves within-image gradations and unevenness.
Identification of Contrast as the Primary Driver: The study empirically demonstrates that lesion-skin pigment contrast is a significantly stronger predictor of segmentation error than global skin tone.
Architectural Agnosticism: The findings hold true across diverse model families (CNNs like UNet/DeepLab and Transformers like DINOv2), suggesting the issue is fundamental to the visual task rather than a specific model flaw.
Quantitative Audit Signal: The authors provide a method to quantify boundary ambiguity using Wasserstein distances, offering a more informative signal for auditing fairness than discrete categories.

4. Key Results

Weak Correlation with Global Tone: Global skin tone metrics (Fitzpatrick grouping and mean ITA) showed weak associations ( $|\rho| < 0.2$ ) with segmentation performance across all models and datasets. This suggests that simply knowing a patient's skin type does not predict segmentation accuracy.
Strong Correlation with Low Contrast: There is a consistent, strong negative correlation between low lesion-skin contrast and segmentation accuracy.
- As the contrast between the lesion and surrounding skin decreases (i.e., the distributions become more similar), segmentation errors (FPR, FNR) increase, and metrics like IoU and Dice Coefficient decrease.
- This effect was observed across all three architectures and both datasets.
Disease-Specific Findings: The contrast-performance relationship held true even when stratified by disease type (e.g., Melanoma, Basal Cell Carcinoma), including clinically critical malignant classes.
Statistical Significance: All reported correlations were statistically significant ( $p < 0.001$ ) with narrow confidence intervals, confirming the robustness of the findings.

5. Significance and Implications

Redefining Fairness in Dermatology: The paper argues that fairness efforts in segmentation should not focus primarily on balancing discrete skin-tone categories in datasets. Instead, the priority should be robust handling of low-contrast lesions, which are the true source of failure and potential bias.
Clinical Safety: Low-contrast cases lead to boundary ambiguity, causing under-segmentation or over-segmentation. These errors propagate to downstream classification and triage systems, potentially leading to misdiagnosis.
Actionable Recommendations:
1. Contrast-Stratified Reporting: Evaluation reports should include performance breakdowns based on contrast levels, not just skin tone.
2. Targeted Data Augmentation: Training data should be augmented to specifically increase the representation of low-contrast cases.
3. Uncertainty-Triggered Review: Systems should use continuous contrast scores as an audit signal to flag low-confidence boundaries for human review.
Limitations: The study is limited to the skin tone ranges present in HAM10000 and ISIC2017, which may underrepresent very dark skin tones. Future work requires validation on more diverse cohorts.

Conclusion

The study concludes that boundary ambiguity caused by low lesion-skin contrast is the dominant factor driving segmentation errors, rather than the patient's global skin tone. By adopting distribution-based, continuous metrics, researchers can better identify and mitigate these failure modes, leading to safer and more equitable AI-driven dermatological pipelines.