🤖 machine learning

Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation

This paper introduces a data-centric framework that adapts Confident Learning to detect and mitigate label bias in image segmentation by leveraging model confident predictions to identify systematic errors and feature space artifacts, thereby achieving equitable performance across demographic subgroups without requiring clean, unbiased ground truth labels.

Original authors: Aditya Parikh, Stella Frank, Sneha Das, Aasa Feragen

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aditya Parikh, Stella Frank, Sneha Das, Aasa Feragen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Broken Ruler"

Imagine you are teaching a robot to draw the outline of a cat in a photo. You give it a stack of 1,000 photos to learn from. However, there's a secret flaw: for every photo of a black cat, a human annotator accidentally drew the outline too small, cutting off the tips of the ears and tail. For every photo of a white cat, the outline was perfect.

The robot learns from these drawings. When it sees a black cat later, it draws a tiny, incomplete cat because that's what it was taught. When it sees a white cat, it draws a perfect one.

This is Label Bias. It's not that the robot is "dumb"; it's that the "ruler" (the training data) is broken for a specific group of people (or in this case, specific types of cats).

The Trap: Why Standard Checks Fail

Usually, when we check if a robot is doing a good job, we use a score called Dice or IoU. Think of this like a teacher grading a test.

The Trap: If the teacher's answer key is also wrong (because it was drawn by the same biased human), the robot gets a high score!
The Paper's Finding: The authors show that standard metrics are like a broken ruler measuring a broken object. They say, "Great job!" even when the robot is failing a specific group. In fact, the paper shows that sometimes the robot looks better on the biased group simply because the "correct" answer it's being compared against is also wrong.

The Solution Part 1: The "Confident Detective" (Auditing)

Since we often don't have a "perfect" answer key (clean ground truth) to check against, how do we find the bias?

The authors adapted a tool called Confident Learning. Imagine a detective who doesn't know the truth but knows how the robot thinks.

The detective asks the robot: "Are you 100% sure this pixel is part of the object?"
If the robot says, "Yes, I'm super confident this is a cat ear," but the training label says, "No, that's background," the detective flags it.
By looking at where the robot is confident but disagrees with the label, the detective can spot the bias.

The Analogy: It's like a student who is confident they spelled a word correctly, but the teacher's answer key says it's wrong. If the student is consistently confident and the teacher is consistently "wrong" for a specific group of students, the detective realizes the teacher's key is the problem, not the student.

The paper introduces new ways to measure this, distinguishing between:

Omission Errors: The label says "nothing here," but the robot sees "something." (The outline is too small).
Commission Errors: The label says "something here," but the robot sees "nothing." (The outline is too big).

The Solution Part 2: The "Special Glasses" (Mitigation)

Once we know the data is biased, how do we fix the robot without throwing away all the data?

The authors noticed something interesting: When the data is biased, the robot's internal "brain" (its feature space) starts treating the two groups as completely different species. It learns to separate them aggressively. Usually, fairness experts try to force the robot to ignore these differences.

The Paper's Twist: Instead of forcing the robot to ignore the difference, they gave it Special Glasses.

They built a system where the robot wears different "glasses" depending on which group the image belongs to.
During Training: The robot learns that Group A needs "thick glasses" (to fix the small outlines) and Group B needs "normal glasses."
During Inference (The Test): Here is the magic trick. When the robot sees a new image, it puts on the "Clean Group Glasses" for everyone, regardless of who they are.

The Analogy: Imagine a tailor who makes suits. They notice that for Group A, the fabric shrinks in the wash, so they cut the fabric extra large. For Group B, the fabric doesn't shrink, so they cut it normally.

Old Way: Try to make the fabric shrink the same amount for everyone (often fails).
Paper's Way: Cut the fabric specifically for the shrinkage issue. Then, when the customer walks in, the tailor simply uses the "Perfect Fit" pattern for everyone, effectively canceling out the shrinkage issue.

The Results

The authors tested this on three different scenarios:

Faces: Artificially shrinking the outlines of female faces.
Cells: Artificially making cell outlines "bloated" for a specific color group.
Skin Lesions: Real-world data where lighter skin tones had larger, more accurate outlines than darker skin tones.

The Outcome:

Standard Fairness Tools: Tried to force the robot to be "blind" to the groups. This failed. The robot still performed poorly on the biased group.
The Paper's Method: By acknowledging the bias and using the "Special Glasses" (Subgroup Conditioning), they fixed the performance gap. The robot became equally accurate for all groups, even though it never saw a single "perfect" label during training.

Summary

The paper argues that in image segmentation, bad data creates a hidden bias that standard tests can't see. To fix it, you don't need to throw away the data or force the AI to be "colorblind." Instead, you need to detect the specific direction of the error (too big or too small) and teach the AI to compensate for that specific error when making final predictions.

Technical Summary: Towards Fairness under Label Bias in Image Segmentation

Problem Definition

Algorithmic bias in AI often manifests as performance disparities across demographic subgroups. While label bias—systematic group-conditional label errors—is well-studied in classification, it remains underexplored in image segmentation. In safety-critical domains like medical imaging, biased annotations (e.g., systematic under- or over-segmentation of tumors for specific patient populations) can propagate directly into clinical decisions.

A critical challenge is that standard segmentation metrics (Dice, IoU) condense complex spatial errors into single scalars, often masking the directional nature of bias. Furthermore, detecting and mitigating this bias typically requires access to clean, unbiased ground-truth labels, which are rarely available in real-world scenarios where annotation protocols are inherently subjective and susceptible to demographic-specific distortions. Existing mitigation tools often assume clean labels or known bias structures, leaving a gap for addressing label bias when true labels are unknown.

Methodology

The paper proposes a data-centric framework for auditing and mitigating label bias in segmentation without requiring clean ground-truth. The approach consists of two main stages:

1. Auditing via Adapted Confident Learning (CL)

The authors adapt Confident Learning (CL), traditionally used for dataset auditing in classification, to the dense prediction task of segmentation.

Mechanism: Instead of diagnosing the model, the framework treats the dataset as the subject of trial. A segmentation model serves as a proxy to estimate unobserved true labels ( $y_{true}$ ) by generating confident predictions ( $\hat{y}$ ) via K-fold cross-validation.
Joint Distribution Estimation: The method computes a pixel-level joint distribution matrix $Q(y_{obs}, \hat{y})$ , where $y_{obs}$ is the observed (potentially biased) label. This matrix isolates off-diagonal entries representing errors where the observed label differs from the model's confident prediction.
Directional Metrics: To overcome the limitations of Dice/IoU, the framework projects $Q$ $Q$ onto two directional axes:
- Omission Rate (OmR): Pixels observed as background but confidently predicted as foreground (under-segmentation).
- Commission Rate (CoR): Pixels observed as foreground but confidently predicted as background (over-segmentation).
Bias Indicators: The framework employs statistical tests to distinguish systematic bias from random noise:
- Chi-square Independence Test: Checks if error types are independent of the subgroup.
- Relative Risk (RR): Quantifies the likelihood of a biased group exhibiting errors compared to a clean group.
- Symmetry Score (S): Measures if error distribution is proportional to group size (high S implies uniform noise; low S implies bias).

2. Mitigation via Subgroup-Conditioned Decoding

The authors observe that label bias induces feature-space separability, where the encoder learns to distinguish subgroups based on the corrupted annotation patterns rather than just visual features. Rather than suppressing this separability (as traditional invariance-based methods do), the framework leverages it.

Subgroup-Conditioned Decoder: The method injects learnable subgroup conditioning into the encoder-decoder pipeline using Feature-wise Linear Modulation (FiLM). During training, subgroup-specific parameters modulate encoder features.
Inference Strategy: At inference, the model is forced to condition on the "clean" reference group ( $g_c$ ) for all inputs, regardless of the actual subgroup identity. This systematically suppresses the learned annotation bias.
Asymmetric Boundary Masking: To prevent the model from learning the biased boundary styles, fine-grained boundary supervision is routed exclusively from the clean subgroup annotations, while interior object mass is leveraged from all samples.
Auto-Conditioning (Unsupervised Variant): When the biased subgroup is unknown, a warm-up phase identifies the "clean" group as the one with the lowest mean loss (stable loss attractors), allowing the system to transition to asymmetric loss automatically.

Key Contributions

Adaptation of Confident Learning to Segmentation: The paper introduces a framework to detect label bias directly in training data without clean ground truth, utilizing directional metrics (Omission/Commission rates) that standard overlap metrics fail to capture.
Empirical Evidence of Feature-Space Separability: The study demonstrates that label bias increases the separability of subgroups in the encoder's latent space. Crucially, it argues that this separability is an artifact of corrupted input and can be constructively leveraged for mitigation rather than suppressed.
Bias Mitigation Strategy: A novel subgroup-conditioned decoding strategy is proposed that achieves equitable performance without clean labels, outperforming existing fairness baselines (Adversarial Debiasing, Domain-Invariant Alignment, Fairness Regularization) which fail to address the root cause of annotation bias.

Results

The framework was evaluated on three datasets:

CelebAMask-HQ: Controlled demographic bias (erosion/dilation applied to female/male masks).
PhC-U373: Semi-synthetic annotation style bias (detailed vs. coarse boundaries).
ISIC Skin Lesions: Real-world clinical bias (systematic over-annotation on light skin tones).

Key Findings:

Detection: Standard metrics (Dice/IoU) often fail to detect bias or provide inverted fairness diagnoses (e.g., the group with the highest observed Dice was actually the most affected by annotation errors). In contrast, the CL audit correctly identified the biased subgroup and the nature of errors (omission vs. commission) across all datasets, even without clean labels.
Mitigation: Traditional invariance-based methods (Adversarial, MMD, CORAL) yielded negligible improvements, as they suppress the demographic signal without correcting the underlying annotation style. The proposed subgroup-conditioned approach significantly reduced performance gaps (e.g., reducing the Dice gap from ~12% to ~0.3% on CelebAMask-HQ at 100% bias) and restored symmetry in error rates.
Feature Separability: Experiments confirmed that as label bias increases, subgroup separability in the feature space increases monotonically, validating the hypothesis that bias corrupts the representation space.

Significance and Claims

The paper claims to address a critical gap in algorithmic fairness for segmentation by shifting the focus from model-centric fairness (assuming clean data) to data-centric bias auditing. It argues that:

Standard metrics are insufficient for fairness evaluation in segmentation because they mask the directional nature of annotation errors.
Label bias is not merely random noise but a systematic, group-conditional geometric corruption that alters the network's representation space.
Constructively leveraging subgroup separability via conditional decoding is a more effective strategy for mitigation than attempting to enforce invariance, which can lead to "leveling down" performance.
The proposed framework enables reliable detection and mitigation of bias in real-world scenarios where clean ground truth is unavailable, offering a principled approach to auditing annotation pipelines in safety-critical applications.

The authors acknowledge limitations, noting that the audit is vulnerable to extreme (100%) uniform bias where disagreement between observed and predicted labels diminishes, and that the mitigation strategy relies on the assumption that at least one subgroup retains relatively unbiased labels. Future work is suggested to extend the framework to non-uniform morphological operations and other dense prediction tasks.