Unbiased Model Prediction Without Using Protected Attribute Information

1. Problem Statement

Deep learning models often exhibit bias, providing disparate performance across different demographic subgroups (e.g., gender, race, age). This bias arises because models inadvertently learn and encode features correlated with protected attributes (sensitive data like race or gender).

• The Limitation of Current Solutions: Most existing debiasing algorithms require a priori knowledge of protected attributes during training to enforce fairness constraints.
• The Real-World Constraint: In many practical scenarios, protected attribute information is unavailable due to privacy laws, legal restrictions, or the sheer difficulty of annotating large datasets with sensitive labels.
• The Core Challenge: How to mitigate bias and achieve fair predictions when the protected attributes are unknown during the training phase.

2. Methodology: Non-Protected Attribute-based Debiasing (NPAD)

The authors propose the NPAD algorithm, which mitigates bias without accessing protected attribute labels. Instead, it leverages non-protected attributes (e.g., "wearing glasses," "smiling," "big nose") that are often available in datasets and may correlate with the unknown protected attributes.

The methodology consists of two main stages:

Stage 1: Intelligent Non-Protected Attribute Selection

Since not all non-protected attributes are equally useful for debiasing, the algorithm intelligently selects a subset of attributes that exhibit high disparity with respect to the target attribute.

1. Disparity Computation: A model is trained to predict a target attribute ($A_i$). This model is then evaluated across the classes of other non-protected attributes. The disparity (standard deviation of classification accuracy) is calculated for each non-protected attribute.
2. Selection Logic: Attributes showing the highest disparity are selected first, as they likely encode information correlated with the protected attribute.
3. Independence Check: To ensure diversity and reduce redundancy, subsequent attributes are selected based on statistical independence (using the $\chi^2$ test) from previously selected attributes.

Stage 2: Bias Mitigation via Novel Loss Functions

Once the non-protected attributes are selected, the model is optimized using a Convolutional Neural Network (CNN) feature extractor combined with two novel loss functions:

1. Debiasing via Attribute Cluster Loss (DACL):

   • Concept: The classes of the target attribute ($A_i$) are combined with the classes of the selected non-protected attributes to form $2^{n+1}$ unique clusters (where $n$ is the number of selected non-protected attributes).
   • Mechanism: DACL acts as a supervision signal to separate these clusters in the feature space. It minimizes the intra-class distance (by reducing the moving standard deviation of feature representations) and maximizes the inter-class distance (by increasing the distance between moving averages of different clusters).
   • Goal: To force the model to learn features that distinguish these specific attribute combinations, thereby breaking the correlation with the unknown protected attribute.

2. Filter Redundancy Loss (FRL):

   • Concept: Standard CNN filters often learn highly correlated (redundant) features, which can hinder fairness.
   • Mechanism: FRL minimizes the dot product between mean-normalized filters. It encourages the model to learn non-redundant, diverse features.
   • Goal: To boost overall model performance and feature expressiveness, acting as a supplement to DACL.

Final Objective Function:
$$\mathcal{L} = \lambda_1 \mathcal{L}_{C} + \lambda_2 \mathcal{L}_{F}$$
Where $\mathcal{L}_{C}$ is DACL and $\mathcal{L}_{F}$ is FRL.

3. Key Contributions

1. NPAD Algorithm: A novel framework for bias mitigation that operates without protected attribute information, utilizing auxiliary non-protected attributes instead.
2. Novel Loss Functions:
   • DACL: Provides class-specific supervision to separate clusters formed by combining target and non-protected attributes.
   • FRL: Reduces feature redundancy to enhance model robustness and fairness.
3. New Evaluation Metric (OPE): The authors introduce Overall Performance Equality (OPE).
   • Problem with existing metrics: Metrics like "Difference in Equality of Opportunity" can be misleading if a model predicts all samples as one class (resulting in zero disparity but poor performance).
   • Solution: OPE measures the disparity in overall performance (considering both true/false positives and negatives) across subgroups, ensuring that fairness does not come at the cost of accuracy.
4. Comprehensive Evaluation: Extensive experiments on LFWA and CelebA datasets for facial attribute prediction.

4. Experimental Results

The paper evaluates the NPAD algorithm against a Baseline Model Training (BMT), a Protected Attribute-based Debiasing (PAD) method (which assumes ground truth is known), and the "Learning from Failure" (LfF) algorithm.

• Performance vs. Fairness:
  • NPAD significantly outperforms BMT in both accuracy and fairness (lower Degree of Bias - DoB, and lower OPE).
  • NPAD achieves performance comparable to, and in some cases better than, the PAD algorithm (which has access to protected labels).
  • Specifically, using two selected non-protected attributes (NPAD Two) often yields the best results, provided the attributes are highly correlated with the protected attribute.
• Ablation Studies:
  • Removing DACL causes a significant drop in accuracy and an increase in bias, proving its necessity for cluster separation.
  • Removing FRL reduces performance, confirming its role in feature expressiveness.
  • Random selection of non-protected attributes performs worse than the intelligent selection strategy used in NPAD.
• Cross-Dataset Generalization: The algorithm demonstrates strong generalizability when models trained on LFWA are tested on CelebA and vice-versa.
• Comparison with ARL: NPAD outperforms Adversarially Reweighted Learning (ARL), another method that does not use protected attributes, by focusing on reducing disparity across all groups rather than just the worst-performing group.

5. Significance and Conclusion

This research addresses a critical gap in AI fairness: the inability to deploy debiasing techniques when sensitive data is legally or practically unavailable.

• Practical Applicability: By relying on non-protected attributes (which are often easier to annotate or already present in datasets), the NPAD algorithm makes fairness achievable in real-world applications where privacy laws (like GDPR) or lack of data prevent the use of protected labels.
• Theoretical Insight: The work demonstrates that the correlation between non-protected and protected attributes can be positively exploited to mitigate bias, rather than just being a source of the problem.
• Metric Innovation: The introduction of OPE provides a more robust way to evaluate fairness, preventing the "unfairly accurate" or "unfairly biased but low-performing" pitfalls of existing metrics.

In summary, the paper presents a robust, label-efficient approach to fair deep learning, proving that intelligent utilization of auxiliary data can effectively replace the need for sensitive protected attribute labels during model training.