Trade-offs Between Individual and Group Fairness in Machine Learning: A Comprehensive Review

Technical Summary: Trade-offs Between Individual and Group Fairness in Machine Learning

Problem Statement
Algorithmic fairness has become a critical concern in high-stakes decision-making systems (e.g., criminal justice, hiring, healthcare). The literature has traditionally bifurcated into two distinct, often conflicting notions of fairness: Group Fairness (GF) and Individual Fairness (IF).

• Group Fairness focuses on parity of outcomes or error rates across demographic subpopulations defined by sensitive attributes (e.g., demographic parity, equalized odds).
• Individual Fairness emphasizes the consistent treatment of similar individuals, regardless of group membership, typically formalized via Lipschitz continuity or similarity metrics.

While both aim for equitable treatment, they are frequently mathematically incompatible. Enforcing strict group parity often requires treating similar individuals differently based on group membership to correct distributional imbalances, thereby violating individual fairness. Conversely, enforcing strict individual consistency can preserve or reproduce group-level disparities. Existing surveys largely address these notions in isolation or categorize methods by their position in the ML pipeline (pre-, in-, post-processing) without explicitly analyzing the trade-offs between GF and IF. This paper addresses the gap in systematically reviewing methods that jointly manage these competing objectives.

Methodology
The authors conducted a systematic literature review to identify studies that explicitly integrate both GF and IF within a unified framework.

• Search Strategy: A Boolean query was executed across Web of Science, IEEE, and arXiv, combining terms related to individual fairness (e.g., "consistency," "Lipschitz") and group fairness (e.g., "demographic parity," "equalized odds").
• Filtering Criteria: Papers were selected only if they formalized a joint optimization, constraint set, or evaluation framework that simultaneously accounts for both fairness notions. Papers that merely mentioned both concepts in motivation without technical integration were excluded.
• Taxonomy Construction: Instead of relying solely on the standard pipeline-stage taxonomy (pre/in/post-processing), the authors developed a fine-grained taxonomy based on the fairness mechanisms employed. This approach groups methods by how they formulate and manage the GF-IF trade-off, regardless of the specific pipeline stage.

Key Contributions and Taxonomy
The paper provides a comprehensive review of hybrid fairness approaches, organizing them into three primary mechanistic families:

1. Strategies based on Representation or Data Modification:

   • Data Transformation: Modifies input data (e.g., label correction, feature editing) to balance joint distributions of sensitive groups and outcomes while preserving local neighborhood structures.
   • Reweighting and Resampling: Adjusts instance-specific weights to approximate a target distribution where sensitive attributes and outcomes are independent, often using Shapley values to assess individual contributions to disparity.
   • Representation and Embedding: Maps individuals into a latent space where similar observations are grouped. This enforces IF by construction (indistinguishable within clusters) while promoting GF through constraints on representation recoverability or cluster composition.
   • Augmentation and Contrastive: Generates synthetic counterfactual or contrastive samples (varying sensitive attributes) to enforce invariance or smoothness in predictions, addressing both local consistency and group disparities.

2. Strategies based on Regularization or Optimization Objectives:

   • Regularization: Augments the loss function with terms penalizing unfair behavior at both group and individual levels (e.g., $\lambda_{GF}\Phi_{group} + \lambda_{IF}\Psi_{individual}$). This allows navigation of the Pareto frontier between accuracy, GF, and IF.
   • Metric-based Approaches: Embeds geometric structure directly into the loss function, penalizing prediction discrepancies for pairs of individuals deemed similar by a predefined metric. This enforces Lipschitz continuity, which can indirectly satisfy group fairness under specific conditions.
   • Adversarial Learning: Trains a predictor to minimize prediction loss while an adversary attempts to recover sensitive attributes. Extensions incorporate pairwise similarity penalties or counterfactual losses to ensure local consistency alongside group invariance.

3. Strategies based on Local or Post-hoc Adjustments:

   • Thresholding: Introduces group-specific decision thresholds to align with criteria like equalized odds. Extensions incorporate similarity metrics to partially align with IF.
   • Reject-option: Operates in an "ambiguity region" near the decision boundary, selectively modifying predictions to reduce disparities and correct local inconsistencies simultaneously.

Results and Empirical Observations
The review analyzed the selected literature regarding implementation and benchmarking:

• Pipeline Distribution: In-processing methods are the most prevalent (60%), followed by pre-processing (25%) and post-processing (19%).
• Dominant Mechanisms: Regularization is the most common sub-taxonomy, followed by metric-based approaches and representation learning.
• Benchmarking: There is a heavy reliance on a narrow set of datasets (Adult, COMPAS, German Credit), which appear in approximately 20% of studies. This raises concerns about generalizability to high-dimensional domains like Computer Vision or NLP.
• Computational Cost: Computational efficiency is rarely a primary focus. While some methods are noted as computationally demanding (e.g., requiring hours of training or iterative retraining), few provide systematic analyses of the trade-offs between fairness objectives, accuracy, and computational cost.
• Reproducibility: Only 56% of the reviewed methods provide publicly accessible code, limiting independent verification and practical adoption.

Significance and Claims
The paper claims to be the first systematic review that explicitly analyzes methods managing the trade-offs between GF and IF across definitions, algorithms, and evaluation settings. Its significance lies in:

1. Synthesizing Hybrid Approaches: It moves beyond studying GF and IF in isolation to clarify how they can be jointly operationalized.
2. Clarifying Trade-offs: It highlights that the conflict between these notions is not merely a technical artifact but reflects deeper normative tensions, and that existing methods offer different ways to navigate the Pareto frontier between them.
3. Identifying Gaps: The authors modestly note that the field is at an early stage of maturity. They identify critical gaps, including the lack of systematic evaluation of computational costs, the scarcity of studies in diverse domains, the absence of guidance on parameter selection for real-world constraints, and the strong assumption of fully labeled sensitive attributes during training.
4. Future Directions: The paper suggests that future work should focus on more flexible frameworks (beyond rigid parity), integration with causal and legal perspectives, and methods that operate under realistic constraints (e.g., censored data, intersectionality).

The authors conclude that while GF and IF are often mathematically incompatible, methods that explicitly manage these trade-offs represent a crucial step toward more responsible and transparent algorithmic decision-making, providing a foundation for designing algorithms that offer reliability at both individual and group levels.