Technical Summary: FairBED: A Bayesian Experimental Design Approach to Gathering Fairer Data

1. Problem Statement

Current frameworks for ensuring fairness in machine learning predominantly focus on learning fair models from existing, pre-collected datasets. However, this approach is often fundamentally limited because the data itself may already encode societal biases and unwanted correlations. Interventions such as data reweighting, constrained optimization, or pre-processing "data repair" attempt to mitigate these issues downstream but cannot fully overcome the inherent unfairness of the source data.

The authors argue that the data acquisition process itself should be modified to gather data that is inherently more suitable for training fair predictors. The core challenge is to design a data acquisition strategy that balances two competing objectives:

1. Informativeness: The data must remain highly informative about the target variables of interest to ensure effective downstream prediction and generalization.
2. Fairness: The data must be uninformative about sensitive attributes (e.g., gender, race, skin pigmentation) to prevent downstream models from exploiting these attributes, thereby naturally improving fairness.

2. Methodology

2.1 Conceptual Framework: "Fair Worlds"

The paper introduces the concept of a "fair world" where sensitive attributes (denoted as $\phi$) are causally independent of the target variables (denoted as $\theta$). In such a world, a fair predictor would not need to use $\phi$ to estimate $\theta$. The authors propose measuring the unfairness of a dataset by quantifying how much it violates this independence. Specifically, if a dataset allows information to be transferred between $\theta$ and $\phi$, it is considered unfair. The goal is to guide the experimental design process to gather data that minimizes this information transfer while maintaining utility for $\theta$.

2.2 Fairness Metrics for Data

The authors define two novel metrics to quantify the fairness of a dataset $D$ based on the information it reveals about sensitive attributes:

• Unconditional Fairness ($F_u$): Defined as the negative of the Expected Information Gain (EIG) about $\phi$. It measures how much the data reveals about $\phi$ marginally.
  $$F_u(\pi_d) = -\max_{m \in \mathcal{M}} \text{EIG}^m_\phi(\pi_d)$$
• Conditional Fairness ($F_c$): Defined as the negative of the conditional EIG about $\phi$ given $\theta$. This is a stricter metric that ensures the data does not reveal $\phi$ for any specific value of the target $\theta$, guarding against "explaining away" effects where $\phi$ becomes recoverable once $\theta$ is known.
  $$F_c(\pi_d) = -\max_{m \in \mathcal{M}} \mathbb{E}_{p(\theta)}[\text{EIG}^m_{\phi|\theta}(\pi_d)]$$

Here, $\pi_d$ represents a design policy, and the maximization over a set of models $\mathcal{M}$ ensures robustness against model deficiencies.

2.3 The FairBED Objective

To balance the need for informative data with the need for fairness, the authors construct two FairBED objectives. These objectives maximize the Expected Information Gain (EIG) about the target $\theta$ while penalizing the EIG about the sensitive attribute $\phi$ (or $\phi|\theta$).

• Unconditional FairBED ($R_u$):
  $$R_u(\pi_d) = \text{EIG}_\theta(\pi_d) - \beta \text{EIG}_\phi(\pi_d)$$
• Conditional FairBED ($R_c$):
  $$R_c(\pi_d) = \text{EIG}_\theta(\pi_d) - \beta \mathbb{E}_{p(\theta)}[\text{EIG}_{\phi|\theta}(\pi_d)]$$

In these equations, $\beta > 0$ is a hyperparameter controlling the strength of the fairness enforcement. The authors note that these objectives are defined under a joint model but can be optimized using marginal models for practicality. Crucially, the authors argue that gathering data informative about $\theta$ is necessary for fairness in the real world to avoid generalization gaps that might manifest as indirect unfairness.

2.4 Application to Active Learning

The framework is adapted for Active Learning (AL) settings. In this context, the target $\theta$ corresponds to the pair of test-time inputs and target labels $(x^*, y^*_\theta)$, while the sensitive attribute $\phi$ corresponds to $(x'^*, y^*_\phi)$. The objective becomes a modification of the Expected Predictive Information Gain (EPIG):
$$R_{\text{FairEPIG}}(x) = I(y_\theta; (x^*, y^*_\theta) | x) - \beta I(y_\phi; (x'^*, y^*_\phi) | x)$$
This allows the selection of query points that improve predictions for the target task while deliberately limiting the learnability of the sensitive attribute.

3. Key Contributions

1. Novel Formulation of Data Fairness: The paper shifts the focus from the fairness of a model to the fairness of the dataset itself, defining fairness as the lack of information leakage about sensitive attributes.
2. FairBED Objectives: The introduction of practical Bayesian Experimental Design (BED) objectives that explicitly trade off target informativeness against sensitive attribute informativeness.
3. Theoretical Link to Demographic Parity: The authors prove that if the conditional fairness metric $F_c(\pi_d) = 0$ (i.e., the data conveys no information about $\phi$ given $\theta$), then downstream predictors trained on this data achieve Demographic Parity (DP) under the assumption that the prior over $\theta$ and $\phi$ is separable and the downstream model does not take $\phi$ as an input.
4. Empirical Validation: Demonstration that FairBED consistently achieves superior fairness-accuracy trade-offs compared to random acquisition and conventional BED across multiple datasets and model architectures.

4. Experimental Results

The authors evaluated FairBED on three distinct tasks:

1. Source Location Finding: A synthetic BED benchmark where the goal is to infer a target coordinate ($\theta$) while avoiding learning a sensitive coordinate ($\phi$). Results showed that FairBED designs spread along the target axis while collapsing to the prior mean of the sensitive axis, effectively suppressing learning of $\phi$ with minimal loss in $\theta$ learning.
2. Student Graduation Prediction: A binary classification task predicting graduation status ($y_\theta$) while treating gender ($y_\phi$) as sensitive.
3. Census Income Prediction: Predicting income levels ($>50k$) while treating gender as sensitive.
4. CelebA: A large-scale image dataset predicting "Smiling" while treating "Gender" as sensitive.

Key Findings:

• Fairness-Accuracy Trade-off: FairBED achieved Pareto-optimal fronts that dominated random acquisition and standard EPIG. It significantly reduced the predictability of the sensitive attribute while maintaining high accuracy on the target task.
• Dataset-Level Fairness: Experiments training various downstream models (Random Forest, XGBoost, MLP, SVM) on the same FairBED-acquired dataset showed consistent improvements in Demographic Parity (DP) and Equalized Odds (EO) ratios. This confirms that the fairness improvement is a property of the acquired data, not an artifact of a specific model.
• Complementarity: The paper demonstrates that FairBED can be combined with downstream fairness interventions (e.g., Learned Fair Representations or Reject Option Classification). The combination of FairBED and downstream methods yielded better results than either method alone, suggesting they are additive rather than substitutive.
• Robustness: The benefits of FairBED generalized across different model architectures and label budgets.

5. Significance and Claims

The paper claims that FairBED provides a new perspective on algorithmic fairness by addressing the root cause of bias: the data acquisition process. By formalizing the utility of data to include a penalty for sensitive attribute information, FairBED offers a principled approach to gathering data that is "inherently fairer."

The authors emphasize that:

• Upstream Intervention is Critical: Downstream mitigation strategies are often undermined by biases already present in the data. FairBED addresses this at the source.
• Generalization: The fairness improvements are intrinsic to the dataset, meaning any model trained on this data (even without specific fairness constraints) will naturally exhibit better fairness properties.
• Practicality: The approach is compatible with standard Bayesian Experimental Design and Active Learning frameworks, requiring only the specification of marginal models for target and sensitive attributes, which can be pre-trained or handcrafted even without sensitive labels during the acquisition phase.

The work opens new research directions for fairness-aware data acquisition, suggesting that the path to fairer AI systems begins with how data is gathered, not just how it is modeled.