Technical Summary: Ensuring Fairness with Transparent Auditing of Quantitative Bias in AI Systems

Problem Statement

The rapid integration of Artificial Intelligence (AI) into critical decision-making processes—ranging from healthcare diagnostics to criminal justice sentencing—has introduced significant risks of algorithmic bias. These biases, stemming from inherent flaws in training data or algorithmic design, can lead to unfair outcomes that amplify systemic injustices and perpetuate discrimination against specific demographic groups. A prominent example cited is the COMPAS system used in the U.S. criminal justice system, which was found to violate fairness standards (specifically equalized odds) by disproportionately favoring racial majority groups over minorities. While various statistical metrics exist to quantify fairness, there is a lack of accessible, transparent, and systematic tools for third-party auditors to rigorously examine AI systems using these metrics.

Methodology

The authors propose a comprehensive auditing framework designed to facilitate the systematic examination of AI systems through a "transparent white-box and statistics-based approach." The methodology is built upon the following core components:

1. Formal Definitions of Fairness: The framework adopts a unified mathematical notation based on the work of Pessach and Shmueli [1] to define a wide array of fairness measures. These include:

   • Disparate Impact & Demographic Parity: Assessing selection rates and representation across groups.
   • Conditional Parity: Accounting for "legitimate" attributes (e.g., prior criminal history) alongside protected attributes.
   • Accuracy-Based Metrics: Including Overall Accuracy Equality, Equalized Odds, Equal Opportunity, Predictive Equality, and Predictive Parity.
   • Calibration and Balance Metrics: Such as Equal Calibration, Positive Balance, and Negative Balance, which consider score distributions and expected values.

2. Abstraction via Predicates: To ensure flexibility and applicability across diverse contexts, the framework abstracts key concepts into programmable predicates:

   • Privileged Predicate ($R$): Defines the privileged group (e.g., race = Caucasian).
   • Positive Predicate ($\hat{P}$): Determines if a prediction is positive.
   • Ground Truth Predicate ($T$): Identifies the actual outcome.
   • Score Predicate ($\hat{S}$): Extracts predicted scores (for non-binary outputs).
   • Legitimate Predicate ($L$) & Calibration Predicate ($C$): Handle conditional attributes and score ranges.

3. Implementation: The framework is implemented as an open-source Python package (fairness-checker). It supports two modes of operation:

   • CSV Input: For auditing existing prediction results.
   • Model Input: For auditing black-box models directly (preferred to prevent data fabrication).
     The tool calculates the probability differences or expected value differences between privileged and unprivileged groups for each metric, comparing them against a user-defined threshold ($\epsilon$).

Key Contributions

• A Unified Auditing Framework: The paper presents a systematic framework that allows third-party auditors, developers, and the public to evaluate AI systems against multiple fairness criteria simultaneously.
• Transparent Tooling: The authors developed and open-sourced a Python tool that implements these statistical metrics. The tool prioritizes transparency by offering access to source code and utilizing a white-box approach where definitions are explicitly defined by the user via predicates.
• Flexibility: By abstracting fairness definitions into predicates, the framework can adapt to various data types (binary, continuous scores) and complex scenarios (e.g., generative model outputs) without requiring fundamental changes to the core logic.

Results

The framework was applied to the ProPublica COMPAS dataset (over 10,000 defendants in Florida) to validate its utility. The analysis yielded the following findings:

• Racial Bias Confirmation: When analyzing the "African-American" group as the unprivileged class, the system failed multiple fairness criteria, including Demographic Parity, Conditional Statistical Parity, Mean Difference, and Equalized Odds (both true and false positive rates). This aligns with the original ProPublica investigation.
• Metric Nuance: The analysis revealed that while the system passed "Disparate Impact" (a ratio-based metric), it failed "Demographic Parity" (a difference-based metric). The authors note that Disparate Impact is less suitable here because a "positive" prediction in COMPAS (high risk) is a disadvantage, whereas the metric assumes a positive outcome is advantageous.
• Group Comparisons:
  • Race: African-American defendants were treated more unfairly than Caucasian or Hispanic defendants across most metrics.
  • Age: The "25–45" age group received relatively fair treatment. However, the "<25" group showed higher false positive rates (worse Predictive Equality), while the ">45" group showed worse performance in both Equal Opportunity and Predictive Equality.
  • Accuracy: Overall Accuracy Equality was low for Asian and Native American groups, attributed to small sample sizes in the dataset (32 and 18 rows, respectively).
• Model Comparison: A simple model trained on the dataset exhibited higher unfairness (Equalized Odds of 0.35/0.46) compared to the original COMPAS scores (0.23/0.22), demonstrating the tool's ability to detect bias in custom-trained models.

Significance and Claims

The paper claims that its primary significance lies in making comprehensive statistical fairness auditing accessible and transparent. By providing a tool that relies on widely accepted scientific definitions and offers a flexible abstraction layer, the authors enable independent verification of AI systems.

The authors emphasize that fairness is a subjective social construct, but statistical metrics provide a necessary objective baseline for discussion. The framework does not claim to solve bias automatically but serves as a reference tool for auditors to identify violations. The results on the COMPAS dataset serve to validate the tool's accuracy, as it successfully reproduced the findings of the ProPublica investigation regarding racial disparities. The open-source nature of the tool is highlighted as a key factor in promoting accountability and transparency in the deployment of AI systems.