Why Aggregate Accuracy is Inadequate for Evaluating Fairness in Law Enforcement Facial Recognition Systems

1. Problem Statement

The paper addresses a critical gap in the evaluation of facial recognition systems (FRS) deployed in high-stakes law enforcement environments. While these systems often report high aggregate accuracy (overall correctness), this metric is fundamentally flawed for assessing fairness.

• The Core Issue: Aggregate accuracy treats all data points equally, obscuring how errors are distributed across demographic subgroups (e.g., race, age, gender).
• The Consequence: A system can appear highly accurate overall while simultaneously exhibiting severe performance disparities, such as disproportionately high False Positive Rates (FPR) or False Negative Rates (FNR) for specific minority groups.
• The Risk: In law enforcement, these hidden disparities lead to tangible harms: wrongful suspicion/arrests (due to high FPR in specific groups) and missed investigations (due to high FNR), creating systemic bias and ethical violations.

2. Methodology

The paper employs a critical analysis and empirical synthesis approach rather than proposing a new algorithm. The methodology involves:

• Literature Review: Synthesizing existing research on algorithmic bias, specifically focusing on studies (e.g., Buolamwini & Gebru) that demonstrate performance gaps in commercial FRS across demographics.
• Metric Analysis: Deconstructing the mathematical limitations of "Accuracy" as a standalone metric, particularly in the context of class imbalance and heterogeneous populations.
• Empirical Illustration: Utilizing hypothetical and reported data to demonstrate how identical aggregate accuracy scores can mask vastly different subgroup error distributions.
  • Example Data: The paper presents a baseline system with 75.47% accuracy but reveals that False Positive Rates (FPR) for race vary from 20.15% to 35.18%, and for age, FPR varies from 24.53% to 46.01%.
• Operational Risk Assessment: Analyzing the specific consequences of different error types (FP vs. FN) within the legal and security context.
• Framework Proposal: Advocating for Model-Agnostic Auditing, a method that evaluates system fairness using only output data (predictions and confidence scores) without requiring access to proprietary model internals or training data.

3. Key Contributions

The paper makes several technical and conceptual contributions to the field of AI ethics and evaluation:

• Demonstration of Metric Inadequacy: It rigorously argues that accuracy is a "blunt instrument" that fails to capture the distribution of errors, making it unsuitable as a primary reliability indicator for high-stakes AI.
• Subgroup-Level Analysis: It emphasizes the necessity of reporting False Positive Rates (FPR) and False Negative Rates (FNR) stratified by demographic attributes (race, age, gender) rather than relying on global averages.
• Operational Risk Mapping: It explicitly links technical metrics to real-world legal consequences, distinguishing between the harms of false positives (wrongful targeting) and false negatives (operational failure).
• Practical Auditing Framework: It proposes model-agnostic evaluation as a viable solution for auditing proprietary, third-party systems where internal model access is restricted.
• Reframing the Accuracy-Fairness Trade-off: It challenges the notion that accuracy is the supreme goal, arguing that in law enforcement, a marginal reduction in aggregate accuracy is ethically justifiable if it significantly reduces demographic disparity and harm.

4. Key Results and Findings

• Disparity Visibility: The analysis confirms that high aggregate accuracy (e.g., 90% or 75%) can coexist with severe subgroup disparities. For instance, a system might have a low FPR for light-skinned males but a significantly higher FPR for darker-skinned individuals.
• Data Evidence: The paper cites empirical ranges showing that FPR and FNR can fluctuate by over 15-20 percentage points across different demographic groups within the same system, a variance invisible to aggregate metrics.
• Class Imbalance Sensitivity: The study highlights that accuracy is highly sensitive to class imbalance; models often achieve high accuracy by correctly predicting the majority class while failing minority classes, a phenomenon common in biased training datasets.
• Transparency Gap: Current deployment practices often lack transparency, relying on vendor-provided aggregate metrics that hide bias. The paper finds that without subgroup-level auditing, these biases remain unmitigated.

5. Significance and Implications

The paper holds significant implications for the deployment of AI in the public sector:

• Policy and Regulation: It provides a technical basis for regulators to mandate disaggregated reporting (subgroup metrics) as a condition for deploying FRS in law enforcement, moving beyond simple accuracy thresholds.
• Ethical AI Deployment: It shifts the paradigm from "accuracy-centric" to "fairness-aware" evaluation, suggesting that responsible AI requires explicit management of the trade-off between performance and equity.
• Operational Safety: By identifying the specific risks of false positives in minority groups, the paper argues for stricter evaluation protocols to prevent wrongful arrests and preserve civil liberties.
• Future Research Direction: It calls for the development of standardized, interpretable, and model-agnostic auditing tools that can be integrated into operational workflows to ensure continuous fairness monitoring post-deployment.

In conclusion, the paper asserts that aggregate accuracy is a necessary but insufficient condition for evaluating facial recognition systems. True reliability in high-stakes environments requires a comprehensive evaluation strategy that prioritizes subgroup-level error analysis, transparency, and the mitigation of demographic disparities.