💻 computer science

Applied and Filtered: An End-to-End Algorithmic Fairness Audit of A Public Employment Agency

This paper presents the first independent end-to-end fairness audit of Barcelona Activa's semi-automated hiring system, revealing that while aggregate gender parity exists, significant disparities persist across salary levels, age groups, and gender identities due to complex interactions between automated processing, human discretion, and vendor opacity that are invisible to traditional model-level assessments.

Original authors: Gemma Galdón-Clavell

Published 2026-08-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gemma Galdón-Clavell

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a job, but instead of handing your resume to a person, you are entering a giant, high-tech maze. This is the world of algorithmic fairness, a field of science that asks a simple but tricky question: When computers help make big decisions like hiring, do they treat everyone the same, or do they accidentally push certain people to dead ends? For a long time, experts mostly checked the "brain" of the computer—the code itself—to see if it was biased. But this paper argues that checking just the brain is like checking a car engine while ignoring the driver, the road conditions, and the traffic rules. It turns out that even if the engine is perfect, the car can still crash if the driver is distracted or the map is wrong. This matters to everyone because hiring algorithms are becoming the gatekeepers to our livelihoods, and if they are broken, they can silently lock people out of opportunities without anyone noticing.

This paper is a detective story about a real-life hiring maze in Barcelona, Spain. The researchers, acting as independent auditors, didn't just look at the computer code; they followed 497,000 job seekers through every single step of a five-year journey (from 2017 to 2022) to see where the system dropped the ball. They looked at a semi-automated system where a human analyst uses a third-party software called "TalentClue" to find candidates for employers. Think of it as a relay race with seven legs: the employer posts a job, a human analyst picks the keywords, the computer filters the list, the human picks a shortlist, and finally, the employer hires someone.

The most surprising twist in the story? If you only looked at the final finish line, the race looked fair. The number of men and women who got hired was almost exactly the same. It was like seeing two teams with the same final score and assuming the game was perfectly balanced. But when the auditors rewound the tape and looked at every leg of the race, they found a different story. The "fair" final score was actually hiding a lot of unfairness happening along the way.

Here is where the system started tripping people up:

The Salary Trap: While women and men were hired at similar rates overall, women were much less likely to be shortlisted for mid-level salary jobs (between €15,000 and €24,000). In fact, for these specific jobs, women were shortlisted at a rate of only 7.43% compared to 9.45% for men. It's as if the system had a hidden rule that said, "You can get the job, but maybe not the better job."
The Age Wall: The system seemed to have a blind spot for older workers. People aged 55 and older made up 15.6% of the local workforce, but they were completely absent from the hiring pipeline. It was as if the maze had a wall that no one over 55 could see, let alone climb over.
The "Other" Problem: For candidates who identified as non-binary (neither strictly male nor female), the odds were incredibly slim. They were shortlisted at less than one-third the rate of men. However, the researchers note that there were very few of these candidates in the data (only 285), so while the disparity is clear, they can't say exactly how big the problem is for the whole population yet.
The Education Paradox: Interestingly, people with higher education were less likely to be shortlisted than those with just a high school diploma. Since women tend to have higher education degrees in this dataset, this rule accidentally hurt women's chances even more.

The paper also highlights a major communication breakdown. The people running the hiring agency (Barcelona Activa) didn't actually know how the computer software (TalentClue) decided who to show them. It was like hiring a chef but not being allowed to see the recipe or the ingredients. The agency couldn't check if the computer was biased because the software company kept the "secret sauce" hidden.

Finally, the researchers found that the system wasn't static; it changed over time. The gap between men and women in getting shortlisted got smaller from 2017 to 2022, but only because fewer people of both genders were getting shortlisted overall. It's like a race where the gap between the runners shrinks because everyone is running slower, not because the track has become fairer.

The big takeaway is that you can't just check a computer's code once and call it "fair." You have to watch the whole process—the humans, the data, the software, and the time it takes. If you only look at the final result, you might miss the fact that the system is quietly steering certain groups away from the best opportunities. The author suggests that instead of just doing a one-time check, companies need to keep a constant watch on their hiring systems, like a lighthouse keeping an eye on the waves, to make sure the path to a job is open for everyone, every day.

Technical Summary: Applied and Filtered: An End-to-End Algorithmic Fairness Audit of a Public Employment Agency

Problem Statement

Algorithmic fairness evaluation typically treats AI systems as bounded technical components, assessing model inputs and outputs in isolation from the organizational context in which they operate. This "component-level" approach creates a significant blind spot in high-risk domains like employment, where automated tools are embedded within cumulative, multi-stage decision pipelines. Disparities often arise not from a single algorithmic error, but from the interaction of automated processing, human discretion, data quality, vendor opacity, and downstream decisions. Furthermore, existing regulatory frameworks (e.g., NYC Local Law 144) and academic practices often rely on point-in-time audits that fail to capture the temporal dynamics of deployed systems or the sociotechnical nature of recruitment.

This paper addresses the gap between isolated model assessment and the reality of deployed hiring systems by conducting an independent, end-to-end fairness audit of a semi-automated recruitment pipeline operated by Barcelona Activa, a public employment agency in Spain.

Methodology

The study utilizes a sociotechnical auditing framework applied to five years of operational data (September 2017 – September 2022), comprising approximately 497,000 candidate-vacancy pipeline entries. The audit covers the full recruitment lifecycle rather than a single algorithmic component.

System Architecture and Scope

The audited system involves a seven-stage pipeline:

Vacancy Receipt: Employer submission of job requirements.
Analyst Assignment: A human analyst interprets requirements.
Keyword Selection: Analysts translate requirements into search filters and keywords (no audit trail exists for this step).
Platform Search: Analysts query the third-party TalentClue platform. The platform's internal matching, ranking, and weighting logic is proprietary and opaque to the deployer and auditors.
Filtered Results: The platform returns a subset of profiles; the excluded pool is unrecorded.
Final Review: Human analysts construct shortlists without formal protocols or reasoning records.
Employer Handoff: Employers make final hiring decisions; no post-shortlist outcome data is returned to the agency.

Analytical Approach

The methodology employs a three-phase analysis:

Pre-processing (Representativeness): The demographic composition of the candidate pool is benchmarked against the active labor force of Barcelona (using EPA data) to identify structural exclusion at the entry point.
In-processing (Disparate Impact & Stratification): The audit calculates the Disparate Impact Ratio (DIR) for transitions between pipeline stages. A DIR < 0.80 is treated as a benchmark for adverse impact. Crucially, the analysis moves beyond aggregate metrics to stratified conditional analysis (by sector, salary band, contract type, education, and age) and intersectional analysis (e.g., age × gender). Statistical significance is tested using chi-square or Fisher's exact tests.
Post-processing (Temporal Consistency): Shortlisting rates are analyzed annually (2017–2022) to assess the stationarity of fairness properties over time.

Key Contributions

The paper makes four primary contributions to the field of algorithmic auditing:

End-to-End Operational Audit: It provides the first independent fairness audit of a semi-automated hiring system using longitudinal operational data, extending the scope of assessment from the automated component to the entire recruitment pipeline.
Evidence of Masked Disparities: It demonstrates that aggregate gender parity can coexist with substantial, statistically significant disparities across specific pipeline stages, demographic intersections, salary levels, and sectors.
Vendor–Deployer Information Asymmetry: It documents how the lack of access to a third-party platform's matching logic and evaluation data constrains the deploying organization's ability to govern the system, identifying this opacity as a critical audit finding.
Evidence for Continuous Evaluation: It empirically shows that fairness outcomes are non-stationary, changing over time even without model retraining, thereby arguing against the sufficiency of point-in-time assessments.

Key Results

1. Aggregate Parity Masks Structural Inequality

Aggregate outcomes appear equitable: women constitute 51.5% of registered candidates and 49.8% of hired candidates, a difference that is not statistically significant. A model-output audit limited to these figures would conclude the system is fair. However, disaggregation reveals severe disparities.

2. Pipeline-Specific Disparities

Salary Levels: Women experience adverse impact in mid-salary shortlisting (€15,000–€24,000) with a DIR of 0.786 ( $p < 0.001$ ). While low-salary roles show parity, the gap widens at higher economic stakes.
Sectoral Segregation: Women are underrepresented in shortlists for male-dominated sectors (e.g., Real Estate, Engineering) even when they are present in the matched candidate pool. Conversely, they are overrepresented in sectors like Healthcare.
Contract Type: Full-time roles show adverse impact for women (DIR = 0.758, $p < 0.001$ ), whereas part-time roles show no disparity.
Salary Gaps: Even within the same sectors, women are matched to vacancies with an average minimum salary €1,147 lower than men ( $p < 0.001$ ). This gap persists in 15 of 20 sectors.

3. Intersectional and Demographic Exclusion

Age and Gender: Older women (46–55) face a compounded disadvantage with a DIR of 0.77.
Structural Absence of Older Adults: Candidates aged 55 and over are entirely absent from the pipeline, despite comprising 15.6% of Barcelona's active labor force.
Non-Binary Candidates: Non-binary candidates are shortlisted at a rate of 3.51% compared to 11.89% for men (DIR = 0.295, $p < 0.001$ ). Zero non-binary candidates were forwarded to employers during the audit period. Note: This finding is flagged as indicative due to a small sample size (N=285).
Education Paradox: Candidates with higher education are shortlisted at rates 40–42% lower than high school graduates. Since higher education categories skew female, this mechanism disproportionately affects women.
Country of Origin: Spanish candidates are shortlisted at significantly higher rates than EU (DIR = 0.565) and non-EU (DIR = 0.631) candidates.

4. Temporal Dynamics

The gender gap in shortlisting narrowed from 6.5 percentage points in 2017 to 1.3 percentage points in 2022. However, this convergence coincided with a decline in overall shortlisting rates for both groups (from ~18% to ~9%), suggesting the change may be driven by increased competition rather than substantive fairness improvements. This highlights that fairness properties are non-stationary.

Significance and Claims

The paper argues that model-level assessment alone is insufficient for understanding fairness in deployed systems. The central empirical claim is that the gap between aggregate fairness metrics and pipeline-specific disparities is likely the rule, not the exception, in AI-assisted decision systems.

The author asserts that:

Sociotechnical Context is Critical: Fairness outcomes are shaped by the interaction of automated processing, human discretion, data practices, and organizational procedures.
Opacity is a Barrier: The inability of the deployer (Barcelona Activa) to access the vendor's (TalentClue) internal logic prevents effective governance and independent verification of fairness.
Continuous Monitoring is Necessary: Because system behavior changes over time due to shifting labor markets, analyst practices, and platform updates, periodic audits are necessary but insufficient. A continuous evaluation layer is required to detect emerging disparities in real-time.

The study concludes that effective accountability requires extending evaluation beyond model outputs to the broader sociotechnical pipeline, ensuring deployers have access to necessary information for third-party systems, and establishing mechanisms for continuous monitoring. This work provides empirical support for the shift toward sociotechnical and end-to-end approaches to algorithmic fairness, particularly in the context of emerging regulations like the EU AI Act.