Open Veins of Algorithmic Auditing: Why AI Assessment Lags Behind Its Deployment in the Global South
Drawing on a decade of audit practice in the Global South, this paper argues that the critical gap between rapid AI deployment and governance is driven not by a lack of capacity but by a funding deficit, urging development and philanthropic funders to mandate independent evaluations to address systemic flaws like invalid proxies and structural bias.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where invisible robots are making big decisions about people's lives: deciding if a child needs help, if a patient is sick, or if a farmer gets a loan. These robots are powered by Artificial Intelligence (AI). But here's the catch: just because a robot is smart doesn't mean it's fair, safe, or even working the way its creators say it does. To find out, we need "AI Audits." Think of an audit like a mechanic inspecting a car before you buy it, or a teacher grading a test to see if the answers are actually right. In the wealthy parts of the world (the "Global North"), there are lots of mechanics and teachers checking these AI robots. But in the developing parts of the world (the "Global South"), the robots are running wild without anyone checking them. This paper asks a simple but urgent question: What happens when we let these powerful robots make life-or-death decisions in places where no one is allowed to look under the hood?
The Great Audit Gap: Robots Running Wild in the South
This paper, written by experts from the Eticas Foundation, tells the story of a massive imbalance. While AI is being installed in hospitals, schools, and government offices across Africa, Latin America, and Asia at a breakneck speed, the people who are supposed to check if these systems are safe are almost entirely missing.
The authors found that while billions of dollars are being poured into building AI in these regions, there are fewer than twenty published reports from independent experts actually testing how these systems work in the real world. To put that in perspective: the United Kingdom alone has more AI auditors than the entire Global South combined. It's like building a thousand new bridges in a country but having only one engineer available to check if they are safe, and that engineer is busy checking bridges in a different country.
What the Few Checks We Did Reveal
The authors didn't just count the missing checks; they actually performed a handful of them. They looked at systems in Brazil, Argentina, and other countries, and what they found was scary, but also very clear.
1. The "Poverty Detector" Trap
In one case, a government used an AI to decide which children were at risk and needed social services. The system was supposed to find kids whose rights were being violated. But when the auditors looked closely, they realized the robot wasn't actually looking for abuse. It was looking for poverty. Because the data it was trained on came from poor neighborhoods, the AI decided that being poor was the same as being at risk. It was like a smoke detector that only goes off when you cook toast, ignoring the actual fire. The system was scoring millions of children, many of whom it had never even seen before, based on a flawed idea of what "risk" looked like.
2. The "94% Accurate" Lie
In another example, a hospital in Brazil used an AI to predict which patients were getting sick. The system claimed to be 94% accurate. That sounds amazing, right? But the auditors found that this number was a trick. The AI was so good at predicting that most people would be fine that it got a high score, even though it was missing the specific people who were actually getting sick. It was like a weather app that says "It will not rain" every single day. It's technically right 99% of the time because it rarely rains, but it's useless when a storm is coming. The auditors found that the system was failing to warn doctors about young women who were actually in danger.
3. The "Black Box" Myth
The paper also argues that we don't need to see the robot's secret code (the "black box") to know if it's broken. You don't need to be a mechanic to know a car is broken if it won't start or if it smells like burning rubber. The auditors found that by just watching what the AI did—who it helped and who it ignored—they could spot the problems. The problem wasn't that the code was too hard to understand; the problem was that no one was watching the results.
Why Is This Happening? (It's Not a "Capacity" Problem)
You might think, "Maybe the countries in the Global South just don't have enough smart people or money to check these robots." The paper says: No, that's not it.
The authors found that they could do these checks. They did them with small teams and small budgets. The problem isn't that the work is impossible; the problem is that no one is paying for it.
Think of it like a school. If the principal (the government) doesn't require a teacher to be graded, and the school board (the funders) doesn't pay for a grader, the teacher might never get checked. In the Global South, the people building these AI systems are often funded by big international banks or charities. But these funders usually just ask, "Did you promise to be nice?" They don't say, "Show us the proof that your system actually works."
Because no one is forcing the builders to show their work, and no one is paying for the inspection, the inspections never happen. It's a funding problem, not a talent problem.
The Solution: Put the Check in the Contract
The paper ends with a clear, simple solution. The people who have the power to fix this are the funders.
If a bank or a charity gives money to build an AI system, they should make it a rule: "You cannot get this money unless an independent expert checks your system first." And they shouldn't just check the code; they should check the results. Did the system actually help the people it was supposed to help?
The authors suggest five simple rules for these funders:
- Mandate the check: Make independent audits a requirement for getting money.
- Check the results, not just the math: Look at how the system affects real people, not just if the code looks good.
- Publish the findings: Don't let the results be secret. If a system is broken, the public needs to know.
- Build local teams: Train people in those countries to do the checking, so they don't have to rely on experts from far away.
- Share the tools: Make the checklists and methods open for everyone to use.
The Bottom Line
The paper argues that we are at a crossroads. We can continue to let AI systems run in the Global South without supervision, which means mistakes will keep happening to the people who can least afford them. Or, we can change the rules of the game. By making independent checks a condition of funding, we can turn these "open veins" of data and decision-making into something that is safe, fair, and accountable.
The tools to fix this already exist. The experts are ready. All that's missing is the decision to pay for the inspection. As the authors say, the next decade will be remembered either as a time when the Global South kept building systems it couldn't control, or as the moment it finally started holding them accountable. That choice is up to the people writing the checks.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.