AI Governance - Social Contestability Framework-vFinal
This paper argues that legitimate AI governance requires shifting focus from internal compliance artifacts to "social contestability"—the ability of affected individuals to understand, challenge, and correct AI-driven decisions—and proposes five institutional conditions and a lifecycle test to operationalize this framework across major governance instruments and real-world use cases.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where a computer program decides whether you get a job, a loan, or medical treatment. In recent years, governments and companies have built a vast system of rules to make sure these programs behave well. They create checklists, write reports, and run audits to prove their software is safe and fair. These rules are important, but they often focus on what the company does inside its own offices. They check if the paperwork is complete, but they rarely ask if the person on the receiving end can actually speak up when something goes wrong. This gap is the heart of a new study by Shiraz Aslam, a researcher at Banque Saudi Fransi, who argues that having a rulebook is not enough if the people affected by a decision cannot challenge it.
The core idea the paper explores is called "social contestability." In simple terms, this means that if a computer makes a decision that changes your life, you must have a real way to find out it happened, understand why it happened, and ask a human being with actual power to fix it if it is wrong. It is not enough for a company to say, "We have a human in the loop," if that human is just a rubber stamp who cannot say no. The study suggests that true fairness requires a direct line of communication between the person harmed and an institution that can reverse the decision. Without this, a company can be perfectly compliant with all the laws and still leave a person with no recourse.
To figure out how to build this safety net, the researcher looked at four major sets of rules used around the world today: the European Union's AI Act, a risk management guide from the US National Institute of Standards and Technology, the principles from the Organisation for Economic Co-operation and Development, and a US government blueprint for an AI Bill of Rights. The study compared these documents to see what they actually allow a regular person to do. The analysis found a recurring problem: these frameworks are excellent at telling companies how to manage their own risks, but they are often silent on how a person can fight back against a bad outcome. A company might have a perfect audit trail showing they checked their work, yet the person denied a loan still has no clear path to an authorized human who can change their mind.
The researcher then examined five real-world examples where automated systems caused harm to see exactly where the system broke down. One case involved a hiring tool that rejected female applicants because it learned from biased past data. Another involved a health algorithm that discriminated against Black patients because it used medical costs as a shortcut for medical need. There was also a grading system in the UK that overrode teachers' judgments with a statistical formula, and a government debt scheme in Australia that forced citizens to prove they were innocent of errors they did not make. In every case, the institution had rules and documentation, but the people affected could not effectively question the decision. The study found that in each failure, the system lacked a specific set of conditions that would have allowed the person to contest the outcome.
From these failures, the paper proposes five specific conditions that must be in place for a system to be truly contestable. First, the rules must be stricter for decisions that cause more harm; a simple chatbot error needs less protection than a decision to deny someone healthcare. Second, the humans who review these decisions must have the real power, time, and information needed to say no to the computer. Third, the company must explain the decision in plain language that tells the person exactly what to do if they disagree. Fourth, it must be clear who is responsible for the mistake, so the person does not get bounced between different departments. Finally, the system must be checked to ensure that the burden of fixing errors does not fall unfairly on vulnerable groups.
The study introduces a practical test that organizations can use at every stage of a project, from the initial idea to the final product. This test asks specific questions like, "Do we have a plan to let people appeal?" and "Do our reviewers have the authority to override the system?" When the researcher applied this test to the Australian debt scandal, it revealed that while the government had followed many internal rules, they had failed the contestability test at almost every step. They had not tested if people could understand the notices, they had not given humans the power to stop the debt notices before they were sent, and they had not assigned clear responsibility when the system started failing. The test showed that the problem was not a lack of rules, but a lack of a working connection between the person and the authority that could fix the problem.
The paper concludes that the goal of AI governance should not just be to prove a system is safe on paper, but to preserve the human ability to say no. It argues that we must shift our focus from internal company documents to the relationship between the institution and the citizen. If a system cannot be challenged, corrected, or stopped by the people it affects, then it is not truly governed, no matter how many checklists the company has filled out. The study does not claim to have solved every problem in artificial intelligence, but it provides a clear, practical way to check if the people on the other end of the screen have a voice. It suggests that the true measure of a fair system is not whether it follows the rules, but whether it allows those it impacts to hold it accountable.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.