Contestability without recourse: post-decision safeguards for disabled learners in AI-supported educational decision-making

Technical Summary: Contestability without Recourse

Problem Statement
The paper addresses a critical asymmetry in the literature concerning Artificial Intelligence (AI) in education, specifically regarding learners with disabilities and special educational needs (SEN). While research has extensively mapped algorithmic bias, fairness, and the technical production of decisions, it has largely displaced the question of contestability: what recourse a learner or their representative has after a consequential decision is made. The author argues that a decision can be demonstrably fair (in terms of its production) yet leave the affected subject with no mechanism to change it. The core problem is the absence of defined post-decision safeguards—mechanisms allowing a learner to challenge, correct, or overturn an AI-supported educational decision.

Methodology
The study employs a systematic review and corpus analysis of research literature, structured as follows:

• Corpus Construction: A search was conducted across 12 bibliographic sources (including Scopus, ERIC, and Crossref) using terms related to "artificial intelligence," "disability/SEN," and "education." Crucially, the search terms excluded concepts related to fairness, bias, appeal, or contestation to ensure the corpus was not biased toward either side of the comparison.
• Selection Criteria: From an initial pool of 1,690 records, 727 were screened by title/abstract, and 138 full-text reports were obtained and analyzed. The inclusion criteria required verification of all three core concepts (AI, disability, education) in the metadata.
• Measurement Protocol: The analysis measured the proximity of specific concepts to "decision" terms (within a 240-character window).
  • Pre-decision concepts: Bias, fairness, ethics, privacy, accuracy, explainability, consent.
  • Post-decision mechanisms: Human reconsideration, data correction, appeal, redress/remedy, and explanation-for-challenge.
• Definitions and Exclusions: The paper rigorously defines "post-decision safeguards" to exclude concepts often conflated with recourse, such as:
  • Explainability: Providing an account of the decision without a route to challenge.
  • Human Oversight: Teacher or clinician involvement in the decision-making process (which remains an operator-side control, not a learner-right).
  • Pre-deployment Audits: Checks occurring before any decision exists.
  • Consent: Permissions for data collection rather than remedies for outcomes.
• Validation: Two independent reviewers assessed the 138 full texts for the presence of learner-invocable safeguards. A secondary, targeted search was also conducted to determine if a literature on recourse for disabled learners exists outside the AI-specific corpus.

Key Contributions

1. Conceptual Distinction: The paper formally distinguishes contestability (the capacity to challenge a decision) from explainability and fairness. It posits that fairness concerns how a decision is produced, whereas contestability concerns what the subject can do afterwards.
2. Operational Definition: It provides a precise, five-part definition of a "post-decision safeguard" (Human reconsideration, Data correction, Appeal, Redress, and Explanation-for-challenge) that is specific enough to be empirically measured in research reports.
3. Legal Gap Analysis: The paper analyzes the General Data Protection Regulation (GDPR), the EU AI Act, and the UN Convention on the Rights of Persons with Disabilities (UNCRPD). It concludes that while these instruments presume a right to contest, none specify a school-level procedure for a learner to initiate reconsideration.
4. Empirical Measurement: It quantifies the disparity in research attention, demonstrating that the literature focuses heavily on decision quality (bias/fairness) while ignoring decision recourse.

Results
The analysis of the 138-document corpus yielded the following findings:

• Pre-decision Dominance: Terms denoting "bias" or "fairness" appeared in proximity to a decision term in 74.6% of documents. "Explainability" appeared in 19.6%.
• Post-decision Absence: Terms denoting "appeal," "challenge," or "due process" appeared in proximity to a decision term in 0.0% of documents.
• Detached Vocabulary: While terms like "redress" or "human reconsideration" appeared in the documents, they were consistently detached from the decision context. They appeared in the abstract or general discussion but were never linked to the specific mechanism of a learner challenging an outcome.
• Teacher-Centric "Oversight": The few instances of "human reconsideration" (6.5% of documents) described capabilities held by teachers or clinicians (e.g., manual override), not rights held by the learner or their family.
• Targeted Search Findings: A separate search for "recourse" and "disability/education" identified 102 relevant records. However, none of these titles mentioned AI, algorithms, or machine learning. This indicates that the literature on educational recourse for disabled learners and the literature on AI in education are distinct, non-overlapping fields.

Significance and Claims
The paper claims that the field of AI in education has developed sophisticated standards for evaluating how decisions are made but has failed to address what happens after they are made.

• The "Fairness" Fallacy: The author argues that focusing on fairness is insufficient; a fair algorithm can still produce a decision that is unchangeable for the affected learner.
• Procedural Necessity: The paper asserts that contestability is a procedural requirement, not a technical one. It requires ceding power to the learner (or their representative) to initiate a review, a step that current "human-in-the-loop" models do not provide.
• Research Blind Spot: The absence of recourse in the literature is not due to a lack of scholarly attention to disability rights generally, but rather a failure to connect that scholarship with AI-supported decision-making.
• Modest Scope: The author explicitly states that their findings describe the research literature, not necessarily the actual practices of schools. They acknowledge that effective reconsideration practices may exist in schools but remain undocumented, or conversely, that statutory rights may exist but be ineffective in practice. The paper concludes that the research community must shift its focus from solely optimizing decision production to establishing mechanisms for post-decision challenge.