Counterfactual Explanations and the Scope of Contestability

Technical Summary: Counterfactual Explanations and the Scope of Contestability

Problem Statement
The automation of consequential decisions via opaque machine learning (ML) models threatens individual agency. While the "right to explanation" is often invoked to restore this agency, particularly through the lens of the General Data Protection Regulation (GDPR), there is a lack of conceptual clarity regarding what it means for explanations to enable the specific ability to contest algorithmic decisions. Existing literature often conflates contestability with related but distinct concepts such as justification (evaluating the model's overall deployment) and recourse (providing forward-looking guidance to alter future outcomes). Furthermore, while counterfactual explanations (e.g., "your loan would be approved if your income were $5,000 higher") are the standard technical approach for recourse, it remains unclear whether they are sufficient to support the backward-looking, adversarial task of contesting a past decision. The paper identifies a "pessimistic" gap: counterfactuals may fail to detect underlying errors (such as data corruption, generalization failures, or discrimination) due to the "Rashomon problem" (multiple valid counterfactuals exist) and the high domain knowledge required to interpret them.

Methodology
The authors employ a conceptual and analytical methodology rather than an empirical or experimental one. The paper proceeds in three stages:

1. Conceptual Taxonomy: The authors develop a tripartite distinction between justification, recourse, and contestability.
   • Justification is defined as the epistemic and moral warrant for deploying a model for a population (forward/backward-looking, expert-driven).
   • Recourse is defined as the forward-looking ability to navigate rules to achieve a desired future outcome (layperson-driven, assumes a cooperative setting).
   • Contestability is defined narrowly as the provision of information sufficient for a decision-subject to demand the revocation of a specific past decision (backward-looking, adversarial setting).
2. Failure Mode Analysis: The authors systematically analyze three categories of algorithmic failure to test the efficacy of counterfactual explanations for contestability:
   • Discrimination: Analyzing whether counterfactuals can reveal reliance on inadmissible features (e.g., race, gender).
   • Failure to Generalize: Analyzing whether counterfactuals can reveal when a model's learned patterns do not apply to a specific individual's unique context (e.g., a vegan with thalassemia).
   • Other Errors: Analyzing whether counterfactuals can detect clerical errors or data inaccuracies.
3. Proposal of Ameliorative Strategies: Based on the identified limitations, the authors propose a modified procedural framework involving multi-shot interactions and institutional support.

Key Contributions

1. Definition of Contestability: The paper establishes a precise definition of contestability as the provision of information sufficient to trigger a review or revocation of a decision. It explicitly demarcates this from justification (which concerns the model's reliability for a population) and recourse (which concerns actionable steps for future success).
2. Critical Analysis of Counterfactuals: The authors demonstrate that counterfactual explanations are not a "silver bullet" for contestability. Key findings include:
   • The Rashomon Problem: Multiple counterfactuals can explain a decision change. In an adversarial contestability context, decision-makers might selectively provide "minimal" counterfactuals that obscure discriminatory or erroneous factors (e.g., providing a recourse-style counterfactual about education rather than a contestability-relevant one about race).
   • Inadmissible Features: Counterfactuals useful for recourse (changeable features) are often disjoint from those useful for contestability (inadmissible features like protected attributes).
   • Epistemic Barriers: Detecting errors via counterfactuals often requires significant domain knowledge (e.g., understanding medical conditions or statistical generalization limits) that lay decision-subjects lack.
   • Inefficiency for Data Errors: Counterfactuals are inefficient for detecting clerical errors or outdated data, as they rely on the model's internal logic rather than verifying input data integrity.
3. Proposed Framework for "Saving" Contestability: The paper argues that counterfactuals can only serve contestability if accompanied by specific procedural modifications:
   • Multi-Shot, Decision-Subject-Initiated Approach: Instead of receiving a single, static explanation, decision-subjects should be allowed to query the model multiple times to test specific hypotheses (e.g., "What if I were a different race?" or "What if my income was different?"). This shifts power from the institution to the subject and mitigates "explanation hacking."
   • Institutional Support (Case Workers): Due to the high cognitive load and domain knowledge required to interpret counterfactuals effectively, the authors propose the involvement of "case workers" (experts in law, domain, or statistics) to guide decision-subjects in formulating queries and interpreting results.
   • Enhanced Model Cards: Model documentation should be extended to provide summary statistics on feature relevance to help subjects frame their queries.

Results and Claims
The paper concludes that the current standard use of counterfactual explanations is insufficient for robust contestability. The "pessimistic" interim conclusion is that without structural changes, counterfactuals risk being a low-cost, standalone solution that fails to address the complexity of algorithmic errors.

However, the authors claim that contestability can be preserved through a "modest attempt" to preserve individual agency. Their proposed solution is not a technical fix to the models themselves, but a procedural one: a multi-shot approach combined with expert support. They argue that while this increases costs (in terms of privacy budgets for queries and the need for case workers), it is necessary to avoid the alternative of stripping individuals of agency entirely. The paper does not claim to solve the "right to explanation" debate definitively but offers a technically and empirically tractable path forward for making contestability a functional reality.

Significance
The paper's significance lies in its rigorous conceptual separation of contestability from recourse, revealing that technical solutions optimized for one (recourse) may be inadequate or even counterproductive for the other (contestability). By highlighting the "Rashomon problem" in an adversarial context and the epistemic burden placed on laypeople, the authors challenge the assumption that counterfactuals are inherently user-friendly for legal or ethical contestation. Their proposal for a multi-shot, supported interaction model provides a concrete, albeit resource-intensive, framework for policymakers and system designers to consider when implementing "right to explanation" safeguards.