All Eyes on the Ranker: Participatory Auditing to Surface Blind Spots in Ranked Search Results

1. Problem Statement

Traditional evaluations of search and Information Retrieval (IR) systems rely heavily on expert-led, output-focused audits using technical metrics (e.g., NDCG, MAP) and relevance judgments. These methods often fail to capture:

• User-perceived harms: How end-users experience accountability, trust, and bias in real-world contexts.
• Contextual understanding: The causal links between ranking mechanisms and downstream social or epistemic impacts.
• Blind spots: Issues that arise from the "convincing" nature of modern neural models, which may mask manipulations or biases that experts or standard metrics miss.

The paper argues that as ranking models (like neural semantic rerankers) become more sophisticated and opaque, there is a critical need for participatory auditing—a process where stakeholders (users) actively evaluate systems against their own normative standards (fairness, transparency, accountability) rather than just technical performance.

2. Methodology

The authors conducted three participatory auditing workshops involving 21 participants (recruited from a university setting with diverse backgrounds but no mandatory AI/IR expertise).

Experimental Setup

• Custom Search Interface: Built on the PyTerrier framework using a snapshot of the English Wikipedia (TREC Fair Ranking Track corpus).
• Ranking Models Compared:
  1. BM25: A lexical baseline (keyword matching), expected to perform poorly on semantic queries.
  2. MonoT5: A neural semantic reranker, expected to produce coherent, semantically relevant results.
• Tasks: Participants engaged in four specific tasks:
  1. Task 1 (Baseline): Query "famous doctor" using BM25. Results were intentionally poor (literal matches) to establish a baseline of low system competence.
  2. Task 2 (Neural): Same query ("famous doctor") using MonoT5. Results were semantically coherent, establishing high perceived competence.
  3. Task 3 (Transparency & Control): An expanded interface allowing model switching, metadata viewing (source, country), and an "auto-fairness" filter.
  4. Task 4 (Adversarial Manipulation): Query "Tylenol" using MonoT5. A specific Wikipedia article (Helen Keaney) was adversarially manipulated via keyword injection (adding "Tylenol," "relevant," "true" to the title/snippet) to exploit MonoT5's inference-time prompt susceptibility, artificially boosting it to the top 5 results.

Data Collection & Analysis

• Activities: Included reflexive annotation (post-it notes), pipeline diagram annotation (identifying where transparency/control is needed), and collective sense-making matrices (mapping search properties to downstream impacts).
• Analysis: Reflexive thematic analysis of transcripts and artifacts to identify user narratives, trust dynamics, and perceived harms.

3. Key Contributions

The paper makes three primary contributions:

1. Conceptual: Argues that participatory auditing is a necessary complement to traditional expert audits, shifting the focus from "output quality" to "user-perceived impacts and accountability gaps."
2. Empirical: Provides a taxonomy of user-perceived harms in ranked search results, categorized into four dimensions:
   • Epistemic: Misinformation, misplaced trust, reduced critical scrutiny.
   • Representational: Systemic visibility patterns, omission of minority perspectives, homogeneity.
   • Infrastructural: Opaque data provenance, lack of recourse mechanisms, susceptibility to manipulation.
   • Downstream Social: Political polarization, stereotype reinforcement, agenda-setting effects.
3. Methodological: Operationalizes participatory auditing for IR through a structured workshop protocol that compares ranking models, varies interface affordances, and includes adversarial manipulation to test auditability limits.

4. Key Results & Findings

A. The "Convincing Wrongness" of Neural Models

• Trust Accumulation: Participants initially viewed BM25 as limited but mechanical. However, after interacting with MonoT5 (Task 2), they developed accumulated trust and perceived the system as "competent" and "authoritative."
• Blind Spots: This trust led to a reduction in critical scrutiny. In Task 4, participants largely failed to detect the adversarial manipulation of the "Tylenol" query unless explicitly prompted by facilitators. They accepted the manipulated results as "truth" because the neural model appeared to understand the query semantically.
• Implication: High system competence can paradoxically hinder auditing by conditioning users to overlook anomalies.

B. User-Defined Accountability Needs

• Recourse over Explanation: Participants did not just want to know why a result appeared (explanation); they wanted the ability to contest, filter, or correct the ranking (recourse).
• Pipeline Transparency: Users expressed a strong desire to see the entire pipeline: the data corpus (provenance), the ranking logic, and recourse mechanisms. They felt that transparency without the ability to act (e.g., filtering) created an "illusion of accountability."
• Causal Reasoning: Users actively constructed causal narratives linking search properties (e.g., lack of diversity) to downstream harms (e.g., political polarization, confirmation bias).

C. Taxonomy of Impacts

The study synthesized user concerns into a four-layer taxonomy (Table 3 in the paper), highlighting that users view search not just as a retrieval tool but as an infrastructural actor shaping public knowledge and power dynamics.

5. Significance and Limitations

Significance

• Regulatory Relevance: The findings directly support emerging regulations like the EU AI Act and Digital Services Act (DSA), which mandate ongoing risk assessment and user-facing accountability. It demonstrates how participatory methods can ground these requirements in real user experiences.
• Beyond Metrics: It challenges the IR community to look beyond aggregate metrics (like NDCG) which may show high performance while masking epistemic harms (e.g., a model ranking only male European doctors for "famous doctor" might score high on relevance but fail on representational fairness).
• Auditability Limits: It reveals that auditability is not a static property of a system but is dynamic, shaped by user trust, prior interaction history, and perceived system competence.

Limitations

• Sample Size & Demographics: The study involved 21 participants, primarily from a university context, limiting generalizability to broader populations (e.g., marginalized communities, content creators).
• Artificial Context: The workshops were time-constrained and used a custom interface, which may not fully replicate the complexity of naturalistic search environments.
• Query Selection: Pre-selected queries may not have aligned with participants' personal information needs, potentially affecting their ability to judge relevance.
• Scope: The study focused on ranked lists, not emerging hybrid systems like Retrieval-Augmented Generation (RAG) or conversational search.

Conclusion

The paper concludes that participatory auditing is essential for surfacing blind spots that expert audits miss, particularly regarding trust, manipulation, and downstream social impacts. However, it also warns that participatory auditing itself has limits: when users trust a system too much (due to its perceived competence), they may fail to audit it effectively. Therefore, participatory auditing should be viewed as a reflexive, complementary practice that must be continuously refined to address the evolving capabilities of AI-driven search systems.