Technical Summary: The Hardness of Achieving Impact in AI for Social Impact Research

1. Problem Statement

The field of Artificial Intelligence for Social Impact (AI4SI), also known as AI for Social Good (AI4SG), has seen rapid growth, with projects addressing United Nations Sustainable Development Goals (UN SDGs) such as healthcare, climate action, and social justice. However, a significant gap exists between the proliferation of research projects and their tangible, on-the-ground deployment. Despite rising interest, approximately 72% of AI4SI projects fail to transition from the research and development (R&D) or proof-of-concept phase to long-term, scaled deployment.

While existing literature offers frameworks for responsible AI design and identifies technical limitations, there is a lack of grounded understanding regarding the non-technical, structural, and operational barriers that prevent deployment in real-world, resource-constrained settings. Specifically, the literature often abstracts away the "lived realities" of field deployment, leaving researchers without practical guidance on navigating the complex ecosystems involving academic institutions, partner organizations (POs) like non-profits and government agencies, and low-resource communities.

2. Methodology

To investigate the factors hindering real-world deployment, the authors conducted a qualitative study employing thematic analysis on data derived from:

• Semi-structured Interviews: 26 interviews were conducted with experienced AI4SI researchers. The sample included primarily academic researchers from institutions in the Global North, along with industry researchers and one startup practitioner.
• Lived Experience: The analysis was supplemented by the authors' own experiences with five AI4SI projects.
• Scope: The study covered 38 distinct AI4SI projects across various sectors (e.g., conservation, public health, social justice).

Data Collection & Analysis:

• Recruitment: Participants were recruited via purposive sampling of authors from key AI4SI venues (e.g., AAAI, AAMAS, KDD workshops) and snowball sampling based on citation networks.
• Protocol: Interviews lasted approximately 43 minutes on average. Participants discussed specific projects, including those that failed to deploy, and provided suggestions for improving collaboration models.
• Analysis: The authors utilized an inductive thematic analysis approach. Two authors independently performed open coding on approximately 1,121 minutes of transcripts, generating 33 preliminary codes. These were refined through iterative theoretical coding to establish higher-order themes, with disagreements resolved through consensus. The analysis was supported by the qualitative data analysis software Taguette.

3. Key Contributions and Findings

The paper identifies four primary categories of challenges that hinder the deployment of AI4SI solutions, moving beyond technical limitations to address structural and organizational hurdles:

3.1 Structural and Organizational Challenges

• Misaligned Academic Incentives: The prevailing "publish or perish" culture prioritizes high-volume publications and theoretical novelty over the long-term, engineering-heavy work required for deployment. Researchers often face a trade-off where focusing on real-world utility reduces their publication output, leading to the deprioritization of AI4SI projects.
• Misaligned Partner Organization (PO) Incentives: POs often lack the budget, staff capacity, or formal mandate to engage in long-term research collaborations. Front-line workers may view AI tools as irrelevant or burdensome compared to immediate needs (e.g., better equipment), creating a disconnect between leadership enthusiasm and operational reality.
• Funding Constraints: There is a chronic lack of funding for the "hidden costs" of deployment, such as project management, infrastructure, and long-term maintenance. Grants often cover initial prototyping but fail to support the sustained engineering effort required for scaling.
• Institutional Inertia and Disciplinary Gaps: Procedural red tape (e.g., data use agreements) slows progress. Furthermore, researchers often lack interdisciplinary training in fields like econometrics, human-centered design, or software engineering, which are essential for field deployment.

3.2 Communication and Understanding Gaps

• Contested Meanings: Divergent understandings of "AI" between researchers and POs lead to mismatched expectations. POs may harbor skepticism or inflated hopes, while researchers struggle to explain technical capabilities in accessible terms.
• Disciplinary Jargon: The use of domain-specific terminology creates barriers to effective collaboration.
• Role of Internal Champions: Successful projects often rely on identifying "champions" within the PO who possess shared expertise (e.g., in economics or game theory) to bridge communication gaps and advocate for the system internally.

3.3 Collaboration Hurdles

• Strategic Partnerships: Deployment requires institutional buy-in from PO leadership with decision-making power. Without this, projects stall. Researchers often rely on "intermediaries" (e.g., social workers or public health scholars) to facilitate trust and logistics, though this can create bottlenecks for new researchers lacking such networks.
• Trust Issues: POs are often skeptical of researcher motives, fearing "extractive" research where data is taken for a paper without lasting benefit. Building trust requires transparency, explainable models, and early delivery of tangible value ("quick wins").

3.4 Operational Hurdles

• Dynamic Environments: POs operate in volatile contexts with high staff turnover (often relying on volunteers) and shifting priorities. Static software solutions often become obsolete as organizational structures or problem definitions change.
• Data Issues: Access to high-quality, relevant data is a major bottleneck. Data is often messy, incomplete, or sensitive, requiring significant time and effort to clean and negotiate access for.
• Scalability and Maintenance: Most projects receive limited funding (<$250K) and are narrowly scoped. Scaling requires substantial software engineering effort that academic labs are rarely equipped or incentivized to provide. Student turnover further disrupts long-term maintenance.

4. Proposed Strategies and Best Practices

While acknowledging there are no "easy fixes," the authors synthesize actionable strategies derived from the interviews and their own experiences:

• For Researchers:
  • Deliver Early Value: Prioritize "quick wins" (e.g., simple data analysis) to build trust and demonstrate utility before aiming for technical novelty.
  • Standardized Documentation: Use tools like Model Cards and Data Cards to reduce communication barriers and clarify limitations.
  • Strategic Labor Delegation: Pair PhD students (focused on core research) with master's/undergraduate students to handle engineering and maintenance tasks.
  • Leverage Funding Mechanisms: Utilize "matching market" style grants (e.g., NSF ReDDDot, NSF TTP) designed to foster early collaboration and trust-building before full-scale deployment.
• For Institutions:
  • Redefine Research Value: Expand evaluation metrics to recognize deployed systems, open-source tools, and public impact alongside algorithmic novelty.
  • Institutional Support: Establish consortia to manage administrative burdens (e.g., data agreements) and connect researchers with POs.
  • Interdisciplinary Training: Develop dedicated training programs that equip students with skills in policy, ethics, and field methods alongside technical AI skills.
• For Partner Organizations:
  • Understand Academic Constraints: Recognize the structural pressures researchers face (e.g., publication cycles) to set realistic expectations and advocate for collaboration structures that benefit both parties.

5. Significance and Positioning

The paper positions itself as a practical guide for AI4SI researchers and organizations, filling a gap in the literature by focusing on the researcher's perspective rather than just the community or PO perspective. It argues that the challenges of AI4SI are not merely technical but are deeply rooted in the misalignment of incentives between academic reward systems and the realities of field deployment.

The authors explicitly state that their findings are most directly applicable to academic research groups in the Global North, acknowledging that perspectives from the Global South, startups, and government sectors are underrepresented in their sample. They emphasize that AI4SI is a vital counterbalance to Big Tech's focus on for-profit solutions, but its success requires addressing the systemic barriers that currently prevent socially impactful work from moving beyond the proof-of-concept stage. The paper concludes that strengthening this field requires collective effort to align incentives, improve training, and foster sustainable partnerships.