AI4Life Open Calls and Public Challenges: why, how, and what we have learned.

Technical Summary: AI4Life Open Calls and Public Challenges

Problem Statement
The paper addresses the persistent gap between the availability of state-of-the-art deep learning (DL) methods for bioimage analysis and their practical application by life scientists. While numerous algorithms exist, the field faces a bottleneck not in methodological innovation, but in the "infrastructure" required to apply them: data readiness, annotation quality, interoperability, and shared norms. The authors posit that scientific AI in biology is currently limited by the lack of FAIR (Findable, Accessible, Interoperable, Reusable) data, annotations, and shared infrastructure, rather than by the algorithms themselves.

Methodology
To investigate the state of the field and the barriers to adoption, the AI4Life consortium (2022–2025) executed two distinct initiatives:

1. Three Open Calls (OCs): A support program where life scientists submitted bioimage analysis projects. Selected projects received six months of technical support from a team of AI and image analysis experts to develop reproducible, FAIR-compliant pipelines. The selection process evolved over three rounds (2023–2025) to include mandatory data uploads, template-based applications, and expert consultation rounds to filter for project readiness.
2. Three Public Challenges (PCs): Computational competitions focused on image denoising (unsupervised, supervised, and domain-specific calcium imaging). These challenges utilized publicly available or synthetically generated datasets to benchmark existing methods and encourage community participation.

Key Contributions and Findings

• The "Readiness" Gap: Analysis of 151 applications across the Open Calls revealed a significant mismatch between applicant expectations and data reality. Only 8 of the 22 selected projects were fully ready at the start of support (with complete, usable data and sufficient annotations). Only 2 projects possessed sufficient ground-truth data to immediately apply deep learning as a first step. Most projects required substantial additional effort for data acquisition, re-imaging, or annotation.
• Data Infrastructure Deficiencies: Data transfer emerged as a critical bottleneck. Across 22 projects, 14 different data-transfer methods were used, ranging from private cloud services to physical hard drives, highlighting a lack of standardized, institutional infrastructure. Furthermore, data sharing was consistently delayed; the average time between the end of project support and public data release was 8.1 months. Only 17 of 22 projects successfully deposited data in a public repository by the time of writing.
• Tool Adoption and Ecosystem: Despite the push for Python-based deep learning, the field remains heavily reliant on established, user-friendly tools. Pipelines developed during the OCs frequently incorporated "cornerstone" tools such as Cellpose, StarDist, Noise2Void, and ImageJ2/Fiji. While new architectures (e.g., transformers, foundation models like SAM2) are being absorbed, they are primarily adopted when they reduce the annotation burden (e.g., using pre-trained models) rather than solely for marginal benchmark improvements.
• Public Challenge Outcomes: The three denoising challenges attracted 225 participants from 41 countries. Results indicated that established methods (e.g., Noise2Void and its variants) remain the default choice for most participants. However, in technically demanding settings (e.g., calcium imaging), alternative architectures like transformer-based models (Restormer3D) demonstrated clear advantages. The challenges also revealed that while community interest is broad, participation is modest compared to computer vision benchmarks, largely due to a lack of monetary or career incentives.
• Selection Process Refinement: The paper details the evolution of the selection process, noting that remote scoring alone was insufficient to gauge project readiness. The introduction of mandatory data uploads and expert consultation rounds significantly improved the identification of solvable projects and reduced attrition.

Significance and Claims
The authors conclude that the rate-limiting step for scientific AI in biology is not the development of new methods, but the maturation of the underlying data ecosystem. They argue that:

• Infrastructure is the bottleneck: The field lacks shared data-transfer infrastructure, sustained annotation efforts, and incentive structures that reward dataset publication on par with method publication.
• Expectation Management: There is a general misunderstanding regarding the data and annotation requirements of deep learning. Successful application requires realistic expectations and early planning for annotation efforts.
• Community Action: The paper calls for a shift in community norms and funding priorities. It recommends that funding bodies and institutions treat data infrastructure, tool maintainers, and annotation efforts as strategic capacity building rather than peripheral services. It suggests that for AI to deliver practical impact in biology, the community must prioritize "unsexy" infrastructure work—shared standards, interoperable tools, and FAIR data practices—over the continuous pursuit of novel algorithms.

The paper does not claim to have solved these systemic issues but provides empirical evidence of their existence through the lens of three years of structured support and benchmarking, offering a roadmap for future initiatives to avoid similar pitfalls.