AI-readiness for Biomedical Data

The NIH Bridge2AI Standards Working Group proposes a comprehensive framework of seven core dimensions—extending beyond FAIR principles to include ethics, provenance, and explainability—to define actionable criteria for ensuring the ethical, robust, and machine-actionable AI-readiness of biomedical data before model training.

The Gist

Imagine you are a chef trying to create the world's most delicious, life-saving soup using a super-smart robot assistant (Artificial Intelligence).

In the past, scientists thought the only thing that mattered was getting the robot to taste the ingredients and learn. But this paper argues that if you hand the robot a bowl of muddy water, old vegetables, or ingredients you stole without asking, the soup will be terrible—or worse, dangerous.

This paper is a recipe for "AI-Ready" biomedical data. It's a set of rules created by a team of experts (the Bridge2AI Standards Working Group) to ensure that the data fed into medical AI is clean, honest, safe, and easy to understand before the AI even starts cooking.

Here is the breakdown of their 7-step "AI-Readiness" checklist, explained with simple analogies:

1. FAIRness (The "Library" Rule)

The Concept: Data must be Findable, Accessible, Interoperable (works with other systems), and Reusable.
The Analogy: Imagine your data is a book in a library.

• Findable: It has a clear label on the spine and is in the catalog.
• Accessible: You can actually walk up to the shelf and read it (even if you need a special key for sensitive books).
• Interoperable: It's written in a language the robot can read, not ancient hieroglyphs.
• Reusable: It comes with a clear note saying, "You are allowed to use this for cooking, but please don't burn it."
• Why it matters: If the robot can't find the book or doesn't understand the language, it can't learn.

2. Deep Provenance (The "Receipt" Rule)

The Concept: You need a complete history of where the data came from and every step it took to get to you.
The Analogy: Think of a high-end restaurant. If you order a steak, you want to know: Which farm did the cow come from? Was it fed organic grass? Who butchered it? How was it transported?

• The Problem: If the data is just "Steak," the robot doesn't know if it's fresh or spoiled.
• The Solution: This rule demands a digital "receipt" that traces the data back to its raw source (like a blood test or a patient's diary) through every computer program that touched it. If the robot sees a weird result, it can look at the receipt to see if the data was corrupted along the way.

3. Characterization (The "Nutrition Label" Rule)

The Concept: You must describe the data in extreme detail, including its flaws.
The Analogy: Think of a food package. You need to know the calories, the ingredients, and crucially, the allergens.

• The Twist: This paper says you must also list the "allergens" of the data. Did the data only come from young men? Is it missing data for people with a certain disease?
• Why it matters: If you feed the robot data that only represents half the population, the robot will think the other half doesn't exist. This leads to biased, unfair medical advice.

4. Pre-model Explainability (The "User Manual" Rule)

The Concept: Before the AI starts learning, there must be a clear, human-readable document explaining what the data is good for and what it is not good for.
The Analogy: Before you buy a new power tool, you read the manual. It tells you, "This saw cuts wood, but do not use it on metal."

• The Solution: The paper proposes a "Datasheet" (like a nutrition label for data) that tells scientists: "This data is great for predicting heart disease, but don't use it to predict how fast a car can go." It prevents people from using the data for the wrong job.

5. Ethics (The "Permission Slip" Rule)

The Concept: The data must be collected legally and ethically, with the permission of the people involved.
The Analogy: Imagine you want to take a photo of a stranger to train your robot. You can't just snap a picture and steal it. You need their permission, and you need to promise to protect their privacy.

• The Solution: The paper requires proof that the people who gave the data (patients, volunteers) signed the right forms (Consent), that their privacy is protected, and that the data isn't being used in ways they didn't agree to.

6. Sustainability (The "Time Capsule" Rule)

The Concept: The data must be stored in a way that it won't disappear or rot in 10 years.
The Analogy: If you bury a time capsule, you need to make sure it's made of rust-proof metal and buried in a safe place, not a swamp.

• The Solution: The data must be stored in trusted, secure digital vaults that guarantee it will still be there and readable 20 years from now, even if the original computer system is gone.

7. Computability (The "Plug-and-Play" Rule)

The Concept: The data must be in a format that computers can actually process automatically.
The Analogy: Imagine you have a pile of loose LEGO bricks. You can build a castle, but it takes forever to sort them. Now imagine the bricks are sorted by color and shape in a box with a clear map.

• The Solution: The data shouldn't be a messy pile of PDFs and handwritten notes. It needs to be organized so the robot can instantly "plug in" and start building its model without a human having to spend months cleaning it up.

The Big Picture: Why Do We Need This?

The authors point out that we are currently in a "Wild West" of medical AI. We have tons of data, but it's often messy, biased, or lacks a history. If we feed this "junk" into powerful AI, we get "junk" results—misdiagnoses, unfair treatments, and broken trust.

The "AI-Readiness" score is like a report card. Instead of just saying "Pass" or "Fail," it gives a score for each of the 7 areas above.

• A dataset might be 100% FAIR (easy to find) but only 50% Ethical (we don't know if the patients consented).
• This score tells scientists: "Hey, this data is great for finding patterns, but be very careful before using it to make medical decisions."

In short: This paper is a guidebook to stop scientists from feeding garbage into their AI robots. It ensures that before the robot learns to heal, we make sure the ingredients are fresh, the recipe is honest, and the kitchen is safe.

Technical Summary

1. Problem Statement

The rapid adoption of Artificial Intelligence (AI) and Machine Learning (ML) in biomedical research faces a critical "readiness gap." While biomedical research generates massive, multi-modal datasets, there are no standardized, actionable definitions for AI-readiness.

• Limitations of Current Standards: Existing frameworks like FAIR (Findable, Accessible, Interoperable, Reusable) principles are necessary but insufficient. They focus on data accessibility and reuse but fail to address the intricate derivation histories, ethical governance, and "pre-model" transparency required for high-stakes biomedical AI.
• The "Ground Truth" Fallacy: Current metadata specifications (e.g., Croissant) often treat datasets as objective ground truths, ignoring the complex computational and biological derivations that precede modeling. This lack of transparency risks propagating bias, error, and "epistemic luck" into AI models, leading to unreliable clinical decisions and scientific retractions.
• Need for Pre-Model Rigor: There is an urgent need for criteria that ensure Pre-model Explainability, ethical integrity, and deep provenance before data is ingested into an AI model.

2. Methodology

The authors developed a comprehensive framework through the NIH Bridge to Artificial Intelligence (Bridge2AI) Standards Working Group, involving experts in AI/ML, data standards, ethics, and data engineering.

• Development Process:
  • Inputs: Leveraged domain expertise from four Bridge2AI Grand Challenges (GCs), alignment with NIH metadata guidelines, extensive literature review (FAIRness, ethical AI, XAI), and formal validation against released Bridge2AI datasets.
  • Theoretical Basis: Built upon the 2019 NIH ACD AI WG report but expanded to address multimodal, large-scale data requirements. The framework incorporates philosophical concepts like the Gettier problem and Justified True Belief (JTB), arguing that transparent provenance is required to distinguish rigorous knowledge from findings derived via opaque processes.
• Technical Implementation:
  • Metadata Architecture: The criteria are operationalized into machine-actionable metadata using RO-Crate (Research Object Crate) packages (v1.2).
  • Ontologies: Utilizes W3C PROV-O for general provenance, extended by the Evidence Graph Ontology (EVI) for biomedical-specific profiles (Datasets, Software, Models, Instruments, Reagents, Samples).
  • Tools: Developed FAIRSCAPE tools (CLI and Server) for automated evaluation, LinkML for semantic modeling, and Pydantic models for validation.
  • Evaluation: Applied the criteria to four Bridge2AI flagship datasets (Functional Genomics, Clinical Care, Precision Public Health, Salutogenesis) to generate "readiness profiles" using radar plots.

3. Key Contributions: The Seven Dimensions of AI-Readiness

The paper defines AI-readiness not as a binary pass/fail, but as a dynamic profile across seven interdependent dimensions:

1. FAIRness (Level 0 Baseline):

   • Ensures data is Findable, Accessible, Interoperable, and Reusable.
   • Crucial distinction: Requires machine-readable metadata even for restricted data and clear licensing (explicitly advising against CC0 for biomedical data to preserve provenance).

2. Provenance (Deep Provenance):

   • Goes beyond simple entity assertions. Requires end-to-end traceability from raw sources (e.g., EHR, NGS reads) through all transformations.
   • Uses PROV-O and EVI to represent data as "computational arguments" with verifiable warrants.
   • Requires software and models used in pipelines to be described with rigor equivalent to Model Cards.

3. Characterization:

   • Semantics: Comprehensive descriptive metadata and vocabularies (e.g., MeSH).
   • Statistics: Domain-appropriate statistical summaries and consistent missing value encoding.
   • Bias: Explicit documentation of known biases, assumptions, and reasons for missingness.
   • Quality: Documentation of QC procedures.

4. Pre-model Explainability:

   • The "summit" of the criteria. Requires Datasheets for Datasets (extended from Gebru et al.) that are both human- and machine-readable.
   • Includes cryptographic hash integrity seals (e.g., SHA256) for data and metadata packages to ensure verifiability.
   • Defines "Fit for Purpose" use cases and limitations.

5. Ethics, Regulatory Compliance, and Governance:

   • Acquisition: Adherence to Belmont, Menlo, and CARE principles.
   • Management: Privacy-preserving processing and human-mediated oversight (Governance Chair, IRB).
   • Licensing: Granular Data Use Agreements (DUAs) or specific Creative Commons licenses (excluding CC0).
   • Security: Compliance with standards like NIST-SP-800-171 for controlled access.

6. Sustainability:

   • Ensures long-term archival in trusted repositories (adhering to TRUST Principles).
   • Requires persistence of both data and metadata to allow future reprocessing.

7. Computability:

   • Standardization: Adherence to documented schemas (JSON Schema, Frictionless Data).
   • Accessibility: Well-documented APIs or exchange protocols.
   • Portability & Context: Documentation of computational resource requirements and data splits/context.

4. Results

The framework was applied to the four Bridge2AI Grand Challenge datasets, evaluating their readiness at two timepoints (Q4 2024 and Q2 2026).

• Evaluation Metrics: Each dataset was scored on a 0–100% scale for each of the seven dimensions based on the satisfaction of sub-criteria.
• Key Findings:
  • Functional Genomics (CM4AI): Achieved 100% readiness across all dimensions in both timepoints, serving as a gold standard.
  • Clinical Care (CHoRUS): Showed significant improvement from ~60-75% in 2024 to 100% in 2026. Initial gaps were in "Characterization" (semantics/bias) and "Sustainability" (domain-appropriate repositories), which were resolved through iterative refinement.
  • Precision Public Health (Voice) & Salutogenesis (AI-READI): Similarly demonstrated progression toward 100% readiness, with initial gaps primarily in "Data Quality," "Contextualization," and "Verifiability" being addressed over time.
• Tooling Validation: The automated tools (FAIRSCAPE) successfully generated RO-Crate packages and validated metadata, proving the feasibility of machine-actionable AI-readiness evaluation.

5. Significance

• Paradigm Shift: Moves the field from treating data as static "ground truth" to viewing it as a verifiable computational argument. This addresses the "Gettier problem" in data science by ensuring justified true belief through transparent provenance.
• Ethical & Clinical Safety: By enforcing pre-model explainability and ethical governance, the framework mitigates downstream risks of bias and error in clinical AI, directly supporting patient safety and regulatory compliance.
• Reproducibility: The emphasis on deep provenance and software traceability directly addresses the reproducibility crisis (citing >10,000 retractions in 2023) by enabling end-to-end traceability of scientific claims.
• Scalability: The use of standard, lightweight formats (RO-Crate, JSON-LD, PROV-O) ensures the framework is not tied to specific proprietary tools, making it applicable across the broader biomedical data ecosystem.
• Future-Proofing: The framework is designed to evolve with the field, acknowledging that definitions of bias and ethics will change, and providing a structured way to update metadata accordingly.

In conclusion, this paper provides the first comprehensive, machine-actionable standard for preparing biomedical data for AI, bridging the gap between raw data collection and reliable, ethical AI modeling.