MetaMuse: A Multi-Agent AI System for Biomedical Metadata Curation and Harmonization

MetaMuse is a modular multi-agent AI framework that autonomously extracts, validates, and standardizes unstructured biomedical metadata using a three-stage architecture of Curator, Arbitrator, and Normalizer agents, achieving over 95% accuracy while ensuring data integrity through conservative handling of ambiguous evidence.

The Gist

Imagine you have a massive library of scientific experiments, like the Gene Expression Omnibus (GEO). This library contains millions of research papers and data sets. However, there's a huge problem: the "index cards" (metadata) that tell you what each experiment is about are written in a chaotic mess.

Some scientists write "Male," others write "M," and some write "1." Some describe a disease as "Breast Cancer," while others write "Mammary Tumor" or just "Cancer." Because the information is unstructured and inconsistent, it's like trying to find a specific book in a library where the shelves are unorganized, the titles are in different languages, and some books are hidden inside other books. This makes it nearly impossible for computers (or even other scientists) to find, reuse, or trust the data.

Enter MetaMuse, a new AI system designed to be the ultimate librarian and editor for this chaotic library.

The Problem: The "Reproducibility Crisis"

The paper starts by saying that science is facing a "reproducibility crisis." Basically, if you can't find the exact details of how an experiment was done (like the patient's age, the type of tissue, or the specific drug used), you can't repeat the experiment to see if the results are true. Currently, too much of this vital information is buried in messy, free-text notes that computers can't read.

The Solution: MetaMuse (The Multi-Agent Team)

Instead of using one giant, dumb robot to try and fix everything at once, MetaMuse uses a team of specialized AI agents, each with a specific job. Think of it like a high-end newsroom or a legal team working on a complex case.

Here is how the team works, step-by-step:

1. The Curator Agents (The Researchers)

Imagine a team of junior researchers. Each one is assigned a specific topic, like "Disease," "Tissue," or "Gender."

• Their Job: They read the messy notes and the scientific paper to find the answer to their specific question.
• The Twist: They are trained to be conservative. If they aren't 100% sure, they won't guess. They would rather say "I don't know" than make up an answer. This prevents "hallucinations" (AI making things up), which is a huge problem in science.
• Analogy: It's like a detective who refuses to accuse a suspect unless they have solid evidence. If the evidence is weak, they leave the suspect alone.

2. The Arbitrator Agent (The Editor-in-Chief)

Once the researchers (Curators) have done their work, their reports are handed to the Editor-in-Chief (The Arbitrator).

• Their Job: This agent looks at the whole picture. It checks if the answers make sense together.
• The Logic Check: If the "Disease" researcher says the patient has "Liver Cancer," but the "Cell Line" researcher says the sample is from a "Lung Cancer" cell line, the Editor-in-Chief spots the contradiction immediately.
• The Fix: The Editor sends the report back to the researchers and says, "Hey, these two don't match. Re-check your notes." This loop continues until the story makes logical sense.

3. The Normalizer Agent (The Translator)

Now that the facts are correct and consistent, they are still written in plain English (e.g., "Breast Cancer"). Computers need a strict, standardized code to sort them.

• Their Job: This agent acts as a translator. It takes the phrase "Breast Cancer" and converts it into the official medical code MONDO:0007254. It does this for "Tissue," "Age," and everything else, turning messy words into a clean, searchable database.
• Analogy: It's like converting a recipe written in a grandmother's handwriting ("a pinch of this," "a cup of that") into a precise digital file with exact gram measurements so a robot chef can cook it perfectly.

Why is this a big deal?

The paper tested MetaMuse on real data and found some amazing results:

• Super Accurate: It got the facts right more than 95% of the time.
• Safe: It almost never made things up. If it missed a detail, it was usually because the note was too vague, not because it guessed wrong.
• Transparent: Every single decision MetaMuse makes is recorded. You can look at the "paper trail" to see exactly why it decided a patient was "Male" or had "Breast Cancer." This builds trust.

The One Weakness

The paper admits that while the team is great at finding the facts, the "Translator" (Normalizer) sometimes struggles with very complex or rare medical terms. It's like a translator who is perfect at common words but gets confused by very specific dialects. The authors plan to fix this in the future.

The Bottom Line

MetaMuse is a smart, multi-agent AI system that cleans up the messy, unorganized data in scientific libraries. By using a team of specialized AI "researchers," "editors," and "translators," it turns chaotic notes into clean, standardized data. This makes it much easier for scientists to find the data they need, repeat experiments, and make new discoveries, ultimately helping to solve the crisis of unreliable science.

Technical Summary

1. Problem Statement

Public biomedical repositories, such as the Gene Expression Omnibus (GEO), suffer from a critical bottleneck: inconsistent and unstructured metadata.

• The Issue: Data submitters often use free-text fields for critical experimental descriptors (e.g., disease state, tissue origin, donor sex) rather than structured schemas. This leads to highly variable annotation quality (e.g., "M", "male", "1" for sex).
• The Consequence: This heterogeneity severely limits data discoverability and reproducibility. Essential biological attributes are "hidden in plain sight," making systematic querying, cross-study comparison, and machine learning applications nearly impossible.
• The Gap: Existing solutions face a trade-off: manual curation is accurate but non-scalable, while automated heuristic pipelines (rule-based) are scalable but lack context, leading to high error rates and hallucinations.

2. Methodology: The MetaMuse Architecture

MetaMuse is a modular, multi-agent AI framework designed to autonomously extract, validate, and standardize unstructured metadata. It operates on a three-stage pipeline using Large Language Models (LLMs) and specialized agents.

A. Data Intake & Preprocessing

• Ingestion: The system pulls raw metadata from GEO (GSM, GSE) and PubMed abstracts, filtering out irrelevant fields (e.g., author names) and packaging data into JSON units.
• Sample Typing: A CuratorAgent (Gemini-2.5-flash) determines if a sample is a Primary Sample (patient-derived) or a Cell Line. This is crucial because it dictates which metadata fields are relevant (e.g., demographics apply to primary samples; cell line IDs apply to cell lines).

B. Conditional Processing (The Core Engine)

This stage utilizes an ensemble of specialized agents:

1. CuratorAgents (Extraction):

   • Approximately ten agents are instantiated per sample, each dedicated to a specific target field (e.g., Disease, Tissue, Treatment).
   • They perform context-aware extraction, scanning abstracts and series metadata to find candidate values.
   • Conservative Strategy: Agents are designed to avoid hallucinations. If evidence is ambiguous, they default to "None reported" (False Negative) rather than guessing (False Positive).
   • Output includes the candidate value, the source context, rationale, and a confidence score.

2. ArbitratorAgent (Validation & Consistency):

   • This agent reviews the outputs of all CuratorAgents holistically to enforce cross-field logical consistency.
   • It detects contradictions (e.g., a "Triple Negative Breast Cancer" cell line paired with a "Liver Cancer" disease annotation).
   • Iterative Loop: If errors are found, the Arbitrator provides feedback (binary correctness, reason, suggested correction) to the specific CuratorAgent. This loop repeats up to 3 times to self-correct before flagging for manual review.

3. NormalizerAgent (Standardization):

   • Once values are curated and validated, this agent maps free-text candidates to formal ontological terms (e.g., mapping "Breast Cancer" to MONDO:0007254).
   • It utilizes SapBERT, a domain-specific semantic search model, to handle synonyms and colloquialisms, returning a ranked list of top matches with confidence scores.

C. Supported Ontologies

The system maps to standard ontologies including:

• Disease: MONDO
• Tissue/Organ: UBERON
• Treatment: ChEMBL
• Organism: NCBI Taxonomy
• Cell Line: CLO
• Ethnicity/Developmental Stage: HANCESTRO/HSAPDV

3. Key Contributions

1. Multi-Agent Architecture: Introduces a novel separation of duties (Curator, Arbitrator, Normalizer) that mimics human expert curation workflows, allowing for specialized reasoning and error correction.
2. Context-Aware Extraction: Moves beyond simple keyword matching by understanding the narrative context of scientific abstracts (e.g., distinguishing between a disease mentioned as a future application vs. the actual sample condition).
3. Cross-Field Logical Arbitration: Solves the problem of isolated field extraction by implementing a feedback loop that ensures biological plausibility across the entire dataset record.
4. Auditable & Conservative Design: The system prioritizes data integrity over recall. It generates a fully transparent audit trail (JSON logs of rationale and evidence) for every decision, allowing human verification.
5. Scalable High-Fidelity Curation: Demonstrates that LLM-based multi-agent systems can achieve manual-level accuracy at a scale impossible for human curators.

4. Results

The system was evaluated on a gold-standard dataset of 100 manually curated GEO samples and scaled to 400 samples.

• Curation Accuracy: Achieved >95% accuracy across key metadata fields (Disease, Tissue, Gender, etc.) on the gold-standard set.
• Error Profile: The system is highly conservative. The majority of errors were False Negatives (missing subtle cues) rather than False Positives (hallucinations). This is a deliberate design choice to prevent corrupting datasets with invented data.
• Scalability: Performance remained robust when scaled to 400 diverse samples, evaluated by an external LLM.
• Normalization Bottleneck: While extraction was highly accurate, normalization (mapping to ontology IDs) showed a drop in fidelity (e.g., Disease accuracy dropped from 95% to 80%; Tissue from 97% to 66%). The SapBERT model struggled with highly granular or composite terms (e.g., specific T-cell subtypes).

5. Significance and Future Directions

• Advancing FAIR Principles: MetaMuse directly addresses the "Interoperable" and "Reusable" aspects of FAIR data principles by converting unstructured text into standardized, machine-readable ontologies.
• Reproducibility: By providing a transparent, auditable pipeline, MetaMuse restores trust in automated data processing, a critical step for reproducible science.
• Limitations & Future Work:
  • The primary limitation is the normalization bottleneck, where semantic search struggles with high-granularity terms. Future work involves fine-tuning SapBERT on specific GEO corpora or using LLMs as a fallback reasoning mechanism.
  • The system relies on the quality of source metadata; sparsely annotated entries without linked publications remain difficult to curate.
  • Future iterations aim to incorporate more sophisticated reasoning to flag complex edge cases for human review.

Conclusion: MetaMuse represents a significant leap forward in biomedical data engineering, offering a scalable, accurate, and transparent solution to the metadata crisis, thereby unlocking the potential of vast public repositories for data-driven discovery.