Advancing FAIR Data Management through AI-Assisted Curation of Morphological Data Matrices

This study presents an AI-assisted curation tool integrated with MorphoBank that leverages Large Language Models to automate the extraction and standardization of morphological character data into NEXUS format, thereby significantly enhancing data accuracy, efficiency, and FAIR compliance in biological and paleontological research.

The Gist

Imagine you are a librarian in a massive, ancient library filled with millions of dusty, handwritten journals about the history of life on Earth. These journals contain the "family trees" of animals and plants, but the information is scattered. Some pages have neat tables, others have messy sketches, and many are just long paragraphs of text describing how a shell looks or how a bone is shaped.

The problem? Most of this information is trapped in these old books. Scientists can't easily search for it, combine it, or use it to build new digital models because the data isn't in a standard computer format. It's like having a million puzzle pieces, but they are all glued to the pages of different books.

This paper introduces a new AI-powered "Digital Librarian" called MatrixCurator that helps solve this mess. Here is how it works, broken down into simple concepts:

1. The Problem: The "Lost in Translation" Puzzle

For decades, scientists have been manually typing data from these old books into a special computer file format called NEXUS. Think of NEXUS as the "Universal Language" that all evolutionary biology computers speak.

• The Old Way: A human curator would sit for hours, reading a paper, finding a table of 100 traits, and manually typing them into the computer. It was slow, boring, and prone to typos (like writing "blue" instead of "blu").
• The Result: Many old datasets were incomplete. They had the "answers" (the data matrix) but were missing the "questions" (the descriptions of what the traits actually meant). Without the questions, the answers are useless.

2. The Solution: The AI "Super-Reader"

The authors built a tool that uses Large Language Models (LLMs)—the same kind of smart AI that can write poems or answer questions—to act as a super-fast reader.

• The Job: The AI reads a scientific paper (PDF or Word doc), finds the specific section describing animal traits, and extracts the information.
• The Magic: It doesn't just copy-paste; it understands the context. If a paper says, "The shell is round," the AI knows that "round" is a state of the character "shell shape." It then organizes this into the strict NEXUS format automatically.

3. How It Works: The "Chef and the Food Critic" Team

The system doesn't just rely on one AI; it uses a clever team of two AI agents working together, like a Chef and a Food Critic:

• The Chef (Retriever Agent): This AI is fast and cheap. Its job is to scan the document and "cook up" a draft of the data. It grabs the character names and states and writes them down in a structured list.
• The Critic (Evaluator Agent): This AI is slower but much smarter and more careful. It takes the Chef's draft and compares it against the original text.
  • Did the Chef miss a trait? The Critic says, "Nope, you missed the one on page 4."
  • Did the Chef make up a trait? The Critic says, "Stop! That word isn't in the book. You're hallucinating."
  • If the Critic finds errors, it sends the draft back to the Chef to fix it. This loop continues until the data is perfect.

4. The Results: From Hours to Minutes

The researchers tested this system on dozens of old scientific papers.

• Speed: What used to take a human curator two hours for a medium-sized dataset was done by the AI in a fraction of the time.
• Accuracy: The best AI combination (using Google's "Gemini" models) got about 91% accuracy on the first try. While not perfect, it meant the human curator only had to do minor "proofreading" instead of rewriting the whole thing from scratch.
• Cost: By using a smart caching system (remembering the book so it doesn't have to re-read the whole thing for every single question), they reduced the cost of processing a single paper by 93%.

5. Why This Matters: Making Data "FAIR"

The paper talks about FAIR data. Think of this as a checklist for making data useful:

• Findable: Now, instead of searching through a PDF to find what "Character 42" means, the computer knows exactly what it is.
• Accessible: The data is self-contained; you don't need the original book to understand the file.
• Interoperable: It speaks the universal NEXUS language, so any scientist can use it.
• Reusable: Future scientists can take this data and mix it with new data to answer big questions about evolution.

The Bottom Line

This isn't a robot that replaces human scientists. Instead, it's a powerful assistant.
Imagine a human curator is a master architect. Before, they had to hand-carve every single brick for a castle. Now, the AI builds the bricks for them, and the architect just inspects them to make sure they are the right shape and color.

This tool allows scientists to unlock the treasure trove of data trapped in thousands of old papers, turning dusty history into a living, breathing digital library that helps us understand how life evolved on Earth.

Technical Summary

1. Problem Statement

The curation of morphological character datasets (traits and their states across taxa) is a critical bottleneck in evolutionary biology and paleontology.

• Data Fragmentation: Morphological data often exists only as tables within published literature or supplementary materials, lacking machine-readable structure.
• Manual Labor & Errors: Converting these unstructured or semi-structured sources into standardized formats (specifically NEXUS, the standard for phylogenetic analysis) is labor-intensive, prone to transcription errors, and inconsistent.
• FAIR Compliance Gap: Many existing datasets in repositories like MorphoBank lack the CHARACTERS block (metadata defining what the traits are), containing only the data matrix. This renders the data difficult to find, interpret, or reuse without referring back to the original publication, violating FAIR (Findable, Accessible, Interoperable, Reusable) principles.
• Scalability: Manual curation takes approximately two hours for a 100-character matrix, making large-scale digitization of legacy literature impossible with current human resources.

2. Methodology

The authors developed MatrixCurator, a proof-of-concept, AI-assisted curation pipeline designed to automate the extraction of character names and states from scientific literature and integrate them into NEXUS files.

Core Architecture: Multi-Agent System

The system employs a multi-agent architecture to ensure accuracy and iterative correction:

1. Retriever Agent: Extracts character definitions and states from the source document and outputs them as a structured JSON object.
2. Evaluator Agent: Independently verifies the Retriever's output against the original source text to assess accuracy.
3. Count Evaluator: Validates that the total number of extracted characters matches the user-specified count.
4. Iterative Loop: If validation fails, the system re-prompts the Retriever with negative constraints (e.g., "Do not include anatomical regions not mentioned") to correct errors before final serialization.

Technical Stack & Workflow

• Document Preprocessing:
  • Supports PDF and DOCX inputs.
  • Uses PyMuPDF (with OCR fallback for scanned images) and LibreOffice (for DOCX to Markdown conversion).
  • Leverages Gemini Native Vision for multimodal parsing (handling tables, charts, and complex layouts).
• Large Language Models (LLMs):
  • Retriever: Uses Gemini 2.5 Flash models for speed and cost-efficiency.
  • Evaluator: Uses Gemini 2.5 Pro for higher reasoning capabilities and accuracy assessment.
  • Context Caching: Implemented to cache the full document content, reducing token usage by ~93% for subsequent character requests.
• Data Schema:
  • Intermediate representation: JSON (structured as a dictionary of character names and arrays of states).
  • Final Output: Converted to NEXUS format, specifically reconstructing the CHARACTERS block with STATELABELS.

3. Key Contributions

• Proof-of-Concept Pipeline: Demonstrates that LLMs can successfully extract structured morphological metadata from heterogeneous scientific literature.
• Hybrid Human-AI Workflow: Proposes a model where AI handles the "drafting" of data extraction, while human curators focus on verification and quality control, effectively redistributing rather than eliminating curatorial effort.
• FAIR Data Enhancement: Provides a mechanism to convert "matrix-only" files into fully self-contained, metadata-rich NEXUS files, significantly improving data interoperability and reusability.
• Open Source Implementation: The tool is implemented as a Streamlit web application, with source code and prompts available on GitHub, and benchmark datasets archived on Zenodo.

4. Results

The system was evaluated on a benchmark of 32 publications (1991–2020) and 12 documents for parsing performance.

• Parsing Performance:
  • Gemini Native Vision achieved the highest text similarity ratio (0.86) compared to PyMuPDF (0.66) and LlamaParse (0.46), proving superior at handling complex tables and layouts.
• Extraction Accuracy:
  • The Gemini 2.5 Flash (Retriever) + Gemini 2.5 Pro (Evaluator) configuration achieved the highest performance:
    • Success Rate: 99.95%
    • Average Accuracy: 90.91%
  • Open-source models (Gemma, Llama) showed high success rates but low factual accuracy (~25–30%), often hallucinating plausible but incorrect states.
• Efficiency & Cost:
  • Context Caching reduced token usage from ~314k to ~21k tokens for a 164-character matrix (93% reduction).
  • Cost Reduction: Estimated cost dropped from $0.47 to $0.03 per matrix due to token savings.
  • Processing Speed: The Flash-based Retriever processed characters in ~1.35 seconds on average.
• Real-World Application:
  • Successfully processed >400 papers and >35,000 character-state entries.
  • Curator review indicated that AI outputs typically required only minor edits (punctuation, formatting), drastically reducing manual transcription time.

5. Significance and Future Directions

• Scalability: The tool offers a viable path to digitize the vast backlog of legacy morphological data, which is currently inaccessible for large-scale computational analysis.
• Scientific Infrastructure: It establishes a framework for "digital curators" that can be extended to other biological data types (e.g., ecological traits, genomic metadata).
• Limitations & Future Work:
  • Current limitations include the need for manual specification of page ranges and character counts (though automated detection is a future goal).
  • Performance degrades with non-English texts, heavily scanned images, or narrative-heavy descriptions without clear tables.
  • Future iterations aim to integrate directly into MorphoBank workflows, improve automated scope detection, and implement structured error logging for continuous model refinement.

Conclusion:
MatrixCurator represents a significant step toward automating the recovery of structured biological data. By combining advanced LLMs with a rigorous multi-agent validation loop, it transforms labor-intensive manual curation into a scalable, cost-effective process, thereby accelerating the transition of morphological data toward full FAIR compliance.