🧬 biology

Beyond Manual Curation: Augmenting Targeted Protein Degradation Databases via Agentic Literature Extraction Workflows

This paper introduces an expert-in-the-loop agentic workflow that leverages prompt refinement to automate the extraction of complex targeted protein degradation data from scientific literature, achieving high accuracy and significantly expanding existing databases while capturing essential experimental context previously lost in manual curation.

Original authors: Yaochen Rao, Farzaneh Jalalypour, N. M. Anoop Krishnan, Rocío Mercado

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yaochen Rao, Farzaneh Jalalypour, N. M. Anoop Krishnan, Rocío Mercado

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine the world of drug discovery as a massive, chaotic library. Every year, scientists publish over 1.5 million new books (research papers) describing how to build tiny molecular machines that can destroy disease-causing proteins. These machines are called Targeted Protein Degraders (TPDs).

The problem? The most important information in these books—the specific recipes, the exact ingredients, and the test results—is locked inside messy paragraphs, scattered tables, and hidden in the back of the book (supplementary files). Currently, a team of human librarians has to read every single book, find these details, and write them down in a neat spreadsheet. This is slow, expensive, and they often miss details or only write down the "best" examples, ignoring the failures that are also important for learning.

This paper introduces a super-smart, automated librarian (an AI workflow) that can read these scientific books much faster and more accurately than humans, while still having a human expert check its work.

Here is how it works, using simple analogies:

1. The "Triangular" Truth Check

To make sure their new AI librarian is good, the authors didn't just ask it to guess. They set up a three-way comparison, like a game of "Telephone" with a referee:

The Old Database: The existing spreadsheets made by human librarians.
The AI's Guess: What the new AI extracted from the papers.
The Gold Standard: A fresh set of papers read and annotated by expert human scientists (the "Ground Truth").

By comparing all three, they could see if the AI was better than the old databases and if it was actually telling the truth compared to the experts.

2. The "Coach and Player" System (Agentic Workflow)

Instead of just giving the AI a static set of instructions, the authors built a dynamic team of three AI "agents" that work together, supervised by a human:

The Extractor (The Player): This agent reads the paper and pulls out the data (like the compound name, the target protein, and the results).
The Analyst (The Coach): If the Extractor makes a mistake, the Analyst looks at the original paper and the "Gold Standard" to figure out why it failed. Did it miss a table? Did it confuse two similar chemical names?
The Refiner (The Trainer): Based on the Analyst's notes, the Refiner tweaks the instructions (the "prompt") for the Extractor to make sure it doesn't make the same mistake next time.

This happens in a loop. The AI tries, gets coached, learns, and tries again until it gets it right. The authors call this CAPO (Cross-validated Agentic Prompt Optimization). It's like training a dog: you don't just tell it once; you correct it, and it learns the rules.

3. The "Magic Trick" of Transfer

The team started by training this system on Molecular Glues (one type of protein degrader) using only seven expert-annotated papers. That's a tiny amount of training data.

Once the system learned how to read Molecular Glue papers, they did something clever: they simply swapped the word "Molecular Glue" with "PROTAC" (a different, but related, type of degrader) in the instructions. They didn't need to retrain the whole system. It was like teaching a chef to bake a chocolate cake, and then just telling them, "Now bake a vanilla cake using the same steps, just swap the flavor." The AI worked perfectly for the new type too.

4. The Results: Filling the Gaps

When they let this AI loose on thousands of papers, the results were impressive:

It found more: The AI added 81% more records to the Molecular Glue database and 92% more to the PROTAC database.
It was accurate: When experts checked the AI's new findings, 92% of the Molecular Glue records and 82.5% of the PROTAC records were correct.
It found the "boring" stuff: The old human-curated databases mostly recorded the "winners" (strong drugs) and missed the context (like how long the test took or what cells were used). The AI recovered all this missing context, which is crucial for scientists who want to compare different studies fairly.

5. What It Didn't Do (The Limits)

The paper is very clear about what this tool cannot do yet:

It can't read pictures: If a crucial experiment result is only shown in a graph or a photo (like a Western blot image) and not written in text or a table, the AI misses it. The old human databases also missed these, but the AI didn't fix that specific problem.
It needs a human in the loop: The system isn't meant to replace scientists entirely. It's designed to do the heavy lifting of reading and sorting, while humans verify the tricky parts.

The Bottom Line

The authors have built a tool that acts like a tireless, hyper-accurate research assistant. It takes the messy, unstructured text of scientific papers and turns it into clean, organized data. By using a small amount of expert help to "teach" the AI how to read, they were able to expand existing databases by nearly double, providing scientists with a much richer and more complete picture of how these protein-degrading drugs work. They have released this tool and the new data for everyone to use.

Problem Statement
The rapid expansion of biomedical literature, with over 1.5 million articles published annually, has created a critical bottleneck for data-driven drug discovery: the difficulty of converting unstructured published results into structured, machine-readable data. This challenge is particularly acute in Targeted Protein Degradation (TPD), a therapeutic modality involving PROTACs and molecular glues (MGs). Existing public resources (e.g., PROTAC-DB, MolGlueDB) rely on manual curation, which is slow to keep pace with publication rates and often lacks the granular experimental context (assay type, cell lines, timepoints, concentrations) required for downstream machine learning modeling. Furthermore, manual curation tends to prioritize highly active compounds, under-reporting weak or inactive examples essential for training robust models. Generic Large Language Model (LLM) extraction pipelines fail in this domain because TPD records require combining disparate fields (compound identity, target, recruiter, assay context, endpoints) often scattered across text, tables, and supplementary files, while navigating inconsistent chemical identifiers and implicit experimental contexts.

Methodology
The authors propose an expert-in-the-loop LLM workflow designed to extract structured TPD assay records without model fine-tuning. The system operates on full-text articles and supplementary materials from PubMed Central (PMC).

Data Acquisition and Triangular Evaluation: The study constructs three datasets for a triangular comparison: a baseline of standardized records from existing databases (PROTAC-DB and MolGlueDB), an LLM prediction set, and an independent ground truth derived from expert annotation of a stratified subset of papers (7 MG and 15 PROTAC papers).
Agentic Workflow (CAPO): The core innovation is CAPO (Cross-validated Agentic Prompt Optimization), a lightweight module that adapts extraction instructions from scarce expert annotations. It employs a dual-loop system:
- Inner Loop: An Extractor agent processes articles through four phases: initial main-text extraction, sentence-level source verification, supplementary material integration, and a final review to complete missing chemical identifiers.
- Outer Loop: An Analyst agent diagnoses errors (false positives/negatives) in papers falling below a performance threshold ( $\tau = 0.9$ ). A Refiner agent then updates the extraction prompt based on these diagnoses. This process runs via Leave-One-Out Cross-Validation (LOOCV) to select a "Champion Prompt" that maximizes F1-score on held-out data.
Post-Processing and Semantic Matching: Extracted data undergoes unit normalization, deduplication, and chemical identifier enrichment (using OPSIN and PubChem). A semantic matching module handles entity synonymy (e.g., "CRBN" vs. "Cereblon") using a constrained LLM-based yes/no verification, cached for consistency.
Evaluation Framework: The system uses hierarchical evaluation modes: Experimental (5-field key: compound, target, recruiter, assay, cell line), Mechanistic (3-field key), and Phenotypic (2-field key). Metrics include precision, recall, and F1 at both record and field levels.

Key Contributions

Formulation and Framework: The paper formulates TPD database extraction as a domain-specific curation task and introduces a triangular evaluation framework comparing LLM predictions, standardized baseline records, and expert ground truth.
Expert-in-the-Loop Workflow: The authors present an end-to-end workflow combining source-grounded extraction, supplementary integration, and the CAPO prompt-refinement module, which successfully adapts extraction rules from a very small set of annotations (7 papers).
Scalable Performance: The workflow achieves high accuracy across both molecular glues and PROTACs, demonstrating that a prompt optimized for one modality can transfer to the other via terminology substitution alone.
Resource Release: The authors release the workflow, optimized prompts, evaluation code, and the newly extracted datasets as open resources.

Results

Accuracy: On the Molecular Glue (MG) cohort, the workflow achieved a record-level F1 of 0.98. When transferred to PROTACs via terminology substitution, it maintained a record-level F1 > 0.93. In contrast, existing baseline databases (MolGlueDB and PROTAC-DB) achieved significantly lower F1 scores (0.43 and 0.69, respectively) under the same Mechanistic mode, primarily due to missing contextual fields.
Database Augmentation: Applied at scale, the workflow expanded the MG database by 81% (from 136 to 246 records) and the PROTAC database by 92% (from 372 to 714 records).
Validation: Manual review of the newly recovered "LLM-only" records confirmed 92% validity for MGs and 82.5% for PROTACs. Conversely, many "baseline-only" records relied on figure-based evidence outside the text-extraction scope.
Context Recovery: The workflow significantly recovered missing contextual fields, increasing assay and cell-line coverage by 498% and 513% for MGs, and 7010% and 98% for PROTACs, respectively. It also recovered kinetic data (timepoints, concentrations) essential for condition-aware modeling.
Efficiency: The prompt optimization converged within three rounds using only seven annotated papers, with total API costs for the entire project remaining under $100.

Significance
The paper argues that the primary bottleneck in TPD curation is coverage rather than accuracy; existing databases are reliable when present but incomplete. The results support a shift from exhaustive manual transcription to expert-in-the-loop curation, where domain experts provide targeted annotations to validate and guide automated pipelines. By recovering rich experimental context (timepoints, concentrations, assay types) that is frequently missed in manual curation, this workflow enables the construction of balanced, condition-aware datasets necessary for training generalizable machine learning models for protein degradation. The authors position this work as a template for AI-assisted scientific curation in domains where structured data remains scattered across literature, emphasizing that the workflow is designed to support, not replace, expert curation.