HARVEST: Unlocking the Dark Bioactivity Data of Pharmaceutical Patents via Agentic AI

The paper introduces HARVEST, an agentic AI pipeline that autonomously extracts 3.36 million previously inaccessible bioactivity records from pharmaceutical patents, revealing significant generalization gaps in current AI models when evaluated on this newly recovered data.

The Gist

Imagine the world of drug discovery as a massive library. For decades, scientists have been trying to find the "secret recipes" for new medicines hidden inside this library.

Most of the library is open and well-organized (like the famous BindingDB, a public database scientists use). But there is a huge, dusty, locked basement filled with millions of boxes. These boxes contain pharmaceutical patents—legal documents where companies describe their new drug experiments.

The problem? These boxes are written in a chaotic mix of messy tables, chemical drawings, and legal jargon. They are technically "public," but no one has the time or money to manually read and organize them. This is what the authors call "Dark Data." It's there, but it's invisible to computers.

Here is how the HARVEST project unlocks this basement:

1. The Problem: The "Dark Data" Basement

Every year, companies file thousands of patents. Inside, they have tables saying, "Compound A binds to Protein B with this strength."

• The Old Way: Human experts would have to read these documents one by one, type the data into a computer, and check for mistakes. To read all the patents in the US, it would take a team of experts 55 years of non-stop work.
• The Result: Most of this data stays in the dark. AI models trying to discover new drugs are like students studying for a test using only a tiny, outdated textbook, missing the vast majority of the answers.

2. The Solution: The "Agentic AI" Team (HARVEST)

Instead of hiring 55 years' worth of humans, the authors built HARVEST, a team of AI agents (specialized robots) that work together like a well-oiled assembly line.

Think of it like a specialized detective squad rather than one super-detective:

• Agent 1 (The Scout): Scans the patent to find where the biological targets (the "locks") are mentioned.
• Agent 2 (The Accountant): Extracts the numbers (how strong the drug is).
• Agent 3 (The Translator): Figures out that "Compound 42" in the text is actually a specific chemical structure.
• Agent 4 & 5 (The Librarians): Convert the messy chemical drawings into a standard digital format (SMILES) and match the protein names to a universal ID card (UniProt).

The Magic Stats:

• Speed: They processed 164,877 patents in less than one week.
• Cost: It cost only $0.11 per document. (Imagine paying 11 cents to read a whole patent!)
• Output: They unlocked 3.36 million new data points, including 1,108 protein targets that no one knew about in public databases before.

3. The Quality Check: Is the AI Hallucinating?

You might worry, "Can a robot really read a messy patent better than a human?"
The authors tested HARVEST against the gold standard (human-curated data).

• Accuracy: The AI agreed with human experts 91% of the time.
• The "Unit Conversion" Win: Humans often make silly mistakes like confusing "nanometers" with "micrometers" (a 1,000x error). The AI made fewer of these specific mistakes than the humans did.
• Coverage: Because it's so cheap, HARVEST read thousands of "boring" patents that humans skipped because they didn't seem worth the effort. This revealed hidden gems.

4. The New Benchmark: H-Bench (The "Final Exam")

Once they unlocked the data, they realized the AI models used for drug discovery were actually quite weak.

• The Analogy: Imagine a student who memorized the answers to a specific practice test (the old public data). They get an A. But when you give them a new test with new questions (the HARVEST data), they fail.
• The Test (H-Bench): The authors created a new, strict exam called H-Bench using the newly unlocked data.
• The Result: When they tested a top-tier AI model (Boltz-2) on this new exam, it struggled. It could guess well if the drug looked like something it had seen before, but if the drug was a new shape or the protein was a new type, the AI got lost.
• The Lesson: Current AI hasn't truly learned the "physics" of how drugs stick to proteins; it's mostly just memorizing patterns. We need to train them on this new "Dark Data" to make them smarter.

Why This Matters

• Democratization: Before, only rich companies with expensive subscriptions could access this data. Now, any university or small lab can download this dataset for free.
• Speed: We can now update our knowledge of drug interactions in real-time as new patents are filed, rather than waiting years for humans to catch up.
• The Future: This proves that "Dark Data" isn't just a problem; it's a goldmine waiting for the right AI tools to dig it up.

In short: HARVEST is a robotic librarian that cleaned up a dusty, chaotic basement of millions of drug patents in one week for pennies, giving the world a massive new library of knowledge to help cure diseases faster.

Technical Summary

1. Problem Statement

Pharmaceutical patents contain the world's largest repository of experimental protein–ligand interaction (PLI) data, including Structure–Activity Relationship (SAR) tables with binding affinities for thousands of compounds. However, this data is effectively "dark":

• Inaccessibility: While technically public, the data is trapped in unstructured archives (PDFs, complex XML tables, chemical diagrams) that existing databases (like BindingDB) have failed to systematically capture.
• Generalization Crisis: AI models for drug discovery are limited by sparse training data. They struggle to generalize to novel chemical scaffolds or uncharacterized protein targets because they are trained on the same limited, manually curated public datasets.
• Economic Bottleneck: Manual curation is too slow and expensive. Processing the full USPTO patent corpus manually would require over 55 years of continuous expert labor, creating a barrier that confines high-quality data to expensive commercial platforms.

2. Methodology: The HARVEST Pipeline

The authors present HARVEST (High-throughput Agent Retrieval of Values for Evaluated Small-molecules and Targets), a multi-agent Large Language Model (LLM) pipeline designed to autonomously extract structured bioactivity records from USPTO patent archives.

Key Architectural Features:

• Multi-Agent Decomposition: Instead of a single monolithic prompt, the system uses five specialized sequential agents to mitigate hallucination and context loss in long documents (up to 500k+ tokens):
  1. Agent 1 (Target Extraction): Identifies biological targets, organisms, and assay conditions.
  2. Agent 2 (Activity Extraction): Extracts quantitative metrics (IC50, Ki, Kd, EC50) and links them to specific compounds and targets.
  3. Agent 3 (Compound Mapping): Resolves internal patent aliases (e.g., "Example 1") to IUPAC names or embedded identifiers.
  4. Agent 4 (Chemical Resolution): Converts chemical identities to canonical SMILES. Crucially, it parses ChemDraw binary (CDX) files directly from the patent XML to avoid errors associated with MOL file corruption or name-to-structure conversion tools.
  5. Agent 5 (Protein Resolution): Maps protein names to canonical UniProt identifiers, handling ambiguous nomenclature and species assignment (defaulting to Homo sapiens).
• Data Source: The system processes USPTO bulk XML data (APPDT), which includes embedded chemical structures and structured tables, avoiding the need for error-prone OCR on PDFs.
• Deduplication: A graph-based algorithm clusters related patent family filings (continuations, divisions) to ensure each unique experimental observation is counted only once.
• Cost Efficiency: The pipeline operates at $0.11 per document, leveraging parallel processing and prompt caching.

3. Key Contributions

1. HARVEST Dataset: The extraction of 3.36 million activity records from 164,877 patent archives (specifically 40,902 patents containing extractable data).
   • Recovered 365,713 unique Murcko scaffolds and 1,108 protein targets entirely absent from BindingDB.
   • Completed a task requiring 55+ years of manual labor in under one week.
2. H-Bench (Benchmark): A structurally guaranteed "held-out" benchmark derived from HARVEST data that is absent from all existing public repositories.
   • Designed to test generalization in two dimensions: Novel Chemistry (new scaffolds on known targets) and Novel Biology (known scaffolds on uncharacterized targets).
   • Includes a "Valid" subset (low structural similarity to training data) and a "Common" subset (higher similarity) to rigorously evaluate model performance without data leakage.
3. Agentic AI Framework: Demonstrates that decomposing complex document understanding into specialized agents achieves human-level fidelity (91% agreement) at 3,500x the throughput of manual curation.

4. Key Results

Data Quality and Coverage:

• Fidelity: HARVEST achieved 91% agreement with human-curated BindingDB records on matched patents.
• Error Reduction: The automated system exhibited lower unit-conversion error rates (e.g., nM vs. µM confusion) than manual curators.
• Novelty: While 34.1% of targets overlap with BindingDB, HARVEST covers 12.7% of targets exclusively, and for shared targets, it provides 37% novel interactions and 43% novel scaffold clusters.
• Complementarity: The datasets are asymmetrically complementary; merging them significantly enriches SAR information for distinct protein subsets.

Model Evaluation (Boltz-2 on H-Bench):
The authors evaluated the leading open-source structure-based model, Boltz-2, on H-Bench to test generalization:

• Two-Dimensional Generalization Gap: Performance degraded significantly when:
  1. The chemical scaffold was novel (not seen in training).
  2. The protein target lacked prior public bioactivity data.
• Metrics:
  • Known Targets + Familiar Chemistry: AUC-ROC ≈ 0.70.
  • Known Targets + Novel Chemistry: AUC-ROC ≈ 0.63.
  • Novel Targets + Novel Chemistry: AUC-ROC ≈ 0.52 (near random).
• Conclusion: Current models rely heavily on structural similarity to training data rather than learning transferable binding physics. They fail to generalize to truly novel biological or chemical spaces.

5. Significance and Impact

• Democratization of Data: By reducing the cost of patent mining to ~$0.11/doc, HARVEST allows academic and smaller research groups to access proprietary-scale data previously locked behind expensive commercial subscriptions (e.g., Reaxys, GOSTAR).
• Ending the "Dark Data" Era: The paper proves that the era of inaccessible patent data is ending, converting billions of dollars of R&D investment into open, machine-actionable scientific infrastructure.
• Benchmarking Standard: H-Bench addresses the critical flaw in AI drug discovery where models are evaluated on data they were trained on. It provides a rigorous, leakage-free standard to measure true generalization capabilities.
• Future Roadmap: The success of HARVEST suggests that similar agentic approaches can be extended to other jurisdictions (e.g., China's CNIPA, which is moving to XML) and other domains (clinical trials, regulatory filings), fundamentally changing how expert knowledge is extracted from unstructured text.

In summary, HARVEST represents a paradigm shift in medicinal chemistry data curation, combining agentic AI scalability with high-fidelity extraction to unlock a massive, previously inaccessible reservoir of scientific knowledge, while simultaneously exposing the current limitations of AI models in generalizing to novel biological and chemical spaces.