BioMiner: A Multi-modal System for Automated Mining of Protein-Ligand Bioactivity Data from Literature

The paper introduces BioMiner, a multi-modal system that decouples bioactivity semantic interpretation from ligand structure construction to automate the extraction of protein-ligand data from literature, validated by a new comprehensive benchmark and demonstrated through applications that significantly improve downstream model performance, accelerate drug discovery workflows, and enhance dataset annotation efficiency.

The Gist

Imagine you are a treasure hunter looking for a specific type of gold: protein-ligand bioactivity data. This is the "secret recipe" that tells scientists how well a specific drug molecule (the ligand) sticks to a specific disease-causing protein. This data is the fuel for modern drug discovery.

The problem? This gold is buried inside millions of scientific papers. For decades, the only way to find it was to hire armies of human experts to read every paper, find the numbers, draw the chemical structures, and type them into a database by hand. It's slow, expensive, and the literature is growing faster than the humans can read.

Enter BIOMINER and its companion map, BIOVISTA. Think of them as a high-tech, automated mining crew designed to solve this problem.

The Problem: A Messy Library

Imagine a library where the books are written in a mix of languages, the important numbers are hidden inside complex diagrams, and the chemical structures are drawn as "skeletons" with blank spots where different parts can be swapped out (these are called Markush structures).

• The Old Way: A human librarian tries to read the book, figure out the chemistry, and write it down. If the drawing is messy or the text is scattered across a table and a figure, the human might get confused or miss it entirely.
• The Challenge: You can't just ask a standard AI (like a basic chatbot) to "read this paper and give me the data." If you ask a general AI to draw a complex chemical structure from a sketch, it might hallucinate (make things up) or draw a molecule that is chemically impossible.

The Solution: BIOMINER (The Specialized Mining Crew)

The authors built BIOMINER, which isn't just one robot; it's a team of specialized agents working together. They use a clever strategy: separation of duties.

Instead of asking one robot to do everything at once (which leads to mistakes), they split the job into two distinct teams:

1. The "Meaning" Team (Bioactivity Agents):

   • Job: Read the text, tables, and figures to find the numbers and names.
   • Analogy: Imagine a team of detectives reading a mystery novel. They don't need to draw the crime scene; they just need to find the clues: "Who did it?" (Protein), "What was the weapon?" (Ligand), and "How effective was it?" (Bioactivity value like IC50).
   • How they work: They use a smart AI (a Multi-Modal Large Language Model) that is great at understanding context and reasoning across different types of media (text and images).

2. The "Builder" Team (Chemical Structure Agents):

   • Job: Turn the sketches and "skeleton" drawings into perfect, 3D-ready chemical models.
   • Analogy: Imagine a master architect and a construction crew. The architect (the AI) looks at a blueprint and says, "This is the main frame, and here are the interchangeable windows." The construction crew (specialized chemistry tools) then takes those instructions and physically builds the exact house, ensuring the bricks fit and the roof doesn't collapse.
   • The Secret Sauce: The AI never tries to draw the molecule itself. It just identifies the parts and tells the chemistry tools exactly how to assemble them. This prevents the AI from making "chemical hallucinations."

The Magic Trick (CSG-VSR):
The paper calls their method Chemical-Structure-Grounded Visual Semantic Reasoning.

• Translation: The AI looks at the picture, finds the chemical drawing, and uses "grounding" (anchoring its thoughts to the actual pixels) to understand what it sees. Then, it hands the "assembly instructions" to a chemistry tool that guarantees the final result is a real, valid molecule.

The Map: BIOVISTA

To prove their mining crew works, they needed a test. They built BIOVISTA, a massive benchmark dataset.

• Analogy: Think of this as a "Gold Standard" training ground. They took 500 real scientific papers, had human experts painstakingly extract every single piece of data, and used this as the "Answer Key."
• Why it matters: Before this, there was no standard way to test if a robot was actually good at this. Now, researchers can run their systems against BIOVISTA to see if they are truly finding the gold or just guessing.

What Did They Achieve? (The Results)

The paper shows three amazing ways this system helps:

1. The Data Dump (Scale):
   They used BIOMINER to scan 11,683 scientific papers in just two days.

   • Result: They extracted over 82,000 data points.
   • Impact: They used this massive new dataset to train AI models for drug discovery. These models got 3.9% better at predicting how drugs work. In the world of AI, a 4% jump is like going from a bicycle to a sports car.

2. The Human-AI Team-Up (NLRP3 Case Study):
   They focused on a specific target for inflammation (NLRP3). They used a "Human-in-the-Loop" workflow.

   • How it works: BIOMINER does the heavy lifting (finding the data), and a human expert just double-checks the work.
   • Result: They doubled the amount of high-quality data available for this target in record time. This helped them find 16 new potential drug candidates that had never been seen before.

3. The Speed Run (PoseBusters):
   They tested how fast they could label existing 3D structures with their bioactivity data.

   • Result: They were 5 times faster than humans and actually more accurate (97% accuracy vs. 86% for humans). Humans get tired and miss things; the robot doesn't.

The Bottom Line

BIOMINER is a game-changer because it stops trying to force a general AI to do a specialized job. Instead, it acts like a smart project manager: it uses AI to understand the story and specialized chemistry tools to build the structure.

By unlocking the "locked" data in millions of papers, BIOMINER is helping scientists move faster from "reading about a drug" to "testing a real drug," potentially saving years of time and millions of dollars in the fight against diseases.

Technical Summary

1. Problem Statement

Protein-ligand bioactivity data is the cornerstone of modern drug discovery, underpinning Structure-Activity Relationship (SAR) analysis, QSAR modeling, and AI-driven virtual screening. However, the extraction of this data faces three critical bottlenecks:

• Multi-modal Complexity: Bioactivity data is distributed across heterogeneous modalities (text, tables, figures, and chemical structures), requiring robust cross-modal reasoning rather than unimodal extraction.
• Chemical Structure Reconstruction: Accurately recognizing and converting chemical structures, particularly Markush structures (which represent groups of related compounds via R-groups), is a major hurdle. Existing Optical Chemical Structure Recognition (OCSR) tools struggle to enumerate specific full molecular structures from these generic representations.
• Lack of Benchmarks: The field lacks standardized, large-scale benchmarks for rigorous evaluation, hindering the development of generalizable automated solutions.
• Scalability: Manual curation cannot keep pace with the exponential growth of scientific literature, creating a data scarcity bottleneck for AI models.

2. Methodology: The BIOMINER Framework

BIOMINER is a multi-modal agentic system designed to decouple bioactivity semantic interpretation from ligand structure construction. Instead of a brittle end-to-end approach, it employs a modular pipeline with specialized agents:

A. Core Architecture

The system processes documents through four stages:

1. Document Parsing: Uses MinerU to extract text, layout, and visual elements (figures/tables).
2. Bioactivity Measurement Extraction: Uses BIOMINER-INSTRUCT (a domain-specialized MLLM fine-tuned from Qwen3-VL-32B) to perform semantic reasoning across text and images to extract protein names, ligand identifiers, and bioactivity values (e.g., IC50, Ki, Kd).
3. Chemical Structure Resolution: This is the core innovation, utilizing a Chemical-Structure-Grounded Visual Semantic Reasoning (CSG-VSR) paradigm:
   • Detection & Recognition: MolDetv2 detects molecular regions, and MOLGLYPH (a custom OCSR model) converts them to SMILES.
   • Grounded Reasoning: The MLLM performs visual-semantic reasoning on augmented images (with indexed bounding boxes) to identify scaffold-R-group relationships and coreferences. Crucially, the MLLM handles relational reasoning but not symbolic construction.
   • Symbolic Construction: Domain tools (RDKit, OPSIN) deterministically "zip" the scaffold SMILES with enumerated R-group substituents (converted from text/IUPAC names) to generate chemically valid full structures.
4. Post-processing: Integrates the extracted semantic tuples with the resolved chemical structures to produce final protein-ligand-bioactivity triplets.

B. The BIOVISTA Benchmark

To evaluate the system, the authors introduced BIOVISTA, the largest benchmark for this task:

• Scale: 16,457 bioactivity entries and 8,735 unique chemical structures curated from 500 publications (indexed in PDBbind v2020).
• Diversity: Data sourced from text (15.8%), figures (11.6%), and tables (72.5%), with 48.7% of structures derived from challenging Markush representations.
• Tasks: Includes 6 evaluation tasks ranging from end-to-end triplet extraction to component-level tasks (Molecule Detection, OCSR, Coreference, Markush Enumeration).
• Rigorous Split: Strictly held-out test sets (90%) and validation sets (10%) to prevent data leakage.

3. Key Contributions

1. BIOMINER System: A novel framework that explicitly separates semantic reasoning from chemical symbolic construction, enabling the scalable enumeration of complex Markush structures—a task previously unaddressed at scale.
2. BIOVISTA Benchmark: A comprehensive, expert-curated dataset with a strict train/test split, establishing a new standard for evaluating bioactivity extraction.
3. CSG-VSR Paradigm: A mechanism that anchors MLLM reasoning to chemically grounded visual representations and delegates exact structure generation to deterministic chemistry tools, ensuring chemical validity.
4. Open Resources: Release of the benchmark, code, and pre-trained models (MOLGLYPH and BIOMINER-INSTRUCT) on GitHub.

4. Results and Performance

Benchmark Performance (BIOVISTA)

• End-to-End Extraction: BIOMINER achieved an F1 score of 0.323 for complete bioactivity triplets. This significantly outperforms a one-shot end-to-end baseline (F1 = 0.00037), highlighting the necessity of task decomposition.
• Component Performance:
  • Bioactivity Measurement Extraction: F1 = 0.626.
  • Chemical Structure Extraction (Full): F1 = 0.565; (Markush): F1 = 0.349.
  • Markush Enumeration (with coreference): F1 = 0.698.
• Error Analysis: The primary error sources were bioactivity measurement extraction (32.68%), OCSR inaccuracies (25.31%), and Markush enumeration (15.91%).

Practical Applications

1. Large-Scale Database Construction:
   • Extracted 82,262 data points from 11,683 papers in ~2 days (cost: ~$0.024/paper).
   • Impact: Pre-training deep learning models (GNNs) on this dataset improved downstream binding affinity prediction RMSE by 3.9% on independent test sets (PDBbind core and CSAR-HiQ) compared to models trained only on curated data.
2. Human-in-the-Loop (HITL) Workflow for NLRP3:
   • Curated 1,592 NLRP3 bioactivity entries from 85 papers in 26 hours (doubling the available ChEMBL data).
   • Impact: QSAR models trained on this expanded dataset showed a 38.6% improvement in EF1% metric. This led to the identification of 16 novel hit candidates with new scaffolds.
3. PoseBusters Annotation:
   • Annotated 242 protein-ligand structures with reported bioactivity.
   • Impact: Achieved a 5-fold speedup and 97.1% accuracy (vs. 86.4% for manual) in associating structures with bioactivity values.

5. Significance

• Unlocking Data: BIOMINER provides a scalable pathway to unlock vast amounts of bioactivity data previously trapped in unstructured literature, addressing the "data bottleneck" in drug discovery.
• Methodological Shift: By decoupling semantic reasoning from chemical construction, the system overcomes the limitations of pure LLMs in handling strict chemical constraints (like Markush enumeration).
• Accelerated Discovery: The demonstrated ability to rapidly build high-quality datasets directly translates to improved AI model performance and the successful identification of novel drug candidates (NLRP3 inhibitors).
• Standardization: The introduction of BIOVISTA fills a critical gap, enabling fair comparison and systematic advancement of automated scientific literature mining tools.

In summary, BIOMINER represents a significant leap forward in automated scientific knowledge extraction, combining the reasoning power of MLLMs with the rigor of domain-specific chemistry tools to solve a complex, multi-modal problem essential for modern drug discovery.