A Large-Scale Dataset and Benchmark: Do Protein-Ligand Models Learn Binding Sites or Just Binding Likelihood?

Technical Summary: InteractBind – A Large-Scale Dataset and Benchmark for Protein–Ligand Binding Site Localization

Problem Statement
Protein–ligand modeling is central to computational drug discovery, yet existing benchmarks primarily evaluate models on binary binding prediction (does interaction occur?) or binding affinity regression (how strong is the interaction?). These tasks measure whether a model predicts the occurrence or strength of binding but provide limited evidence regarding whether models can localize specific binding sites or identify the underlying non-covalent interactions that drive molecular recognition. Current sequence-based datasets rarely offer residue-level interaction annotations at scale, making it difficult to determine if learned interaction maps genuinely capture physicochemical contact patterns or merely correlate with binary binding labels. Consequently, there is a lack of systematic evaluation for the fine-grained capability of models to localize binding residues and distinguish between different types of non-covalent interactions.

Methodology
To address this gap, the authors introduce InteractBind, a large-scale protein–ligand dataset and benchmark derived from experimentally resolved complexes in the Protein Data Bank (PDB).

• Dataset Construction: The dataset comprises approximately 100,000 protein–ligand pairs covering ~11,000 unique proteins and ~9,000 unique ligands. For each pair, the authors generate:

  • Sequence Representations: Canonical FASTA and structure-aware (SA) sequences for proteins; SMILES and SELFIES for ligands.
  • Interaction Annotations: Using tools like PLIP and GetContacts, the authors detect six major non-covalent interaction types: hydrogen bonds, salt bridges, $\pi$–$\pi$ stacking, cation–$\pi$ interactions, hydrophobic contacts, and van der Waals forces.
  • Interaction Maps: These interactions are projected onto sequence-level maps, creating $m \times n$ matrices (ligand atoms $\times$ protein residues) with weighted interaction scores. An aggregated map defines the protein binding site (residues participating in at least one interaction).
  • Labels: Binary binding labels (derived from docking-based affinity scores: $\le -7$ kcal/mol as positive, $> -5$ kcal/mol as negative) and affinity values.

• Benchmark Design:

  • Core Task: Binding-site localization. The primary evaluation metric is Binding Residue Hit Rate at Top-K (BRHR@K), which measures whether the top-$K$ predicted protein residues overlap with ground-truth binding residues.
  • Splits: To ensure realistic generalization assessment, the dataset includes:
    • In-Distribution (ID): An affinity-aware split using 5-fold cross-validation.
    • Out-of-Distribution (OOD): Four protein similarity-controlled splits (max train-test similarity of 25%, 28%, 31%, 33%) to evaluate performance on less familiar protein targets.
  • Baselines: Eight representative sequence-based, interaction-aware models (e.g., MolTrans, TransformerCPI, DrugBAN, FusionDTI) were evaluated.

Key Contributions

1. InteractBind Dataset: A new large-scale resource providing not only binary labels and affinity scores but also residue-level binding-site annotations and six distinct non-covalent interaction maps.
2. Fine-Grained Evaluation Paradigm: The formulation of binding-site localization as a core task, supported by BRHR metrics and interaction-type-specific analysis, moving beyond pair-level outcomes.
3. Systematic Benchmarking: The evaluation of eight state-of-the-art models reveals that while interaction-aware architectures show measurable binding-site localization signals, there is substantial room for improvement, particularly in identifying specific physicochemical contact patterns.

Results

• Binary vs. Localization Performance: A significant discrepancy exists between binary binding prediction and binding-site localization. The best-performing model, FusionDTI, achieved 98.3% AUROC for binary prediction but only 21.6% BRHR@1 for binding-site localization. This indicates that strong binary prediction does not guarantee reliable identification of the specific residues involved in binding.
• Interaction-Type Variability: Performance varies markedly across non-covalent interaction types. Models localize common, spatially broad contacts (van der Waals, hydrophobic, hydrogen bonds) more effectively (e.g., FusionDTI reached 33% BRHR@5 for van der Waals) than specific geometric arrangements like $\pi$–$\pi$ stacking (18.2%) and cation–$\pi$ interactions (~16.1%).
• Generalization: Under protein similarity-controlled OOD splits, performance for both binary prediction and binding-site localization decreases as the similarity between training and test proteins drops, highlighting the difficulty of localizing binding sites for novel protein targets. Ligand similarity had a comparatively smaller effect on performance.

Significance and Claims
The paper claims that InteractBind establishes a new benchmark paradigm that encourages the development of more interpretable and physically grounded protein–ligand models. The authors argue that current models often rely on dataset-specific correlations to predict binding occurrence without truly learning the mechanistic determinants of molecular recognition. By providing explicit residue-level supervision and interaction-type-specific annotations, InteractBind enables a systematic assessment of whether models localize binding sites and identify the non-covalent interactions that stabilize them. The results suggest that while current interaction-aware models capture some binding-site signals, they are not yet reliable for fine-grained localization, particularly for specific interaction types or novel protein targets. The authors position this work as a step toward shifting evaluation from outcome-only metrics to fine-grained assessment, which is critical for improving the reliability of computational drug discovery pipelines and avoiding misleading biological conclusions.

Limitations
The authors explicitly note that the current benchmark focuses on sequence-level evaluation and does not yet include a structure-level benchmark. They acknowledge that extending evaluation to 3D structures is challenging due to pose ambiguity and conformational flexibility, which can introduce noise and reduce interpretability compared to residue-level sequence-based assessment.