💻 bioinformatics

Interrogating contrastive learning embeddings for structure-based virtual screening: a case study on DrugCLIP

This paper systematically evaluates DrugCLIP's contrastive learning embeddings for structure-based virtual screening, revealing that while its pocket embeddings offer fast and robust structural similarity search and its ligand embeddings capture pocket-aware chemical patterns with strong generalization to unseen targets, performance is significantly limited by conformational variations and inaccuracies in predicted protein pockets.

Original authors: Sanchez Utges, J., Jones, D. T., Orengo, C.

Published 2026-09-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sanchez Utges, J., Jones, D. T., Orengo, C.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Finding a new medicine is a slow, expensive gamble. It can take fifteen years and cost over a billion dollars to bring a single drug to market, mostly because most potential candidates fail along the way. To reduce this waste, scientists use computer programs to sift through millions of chemical compounds, looking for the few that might stick to a specific disease-causing protein. This process, called virtual screening, acts as a filter, narrowing down a massive library of possibilities to a manageable list for real-world testing. For decades, this relied on matching the physical shape of a drug to the shape of a protein's binding site, a task that is computationally heavy and often too slow for the vast number of proteins and chemicals now available.

Recently, a new approach has emerged that treats this search like a language translation problem. Instead of calculating the physics of every single interaction, these methods use artificial intelligence to translate both the protein's binding pocket and the drug molecule into a shared, abstract language of numbers. In this shared space, a drug that fits a protein well will have a number pattern very similar to the protein's own pattern, while a bad match will look very different. This allows computers to find good matches by simply comparing these number patterns, a process that is incredibly fast. One such tool, called DrugCLIP, has shown great promise, but until now, scientists did not fully understand what these abstract number patterns actually represented. They knew the tool worked, but they did not know if it was truly understanding the chemistry or just memorizing the answers.

A team of researchers at University College London decided to open the black box of DrugCLIP to see exactly what was happening inside. They treated the tool not as a magic wand, but as a system that could be measured and understood. Their investigation revealed that the tool had learned something far more useful than just matching drugs to proteins. The researchers found that the tool's internal representation of a protein's binding pocket was so accurate that it could identify similar pockets across different proteins better than any existing method, and it did so more than one hundred times faster. This was a surprise because the tool was never explicitly taught to find similar pockets; it was only taught to find drugs that bind. Yet, in the process of learning to find drugs, it had inadvertently learned to recognize the shape and structure of the pockets themselves with remarkable precision.

The team also examined how the tool handled the physical reality of molecules. Proteins and drugs are not rigid statues; they wiggle and shift. The researchers tested whether the tool would get confused if a drug molecule was shown in a slightly different shape or if the protein was in a slightly different position. They found the tool was remarkably robust. Even when the same drug was shown in dozens of different wiggling shapes, the tool recognized them as the same molecule. Similarly, when the protein pocket was shown in different states—some bound to a drug, some empty, and some predicted by other AI models—the tool's internal map remained consistent. However, the study also drew a clear line where the tool's performance began to slip. When the researchers used predicted protein structures that had the wrong amino acids in the binding site, or when the side chains of the protein were pointing in the wrong direction, the tool's ability to find the right drug dropped significantly. This indicated that while the tool is powerful, it is still dependent on having an accurate picture of the protein's physical structure.

Perhaps the most important finding was a test of whether the tool was relying on memorization. The researchers created a strict test where they asked the tool to find drugs for proteins and chemicals it had never seen before, ensuring it could not simply recall answers from its training data. The results were encouraging. For new proteins with completely new chemistry, the tool still managed to rank the correct drug within the top one percent of all possible candidates in about 55 to 75 percent of cases. This proved that the tool was genuinely learning the rules of how proteins and drugs interact, rather than just memorizing a list of known pairs. It showed that the tool could generalize its knowledge to entirely new situations, a critical requirement for real-world drug discovery.

The study concluded that while tools like DrugCLIP represent a massive leap forward in speed and capability, they are not yet perfect. Their success hinges on the accuracy of the protein structure provided to them. If the starting model of the protein is slightly off, or if the predicted binding site includes the wrong atoms, the tool's performance suffers. The researchers suggest that the next step in improving these systems is not just better AI, but better methods for predicting the precise shape of protein pockets. By combining these fast, smart screening tools with more accurate structural predictions, scientists can move closer to a future where finding new medicines is faster, cheaper, and more reliable. The work demystifies a complex technology, showing that it is a powerful new lens for seeing the molecular world, provided the lens itself is held steady.

Technical Summary: Interrogating Contrastive Learning Embeddings for Structure-Based Virtual Screening: A Case Study on DrugCLIP

Problem Statement
Structure-based virtual screening (SBVS) has been reframed as a retrieval problem using contrastive learning methods like DrugCLIP, which project protein pockets and ligands into a shared embedding space. While these methods offer significant speed advantages over traditional docking, the nature of the abstract representations they learn remains opaque. Specifically, it is unclear what these embeddings encode regarding structural and chemical similarity, whether they generalize to unseen proteins and chemistries or merely memorize training data, and how performance degrades under realistic screening conditions (e.g., using apo structures, AlphaFold models, or predicted pockets).

Methodology
The authors conducted a systematic dissection of DrugCLIP's latent space through a series of benchmarks and analyses:

Pocket Matching Benchmark: The study evaluated DrugCLIP's ability to classify pocket pairs as similar or dissimilar using the ProSPECCTs suite (9 datasets). Performance was compared against state-of-the-art descriptors (EPoCS, NRGRank, PocketVec) and baseline methods (sequence identity, ligand code). Variants of DrugCLIP were tested, including different pocket definition thresholds (6 Å vs. 12 Å) and representations derived from screening scores (Sim Vec, Rank Vec).
Representation Analysis: Using four protein-ligand datasets (COACH420, HOLO4K, scPDB, PDBBind), the authors analyzed:
- Conformer Robustness: Cosine similarity between embeddings of different conformers of the same ligand.
- Chemical vs. Embedding Similarity: Correlation between ligand embeddings and Tanimoto similarity of ECFP4 fingerprints.
- Structural vs. Embedding Similarity: Correlation between pocket embeddings and structural similarity (sidechain RMSD) for pockets binding the same ligand.
Bound Ligand Recovery Benchmark: DrugCLIP was tasked with identifying the correct bound ligand from a library of ~49,000 CCD compounds against query pockets. A rigorous "de-leakage" funnel was applied to assess generalization:
- Level 1: Protein sequence identity (SI) relative to the BioLiP fine-tuning set.
- Level 2: Ligand frequency in the fine-tuning set.
- Level 3: Chemical similarity (ECFP4 TS) of unseen ligands to the fine-tuning set.
Performance Degradation Analysis: The study quantified performance drops when moving from ideal experimental conditions to realistic scenarios:
- Using AlphaFold DB (AFDB) predicted structures instead of PDB structures.
- Using apo (unbound) or holo (bound) conformations from the AHoJ-DB.
- Using predicted pockets (via P2Rank) instead of experimentally defined sites.
- Negative controls included random surface patches and incorrect pocket predictions.

Key Results

Pocket Similarity Search: DrugCLIP pocket embeddings, despite not being explicitly trained for similarity search, achieved state-of-the-art performance (AUROC ≥ 0.95 on 7/9 ProSPECCTs sets), outperforming dedicated descriptors like PocketVec and EPoCS on evolutionarily divergent targets. The method is over 100 times faster than existing structural descriptors.
Embedding Characteristics:
- Pocket Embeddings: Showed strong structural coherence. Pocket embedding similarity correlated negatively with sidechain RMSD ( $\rho = -0.83$ ), indicating that structurally similar pockets cluster together.
- Ligand Embeddings: Showed only a modest correlation with chemical fingerprints ( $\rho = 0.23$ ). The authors attribute this to the model encoding a "pocket-aware" chemical similarity, where chemically distinct ligands that bind similar pockets are placed close in the latent space.
- Robustness: The model is robust to ligand conformer variations, with mean cosine similarity > 0.9 for different conformers of the same molecule.
Generalization vs. Memorization: In the bound ligand recovery task, DrugCLIP recovered the correct ligand within the top 1% for 84.8% of the full dataset. However, performance dropped significantly under strict de-leakage conditions (novel proteins with SI ≤ 30% and unseen ligand chemistry), recovering the ligand in the top 1% for 55–75% of novel targets. This confirms generalization beyond simple memorization, though with reduced efficacy on novel chemistries.
Impact of Realistic Conditions:
- Conformational Variation: Performance degradation in AFDB, apo, and holo scenarios is primarily driven by sidechain reorientation. AFDB models tend to resemble holo conformations more closely than apo structures.
- Pocket Prediction: Using predicted pockets (P2Rank) caused the most significant performance drop. This degradation is attributed to residue membership mismatch (low Intersection over Union, IOU) rather than spatial distance alone. High IOU between predicted and observed pockets correlated strongly with preserved screening scores.

Significance and Claims
The paper claims to provide the first systematic dissection of representations learned by a contrastive learning method for virtual screening. The authors assert that:

State-of-the-Art Efficiency: DrugCLIP sets a new benchmark for pocket similarity search, offering high accuracy with a speed advantage of >100x over current methods.
Interpretability: The latent space is structured and interpretable; pocket embeddings capture geometric and structural context, while ligand embeddings reflect a context-dependent chemical similarity.
Generalization Boundaries: The study delineates the practical limits of DrugCLIP, demonstrating that while it generalizes to unseen targets, its performance is highly sensitive to the accuracy of pocket definition (residue selection) and sidechain orientation.
Framework for Future Work: The analytical framework presented is transferable to other contrastive pocket-ligand encoders (e.g., DrugHash, BindCLIP, ConGLUDe), offering a template for interpreting their latent spaces and identifying bottlenecks such as pocket prediction accuracy as a key area for improvement.

The authors conclude that accurate pocket selection and sidechain placement are critical for maximizing the utility of contrastive screening approaches, suggesting that integrating better pocket predictors or conformational refinement tools (like GenPack) could further enhance performance.

More like this