🧬 biology

A z-averaged ensemble of three orthogonal models improves protein–ligand virtual screening on leakage-controlled benchmarks

This paper demonstrates that a z-averaged ensemble of three representationally orthogonal deep-learning models significantly improves protein–ligand virtual screening performance on leakage-controlled benchmarks by achieving genuine out-of-distribution generalization, thereby overcoming the limitations of single-model approaches on novel targets.

Original authors: Yuguang Mu, Yang Liu, weifeng li

Published 2026-07-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuguang Mu, Yang Liu, weifeng li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Problem: The "Cheat Sheet" Trap

Imagine you are taking a difficult exam to predict which drugs will work on a specific virus. For years, computer models have claimed to be geniuses, scoring 90% or higher on practice tests.

However, researchers discovered these models were cheating. The practice tests (benchmarks) used "homology leakage." This is like giving a student a practice test where the questions are almost identical to the ones they studied in their textbook. The models weren't actually learning how to find new drugs; they were just memorizing patterns from proteins they had already seen during training.

When scientists created "leakage-controlled" tests—where the virus is completely new and has no relatives in the model's training data—the models' scores crashed. They dropped from 90% down to about 70-75%. It turns out that making a single, super-smart model wasn't working anymore.

The Solution: The "Three-Headed Monster"

Instead of trying to build one perfect, super-intelligent model, the authors tried a different approach: Teamwork.

They took three existing models that were built very differently and combined them. Think of it like hiring three different detectives to solve a mystery:

Detective A (BIND-z): Looks only at the protein's genetic code (the sequence).
Detective B (SaProt-v2): Looks at the genetic code but also pays attention to the 3D shape of the protein's "folding."
Detective C (DrugCLIP): Doesn't look at the code at all; instead, it looks at the 3D physical structure of the protein's pocket and compares it to the drug's shape.

How They Combined Them: The "Z-Score" Average

You can't just add their scores together because they speak different "languages." Detective A might give a score of 100, while Detective C gives a score of 0.5.

The authors used a clever trick called z-score averaging.

The Analogy: Imagine three judges in a talent show. Judge 1 is harsh and gives scores between 1 and 5. Judge 2 is lenient and gives scores between 80 and 100. Judge 3 is weird and gives scores between -10 and +10.
If you just add their numbers, Judge 2's opinion dominates.
Instead, the authors asked: "How much better is this contestant than the average for this specific judge?"
They converted every score into a "relative rank" (a z-score) and then took the average. This gave every detective an equal voice, regardless of their scoring style.

The Results: Why Three is Better Than One

When they tested this "three-detective team" on the hard, new viruses (the leakage-controlled benchmarks), they found something surprising:

They didn't agree much: The three models often gave different answers for the same drug. Their opinions were "orthogonal" (independent). When one model was confused, the others were often right.
The Team Won: By averaging their independent opinions, the team improved the accuracy by a small but statistically significant amount (from 0.817 to 0.834).
The "Leakage" Gap Shrank: The team was less reliant on having seen similar viruses before. Adding the third detective (DrugCLIP) made the whole group more robust against "cheating" by memorization.

The Key Takeaway

The paper argues that diversity is more important than raw power.

Old Way: Try to build one giant, super-capable model. (This has stalled).
New Way: Take three different, moderately capable models that look at the problem from totally different angles, and let them vote.

Because the models make different kinds of mistakes, averaging them cancels out the errors. The paper concludes that for finding new drugs on truly novel targets, inter-model orthogonality (having different perspectives) is the secret sauce, not just making the models bigger or smarter.

What They Didn't Claim

They did not claim this cures diseases or works in a hospital yet.
They did not claim this works on every possible benchmark (it actually struggled on one specific dataset called LIT-PCBA).
They did not claim that adding a fourth model would definitely help; they suggest that adding a fourth model that thinks like the first two would be useless. You need a new perspective to get more benefit.

Technical Summary: A z-Averaged Ensemble of Three Orthogonal Models for Protein–Ligand Virtual Screening

Problem Statement
Deep-learning models for protein–ligand virtual screening (VS) have reported high performance (AUROC > 0.90) on standard benchmarks like DUD-E. However, the authors argue these metrics are inflated by two primary failure modes:

Training-Test Sequence Homology Leakage: Most benchmark proteins have near-identical homologues in the training sets of models trained on BindingDB or PDBBind. Pretrained protein-language model embeddings make generalization trivial in these cases, allowing models to rely on homology rather than true binding physics.
Decoy Bias: Benchmarks like DUD-E contain structural asymmetries where actives and decoys differ in physico-chemical descriptors (e.g., rotatable bonds, polar surface area), allowing classifiers to discriminate without conditioning on the protein.

When evaluated on "leakage-controlled" targets (where training-test homology is strictly filtered, e.g., <30% identity), single-model performance stalls in the 0.70–0.75 AUROC range. Recent architectural improvements (fine-tuning, pocket-aware layers, contrastive losses) have failed to significantly close this gap on truly novel targets.

Methodology
The authors propose a post-hoc, model-agnostic ensemble strategy rather than building a single, more complex model. The approach involves combining three independently trained, representationally orthogonal scoring models:

BIND-z: A sequence-only model using a frozen ESM-2 encoder with cross-attention and Huber affinity heads, trained on a homology-filtered BindingDB subset.
SaProt-v2: A structure-aware sequence encoder (frozen SaProt-650M with AA+3Di tokens) trained with a contrastive InfoNCE loss and an attention-pool head.
DrugCLIP: A published CLIP-style retrieval model aligning 3D pocket-atom features (Uni-Mol) with 2D ligand features.

Fusion Strategy:
The models produce scores on different scales. To combine them, the authors employ a per-target z-score normalization followed by arithmetic averaging. For each target $t$ , the raw score $s_{m,t,i}$ for ligand $i$ from model $m$ is normalized:
$z_{m,t,i} = \frac{s_{m,t,i} - \mu_{m,t}}{\sigma_{m,t}}$
where $\mu$ and $\sigma$ are the mean and standard deviation of the model's scores over all ligands for that specific target. The ensemble scores are then:

ens2: Average of BIND-z and SaProt-v2.
ens3: Average of BIND-z, SaProt-v2, and DrugCLIP.

Key Contributions

Identification of Orthogonality: The authors demonstrate that the three models are weakly correlated at the per-ligand level. The Pearson correlation between the two sequence-based models (BIND-z and SaProt-v2) is $r \approx 0.22$ , while DrugCLIP shows near-zero correlation ( $r \approx 0.05\text{--}0.08$ ) with either sequence model. This suggests DrugCLIP provides genuinely independent information.
Statistically Significant Gain: The three-model ensemble (ens3) achieves a mean AUROC of 0.834 across 204 pooled DUD-E and DUD-AD evaluations, compared to 0.817 for the best two-model ensemble (ens2). This represents a +0.017 AUROC gain (paired $t=3.50$ , $p=5.8 \times 10^{-4}$ ). The gain is consistent directionally across six benchmarks, including strict leakage-clean subsets.
Reduced Homology Dependence: The ensemble reduces reliance on training-test homology. The "leakage gap" (performance drop when moving from homology-rich to leakage-clean targets) is +0.085 for ens3, compared to +0.091 for ens2. This indicates the ensemble generalizes better to out-of-distribution targets.

Results

Zero-Shot Performance: On the primary pooled benchmark ( $n=204$ ), ens3 outperforms ens2 with high statistical significance. On strict leakage-clean subsets (e.g., DUD-E-strict, $n=7$ ), ens3 reaches 0.779 AUROC, a significant improvement over single models, though statistical significance is limited by sample size.
Failure Modes: The ensemble does not solve the LIT-PCBA benchmark (absolute AUROC ~0.608), which is designed to be resistant to current sequence/pocket-only models.
In-Distribution Deployment: When evaluated in an "in-distribution" regime (where training data includes homologs of test targets), the ensemble (combining an all-data SaProt tower and BIND-full) still yields a +0.012 to +0.015 AUROC gain over the strongest single component, confirming that orthogonality benefits deployment even without strict zero-shot constraints.
Single-Model Limits: The authors systematically tested single-model improvements (encoder fine-tuning, pocket redesigns, alternative losses, pose-aware layers) and found them to yield negligible or negative gains on leakage-clean targets, suggesting the bottleneck is the training distribution, not model capacity.

Significance and Claims
The paper argues that inter-model orthogonality, rather than single-model capacity, is the practical lever for improving virtual screening on novel protein targets.

Mechanism: The gain arises from variance reduction on statistically near-independent error structures. Adding a fourth model with similar architecture (e.g., another ESM-based scorer) would likely yield diminishing returns due to high error correlation.
Practical Implication: A simple, post-hoc fusion of existing, diverse models is more effective than training a single, larger, or more complex model.
Benchmarking: The authors advocate for the routine reporting of "leakage-clean" subsets alongside standard benchmarks to distinguish between memorization and true generalization.
Scope: The ensemble is presented as a robust solution for targets lacking close training homologs, while a separate "deployment ensemble" (using unfiltered training data) is recommended for scenarios where homologs are available and inference cost is a secondary concern.

The authors conclude that the path forward for trustworthy virtual screening lies in combining diverse, orthogonal signals rather than pursuing incremental architectural capacity increases within a single model family.