🧬 biology

Frequent Hitter Prediction Beyond Classification Models

This study demonstrates that direct regression of empirical Bayes-corrected hit rates and multi-label aggregation strategies outperform traditional threshold-specific binary classification for predicting frequent hitters in high-throughput screening, offering a more robust, cutoff-agnostic approach that improves early enrichment and global ranking.

Original authors: Samuli Näppi, Steffen Renner, Jutta Blank, Ansgar Schuffenhauer, Paul Selzer

Published 2026-08-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samuli Näppi, Steffen Renner, Jutta Blank, Ansgar Schuffenhauer, Paul Selzer

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the early stages of drug discovery, scientists run massive experiments called high-throughput screening. Imagine a laboratory where robots test hundreds of thousands of tiny chemical molecules against biological targets, looking for any sign that a molecule might stick to a disease-causing protein or disrupt a harmful cell process. The goal is to find the rare "hits"—molecules that show genuine promise as future medicines. However, the process is noisy. Many molecules produce false alarms. Some clump together in ways that block the test, others react chemically with the equipment, and some simply interfere with the way the machine reads the results. These troublemakers are known as "frequent hitters." They appear as active in dozens or even hundreds of different tests, not because they are powerful drugs, but because they are chemically noisy. If researchers cannot distinguish these noisy compounds from genuine drug candidates, they waste time and money chasing dead ends. The challenge is to build a system that can spot these frequent hitters early, sorting the signal from the noise before expensive experiments begin.

For years, the standard way to handle this problem has been to draw a hard line. Scientists would calculate how often a molecule appeared as a hit across all its tests and then set a fixed cutoff. If a molecule's hit rate was above that line, it was labeled a "frequent hitter" and discarded. If it was below, it was kept. This approach, while simple, has a significant flaw. It treats the decision as a simple yes or no, throwing away valuable information about molecules that sit right near the line. A molecule that is just barely above the cutoff is treated exactly the same as one that is a massive offender, while a molecule just below the cutoff is treated as perfectly safe, even if it is dangerously close to being a problem. Furthermore, if a lab decides to change its rules and wants to be stricter or more lenient, they have to build an entirely new computer model from scratch. This rigidity limits how well scientists can prioritize the most promising leads.

A team of researchers from the Swiss Federal Institute of Technology and Novartis BioMedical Research decided to rethink this process. Instead of forcing every molecule into a binary box, they asked if they could predict a continuous score that reflects the true likelihood of a molecule being a frequent hitter. They developed a new approach using advanced computer models that look at the shape of the molecules, known as graph neural networks. Rather than training these models to say "yes" or "no," they trained them to predict a specific number: a corrected hit rate that accounts for how many tests a molecule has actually undergone. This method, which uses a statistical technique called empirical Bayes to smooth out the noise, allows the model to see the full spectrum of risk. It can tell the difference between a molecule that is a clear nuisance, one that is a clear success, and those in the middle that require closer inspection.

The researchers tested this new method against the old way of doing things using two massive collections of data. One set came from a public database containing over a thousand different biological tests, and the other came from Novartis's own private library of screening results. They separated the data into groups based on whether the tests were done in test tubes (biochemical) or inside living cells (cellular), ensuring that the computer models were tested on chemical structures they had never seen before. They compared their new continuous prediction models against the traditional models that relied on fixed cutoffs. The results showed that the new approach was superior. The continuous models were better at ranking molecules correctly, especially when the researchers needed to be very strict about filtering out the worst offenders. They could identify the most problematic compounds earlier in the list, which is crucial for saving time in the lab.

Perhaps most importantly, the new method proved to be much more flexible. Because the model predicts a continuous score rather than a fixed label, scientists can adjust their decision rules at any time without retraining the computer. If a project requires a very high safety margin, they can simply raise the threshold for what counts as a problem. If they need to cast a wider net, they can lower it. The study found that this flexibility, combined with better accuracy, made the continuous approach a more practical tool for real-world drug discovery. The researchers also explored a second strategy where the model predicts the outcome for each specific test individually and then combines those predictions into a single score. This method also performed well, offering a different way to understand why a molecule might be causing trouble.

The findings suggest that the future of screening lies in moving away from rigid, black-and-white classifications toward a more nuanced understanding of risk. By treating frequent hitting as a spectrum rather than a category, these new models provide a clearer, more reliable way to prioritize which molecules deserve further study. This does not mean that the old methods are useless, but it does show that they are less efficient than they could be. The continuous models act as a robust baseline, capable of handling the messy reality of chemical data without discarding the subtle differences that often matter most. As drug discovery continues to rely on massive amounts of data, having a system that can adapt to changing needs and provide a detailed risk profile for every molecule will be essential for finding the next generation of life-saving medicines.

Technical Summary: Frequent Hitter Prediction Beyond Classification Models

Problem Statement

High-throughput screening (HTS) is fundamental to early drug discovery but is complicated by "frequent hitters"—compounds that repeatedly appear as active across diverse assays. These hits may stem from genuine polypharmacology or, more problematically, from assay interference (e.g., colloidal aggregation, chemical reactivity, fluorescence interference). Distinguishing between these mechanisms is critical for effective triage.

Current computational workflows for predicting frequent hitters typically rely on binary classification. These approaches reduce continuous hit rates to binary labels (hit vs. non-hit) based on a fixed threshold (e.g., top 10% or 20% of active compounds). This methodology introduces several limitations:

Information Loss: Discarding data near the decision boundary (the "gray zone") reduces the model's ability to rank compounds with intermediate risk.
Threshold Rigidity: Changing operational thresholds (e.g., moving from a 10% to a 1% cutoff) requires training entirely new models.
Coverage Bias: Naïve hit rates are unstable for compounds with low screening coverage, leading to noisy targets.

Methodology

The authors propose and evaluate three Graph Neural Network (GNN) strategies using data from PubChem BioAssay and a large proprietary Novartis screening corpus. The study utilizes scaffold-disjoint splits to ensure rigorous evaluation of generalizability.

1. Data Preparation and Target Construction

Data Sources: The study curated 1,105 PubChem assays and 1,212 Novartis assays, stratified into biochemical and cellular cohorts.
Hit Calling: A uniform R-score based on median absolute deviation (MAD) was applied to primary screening readouts to generate consistent per-assay hit labels, replacing inconsistent depositor labels where necessary.
Empirical Bayes Correction (EBHR): To address coverage bias, the authors computed an Empirical Bayes-corrected Hit Rate (EBHR). This involves fitting a Beta-Binomial model to the active-to-tested ratios (ATR) of compounds screened at least 50 times. The posterior mean provides a stabilized, continuous hit rate estimate.
Scaffold Splits: Five scaffold-disjoint splits (80/10/10 train/validation/test) were generated to prevent scaffold leakage.

2. Modeling Strategies

All models utilized Chemprop (v2.2.1) with a Message Passing Neural Network (MPNN) architecture and standard molecular graph featurization.

Strategy A: Threshold-Specific Binary Classifiers (Baseline)
- Models were trained to classify compounds as "frequent hitters" vs. "non-frequent hitters" based on specific percentiles of the training-set EBHR distribution (e.g., Top 1%, 2%, 5%, 10%, 20%).
- Compounds falling between the negative class cutoff (bottom 80%) and the positive class cutoff formed a "gray zone" and were excluded from training for that specific model.
- Separate models were required for each threshold.
Strategy B: Direct Regression on EBHR
- A single model per split was trained to predict the continuous EBHR value directly using Mean Squared Error (MSE) loss.
- This approach is cutoff-agnostic, allowing decision thresholds to be applied at inference time without retraining.
Strategy C: Multi-Label Prediction with Aggregation
- Models were trained to predict per-assay hit probabilities ( $p_{ij}$ ) for each compound-assay pair using masked binary cross-entropy.
- Compound-level scores were derived by aggregating per-assay probabilities. The authors used a coverage-weighted mean, where assays with more screened outcomes contributed more heavily to the final score.

Key Results

The models were evaluated using PR-AUC, ROC-AUC, Spearman correlation with observed EBHR, and Early Enrichment Factors (EF@1%).

Performance on Restricted Test Sets:
- When evaluated on test sets excluding the "gray zone" (mirroring the baseline training conditions), both Direct Regression and Multi-Label models matched or exceeded the performance of the threshold-specific baselines.
- The continuous models showed the most significant advantage at stringent cutoffs (e.g., Top 1%).
Performance on Full Test Sets:
- In a more realistic prospective setting where "gray zone" compounds are included, the binary baselines suffered a significant drop in PR-AUC, particularly at stringent thresholds.
- The continuous approaches (Regression and Multi-Label) maintained higher PR-AUC across all cohorts and definitions, demonstrating their ability to rank compounds across the full spectrum of hit rates.
Ranking and Enrichment:
- Direct Regression achieved a strong Spearman correlation ( $\rho \approx 0.70$ ) with observed EBHR, validating its utility for global ranking.
- Both continuous methods demonstrated superior Early Enrichment (EF@1%) compared to the baselines, particularly for the most stringent frequent-hitter definitions.
Robustness:
- Performance (Spearman correlation) remained high for test compounds with low-to-moderate Tanimoto distance to the training set, though it decreased for the most structurally distinct compounds, indicating the models learn generalizable features rather than merely memorizing specific scaffolds.

Significance and Claims

The paper claims that modeling frequent-hitter behavior as a continuous spectrum rather than a binary classification problem offers a more robust and maintainable solution for HTS triage.

Practical Baseline: The authors propose EBHR regression as a practical, cutoff-agnostic baseline. It unifies ranking and threshold-based decision-making within a single model, eliminating the need to retrain when operational priorities change.
Assay-Resolved Insights: The multi-label approach serves as a viable alternative when per-assay predictions are valuable for distinguishing between broad promiscuity and assay-specific interference patterns.
Superiority in Stringent Definitions: The study demonstrates that continuous approaches are particularly advantageous when identifying the most problematic frequent hitters (stringent cutoffs) and when ranking compounds that fall into the ambiguous "gray zone."

The authors emphasize that these models are intended to support, not replace, experimental verification. They provide a computational baseline to prioritize hit lists, acknowledging that frequent hitters can arise from both nuisance chemistry and genuine polypharmacology. The study is retrospective, and while the internal Novartis data and public PubChem data yield consistent patterns, prospective validation in live screening campaigns is noted as necessary to confirm operational impact.