🧬 biology

Benchmarking Data Augmentation Strategies for Drug--Target Interaction Prediction under Class Imbalance: A Controlled Comparative Study

This study presents a controlled benchmarking analysis of fourteen data augmentation strategies for drug-target interaction prediction under class imbalance, revealing that boundary-aware cleaning and latent-structure-aware augmentation outperform oversampling in improving F1-score and PR-AUC across four protein families.

Original authors: Vitor Magalhaes Silva, Khadidja Henni, Neila Mezghani

Published 2026-11-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Vitor Magalhaes Silva, Khadidja Henni, Neila Mezghani

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the vast, silent library of biology, there exists a fundamental question that drives modern medicine: which chemical compounds will bind to which proteins to alter a disease? This is the realm of drug-target interaction, a field where scientists attempt to map the invisible handshake between a potential medicine and the biological machinery it is meant to fix. The challenge is not a lack of data, but a lack of balance. In the real world, confirmed interactions are rare, like finding a few specific keys that fit a lock among millions of useless ones. The vast majority of possible pairings are unknown or known to be non-functional. This extreme imbalance creates a blind spot for computer models, which tend to ignore the rare, important matches in favor of the common, safe ones. To solve this, researchers have long tried to teach these models by showing them more examples of the rare matches, artificially creating new data points to fill the gaps. But a critical question remains: does making up data actually help, or does it just confuse the model?

A team of researchers at TELUQ University and the CHUM Research Center in Montreal set out to answer this with a rigorous, controlled experiment. They did not simply propose a new method; they built a testing ground to compare fourteen different strategies for creating synthetic data. These strategies ranged from simple copying and pasting of existing examples to complex generative models that attempt to invent entirely new, plausible drug-protein pairs from scratch. The team applied these methods to four distinct families of proteins—enzymes, G-protein-coupled receptors, ion channels, and nuclear receptors—using a standard benchmark dataset that serves as a common language for the field. They ensured that every method was tested under the exact same conditions, using the same data splits and the same final prediction engine, so that the results would be a fair comparison rather than a collection of isolated anecdotes.

The findings were surprisingly nuanced. The researchers discovered that there is no single "best" way to augment data that works for every situation. In fact, the most sophisticated, high-tech methods did not consistently outperform the simpler, older techniques. For some protein families, the best results came from basic methods that simply adjusted the number of examples or cleaned up the edges where different classes of data overlapped. In other cases, more complex generative models, which use advanced mathematics to learn the underlying structure of the data and create new samples, offered slight improvements in specific metrics, but they did not guarantee a better overall prediction. One of the most significant discoveries was that the quality of the underlying representation of the data mattered more than the method used to generate new samples. If the initial way the computer understood the drug and protein was weak, no amount of synthetic data could fix the final prediction.

The study also highlighted a crucial trade-off in how these models are evaluated. When the researchers forced the training data to be perfectly balanced, with equal numbers of interacting and non-interacting pairs, the models often became better at finding the rare interactions but worse at avoiding false alarms. Conversely, leaving the data in its natural, imbalanced state sometimes yielded more reliable overall results. The team found that for certain protein families, the best performance was achieved not by the most complex generative AI, but by a balanced approach that respected the natural structure of the data. They concluded that adding artificial data is not a magic bullet that automatically improves drug discovery. Instead, the success of these techniques depends heavily on the specific biological context, the quality of the initial data representation, and the specific goal of the prediction. The work serves as a necessary reality check, suggesting that in the quest to predict how drugs work, careful, controlled experimentation often yields better insights than simply generating more data.

Technical Summary: Benchmarking Data Augmentation Strategies for Drug–Target Interaction Prediction under Class Imbalance

1. Problem Statement

Drug–target interaction (DTI) prediction is a critical component of computational drug discovery, yet it faces significant challenges due to extreme class imbalance and sparse interaction matrices. In benchmark and real-world datasets, confirmed positive interactions represent only a small fraction of possible drug–target pairs. Models trained under these conditions often favor the majority class (non-interactions), resulting in limited sensitivity to true interactions and misleading performance estimates.

While data augmentation is frequently employed to address sparsity and imbalance, the literature lacks a unified evaluation framework. Existing studies utilize heterogeneous strategies (e.g., resampling, generative models, matrix factorization) and inconsistent evaluation protocols (e.g., varying data splitting, leakage control, and negative sampling), which limits comparability and obscures the robustness of specific methods.

2. Methodology

This study presents a controlled benchmarking analysis of 14 distinct data augmentation, resampling, filtering, and generative methods. The evaluation was conducted across four protein families from the Yamanishi benchmark: Enzyme, G-protein-coupled receptor (GPCR), Ion Channel, and Nuclear Receptor.

Experimental Pipeline

The study employed a unified experimental pipeline designed to eliminate confounding variables:

Data Partitioning: Fixed stratified five-fold outer cross-validation was used. Within each outer-training fold, a fixed stratified 80/20 split defined inner-training and validation subsets.
Leakage Control: Matrix factorization models were fitted using only inner-training labels. Validation and outer-test labels were masked from the factorization loss and excluded from augmentation procedures.
Representation Learning: Three matrix factorization backbones were utilized to generate latent representations:
1. Multiple Similarities Collaborative Matrix Factorization (MSCMF): Incorporates drug and target similarity matrices via graph-Laplacian regularization.
2. Non-negative Matrix Factorization (NNMF): Constrains latent factors to be non-negative.
3. Logistic Matrix Factorization (LMF): Models binary interaction probability directly.
- Latent dimensions were fixed at $k=50$ . Drug-target pairs were represented by concatenating 50-dimensional drug and target embeddings (100-dimensional total).
Augmentation Strategies: 14 methods were applied exclusively to the inner-training subset to modify class distributions to target ratios of 50/50, 80/20, 85/15, and 90/10 (non-interaction/interaction). Methods included:
- Resampling/Cleaning: Random Over-Sampling (ROS), Random Under-Sampling (RUS), Tomek Links, CNN + Tomek Links.
- Synthetic Oversampling: SMOTE, Basic SMOTE, SMOTE + Tomek Links, ADASYN, k-Means SMOTE, Borderline-SMOTE.
- Feature Perturbation: Noise Injection, Latent Sampling.
- Generative Models: Variational Autoencoders (VAE), Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP), and Latent Diffusion Models.
Downstream Classification: A common Deep Neural Network (DNN) architecture (512–256–128 hidden units with ReLU) was trained on the augmented data.
Evaluation: Each configuration was run 10 times with stochastic DNN training. Metrics included Accuracy, Precision, Recall, F1-score, ROC-AUC, and PR-AUC, computed from pooled out-of-fold predictions.

3. Key Contributions

Unified Benchmark: The study provides a controlled comparison of 14 augmentation strategies across four protein families using a single, leakage-safe experimental pipeline.
Generative vs. Classical: It evaluates modern generative approaches (WGAN-GP, VAE, Latent Diffusion) against classical resampling and boundary-cleaning techniques within the same framework.
Representation Dependency: The study explicitly investigates how the choice of latent representation (MSCMF, NNMF, LMF) interacts with generative augmentation methods.
Ratio Sensitivity: It analyzes the impact of varying target class distributions (from 50/50 to 90/10) on model performance.

4. Results

The results indicate that no single augmentation method or class ratio dominates across all protein families or evaluation metrics.

Performance Variability:
- Enzyme & Ion Channel: These families showed high baseline performance. The MSCMF configuration without augmentation was competitive, often matching or closely approaching the best augmented results. For Enzyme, the highest F1-score (0.7824) was achieved by k-Means SMOTE at a 90/10 ratio.
- GPCR & Nuclear Receptor: These families exhibited greater variation. For GPCR, Tomek Links (with ratio adjustment) at 90/10 yielded the highest F1-score (0.5833). For Nuclear Receptor, Random Under-Sampling at 90/10 achieved the highest F1-score (0.3705).
Generative Models: Generative methods (WGAN-GP, VAE, Latent Diffusion) did not consistently outperform classical methods in terms of F1-score.
- VAE achieved the highest PR-AUC for GPCR.
- Latent Diffusion achieved the highest ROC-AUC for Nuclear Receptor.
- However, gains in ranking metrics (ROC-AUC, PR-AUC) did not necessarily translate to higher F1-scores.
Impact of Latent Representation: The choice of backbone significantly influenced generative performance. MSCMF consistently yielded higher F1-scores than NNMF or LMF when used with WGAN-GP and VAE, suggesting that generative augmentation cannot compensate for weaker underlying representations.
Class Ratio Effects: The 80/20 and 90/10 settings frequently produced higher F1-scores than the balanced 50/50 setting. Standard oversampling tended to inflate recall at the expense of precision, while boundary-aware cleaning (e.g., Tomek Links) improved F1-scores in several settings.

5. Significance and Claims

The paper claims that augmentation performance is inextricably linked to the latent representation, class ratio, and evaluation metric.

No Universal Solution: The study concludes that there is no "one-size-fits-all" augmentation strategy for DTI prediction. The effectiveness of a method depends heavily on the specific protein family and the chosen performance metric.
Baseline Competitiveness: The MSCMF configuration without augmentation remains highly competitive, particularly for Enzyme and Ion Channel families. This suggests that modifying the class distribution does not automatically guarantee improved classification performance and that the quality of the underlying representation is paramount.
Evaluation Protocol: The study emphasizes the importance of controlled evaluation protocols (fixed partitions, leakage control) to ensure that observed differences are attributable to the augmentation strategy rather than experimental artifacts.
Modest Claims on Generative AI: While acknowledging the theoretical potential of generative models, the authors modestly state that increasing generative-model complexity does not, by itself, produce consistent improvements over simpler resampling or perturbation methods for these specific latent DTI representations.

In summary, the study advocates for evaluating augmentation strategies jointly with the representation learning method and class distribution, rather than treating augmentation as an independent preprocessing step.