Optimized Gradient Boosting Decision Tree Ensembles for Honeybee Toxicity Prediction Using the ApisTox Dataset

Technical Summary: Optimized Gradient Boosting Decision Tree Ensembles for Honeybee Toxicity Prediction

Problem Statement
Accurate prediction of honeybee toxicity is critical for agrochemical risk assessment and sustainable pesticide development. However, conventional in vivo LD50 bioassays are labor-intensive, costly, and unsuitable for large-scale chemical screening. While Quantitative Structure–Activity Relationship (QSAR) modeling offers a computational alternative, existing studies utilizing the ApisTox benchmark—a standardized dataset of 1,035 compounds—have shown limited performance. Previous benchmarks reported Matthews Correlation Coefficient (MCC) scores ranging from 0.43 to 0.55, indicating a need for more robust machine learning frameworks that can handle the dataset's moderate class imbalance and complex chemical diversity, particularly under challenging generalization scenarios like time-based and structurally diverse (max-min) splits.

Methodology
The study proposes a meta-feature-enriched Gradient Boosting Decision Tree (GBDT) framework designed to optimize binary toxicity prediction. The methodology integrates four core components:

1. Feature Engineering and Representation:

   • Molecular Representations: Three distinct fingerprint/descriptor sets were employed: Extended Connectivity Fingerprints (ECFP), Avalon fingerprints, and a combined set of Physicochemical and Molecular Topological descriptors (PhysChem&MolTop).
   • Meta-Features: To enrich the input space, four binary-encoded meta-features representing agrochemical functional classes (herbicide, fungicide, insecticide, and other) were added to the molecular descriptors.
   • Data Splits: The framework was evaluated using two rigorous partitioning strategies provided by ApisTox:
     • Time-based split: Tests out-of-domain generalization by predicting toxicity for compounds introduced after the training period.
     • Max-min split: Tests in-domain generalization by maximizing structural diversity between training and test sets using ECFP fingerprints and Dice distance.

2. Model Optimization:

   • Three state-of-the-art GBDT algorithms—CatBoost, LightGBM, and XGBoost—were utilized.
   • Hyperparameters were systematically optimized using the Optuna framework with 500 trials per model. The Tree-structured Parzen Estimator (TPE) algorithm was used to maximize the MCC.
   • Class imbalance was addressed via 'balanced' class weights (CatBoost/LightGBM) and dynamic scale_pos_weight (XGBoost).

3. Ensemble Strategies:

   • To leverage complementary strengths, four ensemble schemes were implemented: Hard Voting, Simple Soft Voting, Weighted Soft Voting, and Stacking.
   • Base learners included pairwise and triplet combinations of the optimized GBDT models.
   • Meta-learners for the stacking approach included Logistic Regression, Extra Trees, and other GBDT variants.

4. Evaluation Metrics:

   • Performance was assessed using MCC (primary metric for class imbalance), AUROC, macro-averaged precision, and macro-averaged recall.

Key Results
The proposed framework demonstrated significant improvements over previous benchmarks and baseline models:

• Impact of Meta-Features: The inclusion of agrochemical functional class meta-features consistently improved performance.
  • Time-based split: MCC increased from 0.90 (ECFP only) to 0.91.
  • Max-min split: MCC increased from 0.58 (Avalon only) to 0.66.
• Optimization vs. Benchmarks: Systematic hyperparameter optimization alone yielded substantial gains.
  • Time-based split: Improved MCC from the original benchmark's 0.48 (Random Forest) to 0.90 (+87.5%) with fingerprints alone, and 0.91 (+89.6%) with meta-features.
  • Max-min split: Improved MCC from the benchmark's 0.49 (WL-OA SVM) to 0.58 (+18.4%) with optimized Avalon, and 0.66 (+34.7%) with meta-features.
  • 5-Fold Cross-Validation: The proposed framework achieved an average MCC of 0.87, a 57.6% improvement over a recent GNN-based study (MCC 0.552).
• Ensemble Performance:
  • Time-based split: Stacking ensembles outperformed voting strategies. The best configuration (ECFP features, CatBoost/XGBoost base models, LGBM meta-learner) achieved an MCC of 0.9104.
  • Max-min split: Ensemble learning provided only marginal benefits over the best single optimized model. The best hard voting ensemble (Avalon features, LightGBM/XGBoost) achieved an MCC of 0.6663, nearly identical to the best single model, suggesting that well-optimized individual GBDT models are sufficient for structurally diverse splits.
• Feature Sensitivity: The optimal molecular representation depended on the split strategy. ECFP features paired with XGBoost were superior for time-based splits (novel compounds), while Avalon fingerprints performed more stably for max-min splits (structural diversity).

Significance and Claims
The paper claims that the integration of optimized GBDT models, domain-specific meta-features, and appropriate ensemble strategies creates a robust framework for honeybee toxicity prediction. Key takeaways include:

1. Meta-Feature Utility: Incorporating agrochemical functional class information provides significant predictive value, particularly in chemically diverse evaluation scenarios (max-min split).
2. Optimization Efficacy: Systematic Bayesian hyperparameter tuning produces the largest performance gains, significantly outperforming previous studies that used default or less optimized configurations.
3. Context-Dependent Ensembling: While ensemble learning (specifically stacking) significantly enhances robustness for predicting temporally novel compounds, it offers diminishing returns for structurally diverse splits where single, well-tuned models are already highly effective.
4. Benchmark Advancement: The study establishes new state-of-the-art results on the ApisTox benchmark, demonstrating that advanced tree-based ensembles can outperform deep learning approaches (like GNNs) on this specific dataset size and structure, offering a computationally efficient and interpretable alternative for regulatory risk assessment.