Technical Summary: Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction

Problem Statement

The study addresses the critical need for accurate prediction of Estrogen Receptor (ER) status in breast cancer, a biomarker essential for diagnosis, prognosis, and treatment selection. While high-throughput sequencing has enabled the generation of comprehensive multi-omics datasets (genomic, transcriptomic, and proteomic), effectively modeling these data presents significant computational challenges. Specifically, the TCGA-BRCA cohort presents a "small-sample, high-dimensional" learning problem characterized by:

• High Dimensionality: A feature space of 1,687 molecular variables against only 549 usable patient samples, increasing the risk of overfitting.
• Class Imbalance: A skewed distribution where ER-positive samples (75.4%) significantly outnumber ER-negative samples (24.6%).
• Data Leakage Risks: Improper preprocessing can lead to overly optimistic evaluation metrics.
• Model Suitability: Deep learning architectures, while dominant in other AI domains, typically require larger sample sizes for reliable generalization, making their application to small genomic datasets challenging.

The primary objective is to systematically benchmark classical machine learning models to determine their efficacy in predicting ER status using single-omic and multi-omic data, while ensuring rigorous evaluation practices and biological interpretability.

Methodology

The researchers employed a rigorous experimental framework using data from the TCGA-BRCA cohort obtained via the UCSC Xena Browser. The dataset comprised 549 patients with matched RNA expression (604 features), Copy Number Variation (CNV) (860 features), and Reverse Phase Protein Array (RPPA) protein expression (223 features).

Data Preprocessing and Experimental Design

• Cleaning and Encoding: Ambiguous ER-status labels were removed. Labels were encoded as binary (0 for ER-negative, 1 for ER-positive).
• Stratified Splitting: An 80/20 stratified train-test split was used to preserve class distribution.
• Cross-Validation: A stratified five-fold cross-validation was employed to ensure robust performance estimation.
• Feature Selection: To prevent data leakage, feature selection using ANOVA F-statistics (SelectKBest) was performed independently within each training fold, selecting the top 300 features.
• Class Imbalance Handling: Balanced class weighting was applied during training (e.g., scale_pos_weight in XGBoost) to mitigate bias toward the majority class.
• Scaling: Feature scaling (StandardScaler) was applied only for Support Vector Machines (SVM) and Logistic Regression, while tree-based models utilized original feature values.

Models Evaluated

Six classical machine learning algorithms were benchmarked across four experimental phases:

1. RNA-Only: Transcriptomic data.
2. CNV-Only: Genomic structural alterations.
3. RPPA-Only: Proteomic data.
4. Multi-Omic Integration: Combined RNA, CNV, and RPPA features.

The models included: Random Forest, XGBoost, LightGBM, CatBoost, Support Vector Machines (SVM with RBF kernel), and Logistic Regression. A Multi-Layer Perceptron (MLP) was also evaluated as a deep learning baseline.

Evaluation Metrics

Given the class imbalance, the study prioritized metrics beyond standard accuracy:

• Balanced Accuracy: The primary metric, averaging sensitivity and specificity.
• ROC-AUC: To evaluate threshold-independent discriminative ability.
• Macro F1-Score, Precision, and Recall: To assess performance across both classes equally.

Key Results

1. Single-Omic Performance

• RNA Expression: Consistently provided the strongest predictive signal, reflecting the biological role of ER as a transcription factor. Random Forest achieved the highest cross-validation accuracy (92.35%), while XGBoost achieved the highest ROC-AUC (95.61%). Non-linear models significantly outperformed the linear Logistic Regression baseline.
• CNV Data: Provided meaningful but weaker predictive power compared to RNA. SVM (RBF) emerged as the best performer for CNV alone, achieving 91.80% accuracy and 95.50% ROC-AUC, suggesting nonlinear kernel methods are well-suited for modeling structural genomic alterations.
• Protein Expression (RPPA): Served as a strong secondary predictor. XGBoost achieved the best performance (91.82% accuracy, 90.20% balanced accuracy), capturing downstream pathway activity.

2. Multi-Omic Integration

Integrating RNA, CNV, and RPPA data yielded the best overall performance, demonstrating that complementary molecular information enhances model robustness.

• Top Performer: Random Forest achieved the highest performance in the integrated setting, with a Balanced Accuracy of 90.30% and an ROC-AUC of 97.10% on the held-out test set.
• Comparison: Other ensemble methods (XGBoost, CatBoost, LightGBM) and SVM showed competitive results (Balanced Accuracy ~89.60%), while Logistic Regression remained the weakest.
• Deep Learning Baseline: The MLP baseline achieved an accuracy of 90.00% and ROC-AUC of 90.41%. While competitive, it did not outperform the best classical models, supporting the hypothesis that tree-based ensembles generalize better on small, high-dimensional genomic datasets.

3. Biological Interpretability

Feature selection stability analysis revealed the recurrent selection of biologically relevant genes across cross-validation folds, including ESR1, PGR, FOXA1, and GATA3 (associated with ER-positive/luminal subtypes) and KRT5, KRT17, SOX10 (associated with ER-negative/basal-like subtypes). This consistency validates that the models learned clinically meaningful biological patterns rather than dataset-specific noise.

Significance and Claims

The paper claims that carefully regularized classical machine learning methods remain highly effective for small, high-dimensional genomic datasets, often outperforming deep learning approaches in this specific context. The study demonstrates that:

1. RNA expression is the dominant predictor for ER status, consistent with the transcriptional nature of the receptor.
2. Multi-omic integration provides modest but consistent improvements over single-omic models, confirming the value of complementary genomic and proteomic data.
3. Ensemble tree methods (specifically Random Forest) offer the optimal balance of predictive accuracy, generalization, and interpretability for this task.
4. Rigorous experimental design (stratified splitting, fold-specific feature selection, and class weighting) is critical to prevent data leakage and ensure reliable evaluation in high-dimensional, imbalanced settings.

The authors conclude that their framework provides a comprehensive benchmark for multi-omics-based ER prediction, offering a robust foundation for future precision medicine workflows while acknowledging the need for external validation on independent cohorts and the potential for future integration of additional omics modalities.