Ensemble learning of pathology foundation models for precision oncology

Technical Summary: Ensemble Learning of Pathology Foundation Models for Precision Oncology

Problem Statement
Histopathology is the gold standard for cancer diagnosis and treatment selection. While pathology foundation models (PFMs) pretrained on vast collections of whole-slide images (WSIs) have shown promise, current approaches face significant limitations. Existing PFMs are typically trained on disparate datasets using varying strategies, leading to inconsistent performance and limited generalizability across different tasks and anatomical sites. Benchmark studies indicate that no single foundation model outperforms others across all clinical tasks. Furthermore, attempts to combine models via knowledge distillation often result in an "average" feature representation that reduces diversity and fails to outperform existing models. Additionally, traditional tile-level approaches require large labeled datasets for downstream aggregation, which are often unavailable in specific clinical settings like therapeutic response prediction.

Methodology: ELF (Ensemble Learning of Foundation models)
The authors introduce ELF, an ensemble framework designed to integrate multiple pretrained pathology foundation models into a unified slide-level representation.

• Architecture: ELF incorporates five independently trained tile-level foundation models: GigaPath, CONCH, Virchow2, H-Optimus-0, and UNI. Instead of concatenating features or distilling knowledge, ELF employs a shared slide encoder based on attention-based multiple instance learning (ABMIL) with 8 attention heads.
• Pretraining Strategy: The model was pretrained on 53,699 WSIs spanning 20 anatomical sites from 11 public datasets. The training objective combines:
  1. Contrastive Learning: Using a momentum contrastive framework (MoCoV3) to align slide-level embeddings derived from different foundation models (cross-model consistency) and distinct spatial views of the same WSI (cross-view consistency).
  2. Weakly Supervised Learning: Auxiliary classification heads for binary cancer detection and multi-class organ classification (20 organs) to enhance pan-cancer representation.
• Inference: During inference, a WSI is processed through each of the five base models to generate per-model embeddings. These are fused by the shared slide encoder into a unified slide-level feature representation.
• Efficiency: ELF is designed as a pretrained slide-level foundation model, reducing the need for large task-specific labeled datasets compared to traditional tile-level aggregation methods. It operates at 10× magnification to balance computational cost and performance.

Key Contributions

1. Novel Ensemble Framework: ELF demonstrates that integrating diverse foundation models via an ensemble strategy yields superior performance compared to individual models or existing slide-level aggregators (e.g., COBRAII, TITAN).
2. Data Efficiency: By generating unified slide-level representations, ELF facilitates data-efficient downstream evaluation, particularly in settings with limited data such as therapeutic response prediction.
3. Comprehensive Evaluation: The model was evaluated across 125 diverse clinical tasks, including disease subtyping, molecular biomarker detection, and response prediction for anticancer therapies and immunotherapies.

Results
ELF was evaluated against state-of-the-art slide-level foundation models (Prov-GigaPath, CHIEF, TITAN) and individual tile-level models across multiple domains:

• Disease Classification and Subtyping: ELF achieved the highest balanced accuracy (BA) across all tasks. Notably, on the BRACS dataset (7-class breast cancer subtyping), ELF achieved a BA of 0.457, a 16.3% relative improvement over the second-best model (TITAN). On the EBRAINS brain tumor dataset, ELF outperformed TITAN by approximately 1%.
• Molecular Biomarker Detection: In the TCGA pan-cancer cohort (28 biomarkers, 14 cancer types), ELF achieved a mean AUC of 0.719, significantly surpassing TITAN (0.701), CHIEF (0.676), and Prov-GigaPath (0.678). For specific actionable biomarkers like BRAF in thyroid cancer and KRAS in lung cancer, AUCs exceeded 0.85. In external validation for colorectal cancer (CRC), ELF achieved high AUCs for BRAF (0.806), KRAS, and MSI status (0.886), with positive predictive values (PPV) significantly higher than baseline models at high negative predictive values (NPV). ELF also outperformed baselines in predicting aneuploidy scores, whole-genome doubling, and tumor mutational burden (TMB).
• Therapy Response Prediction:
  • Anticancer Therapy: ELF predicted response to chemotherapy and targeted therapies (e.g., trastuzumab, bevacizumab) across eight cohorts with a mean AUC of 0.759, outperforming baselines by 7–9%. Kaplan-Meier analyses confirmed that ELF risk scores significantly stratified progression-free and recurrence-free survival.
  • Immunotherapy: Across 13 cohorts and eight cancer types, ELF achieved a mean AUC of 0.724 for predicting durable response to immune checkpoint inhibitors (ICIs), outperforming baselines (mean AUC 0.612–0.664). In NSCLC cohorts, combining ELF with PD-L1 and TMB improved predictive performance (AUC 0.773 in Stanford cohort) compared to individual biomarkers. ELF-based risk stratification significantly separated high- and low-risk groups in external validation (MSKCC), maintaining consistency across subgroups.
• Interpretability: Attention heatmaps revealed that ELF focused on histologically relevant features, such as tumor-infiltrating lymphocytes and stromal reactions, aligning with known biological mechanisms of treatment response.

Significance and Claims
The paper claims that ELF supports the hypothesis that ensemble learning can effectively leverage the complementary strengths of diverse pathology foundation models without requiring the training of new tile-level models from scratch. By integrating model-invariant and view-consistent representations, ELF constructs richer slide-level features that enhance predictive performance across a wide range of oncology tasks.

The authors position ELF as a scalable framework for precision oncology that addresses the conundrum of selecting a single foundation model for a specific application. They emphasize that this approach reduces the environmental impact and data requirements associated with training new models, making it particularly valuable for data-limited clinical settings. While acknowledging limitations regarding the retrospective nature of some cohorts and the need for prospective validation, the study concludes that ensemble learning represents a robust strategy for integrating pathology foundation models to support clinical decision-making in disease diagnosis, biomarker detection, and treatment response prediction.