🧬 biology

Comparative Evaluation of Pathology Foundation Models and Transcriptomic Integration for Glioblastoma Survival Prediction

This study demonstrates that while integrating transcriptomic data with whole-slide histopathology images improves glioblastoma survival prediction across various encoders, including state-of-the-art foundation models, these foundation models did not consistently outperform a standard ImageNet-pretrained ResNet50 baseline in this limited-size, frozen-encoder setting.

Original authors: Federica Rignanese, Gianmarco Sabbatini, Lisa Novello, Stefano Bovo, Flavio Ragni, Annarita Barone, Lorenzo Manganaro, Shahryar Noei, Matteo Pozzi, Marco Chierici, Paolo Falco, Giuseppe Jurman

Published 2026-09-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Federica Rignanese, Gianmarco Sabbatini, Lisa Novello, Stefano Bovo, Flavio Ragni, Annarita Barone, Lorenzo Manganaro, Shahryar Noei, Matteo Pozzi, Marco Chierici, Paolo Falco, Giuseppe Jurman

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the fight against glioblastoma, the most aggressive form of brain cancer, doctors face a daunting challenge: the disease is a master of disguise. It grows with such speed and variability that predicting how long a patient might survive is often a guess rather than a calculation. For decades, the primary tool for understanding this cancer has been the microscope. Pathologists examine thin slices of tumor tissue, looking for specific shapes and patterns in the cells that hint at how dangerous the tumor is. This visual inspection, known as histopathology, remains the standard of care. However, the tumor also carries a hidden molecular signature, a complex set of instructions written in RNA that dictates how the cells behave, grow, and resist treatment. While scientists have long known that both the look of the tissue and its molecular code hold clues, it has remained unclear which source of information is more powerful, or if combining them could offer a clearer picture of the future.

A team of researchers set out to answer these questions by building a new kind of digital tool to predict survival for patients with glioblastoma. They turned to a vast public database containing records from hundreds of patients, each with both high-resolution digital images of their tumor tissue and detailed genetic profiles. The researchers trained artificial intelligence systems to learn from these records, testing whether the computer could spot the signs of a shorter or longer life. They compared three approaches: one that looked only at the tissue images, one that analyzed only the genetic data, and a third that tried to learn from both at the same time. They also tested whether newer, specialized AI models designed specifically for medical images were better than older, more general models that had been trained on everyday photographs.

The results offered a surprising hierarchy of information. When the researchers let the artificial intelligence rely solely on the genetic data, it proved to be the most reliable single source for predicting survival. The model trained on this molecular information consistently outperformed the models that looked only at the tissue images, even when the image models were given more data to learn from. This suggests that the invisible molecular instructions inside the tumor cells carry a stronger signal about the patient's fate than the visible shapes of the cells themselves. However, the story did not end there. When the researchers combined the genetic data with the tissue images, the predictions became even more accurate. The best-performing system, which used a specific type of advanced image analysis paired with the genetic data, achieved the highest numerical accuracy of any model tested. However, the researchers noted that while this combined approach showed a positive trend over using genetic data alone, the difference was not statistically significant in their analysis. This indicates that while the genetic code is the dominant clue, the visual patterns of the tumor may hold valuable, complementary information that could refine the prediction, though this potential benefit requires further confirmation.

One of the most significant findings challenged a popular assumption in the field of medical artificial intelligence. In recent years, scientists have developed powerful "foundation models," which are massive AI systems trained on millions of medical images to recognize subtle details that human eyes might miss. It was widely expected that these specialized tools would vastly outperform older, general-purpose models that were originally trained to recognize cats, cars, and landscapes. Yet, in this specific study, the older, general models performed just as well as the new, specialized ones. The researchers found that the newer medical models did not provide a consistent advantage over the standard baseline. This suggests that for the specific task of predicting survival in this type of cancer, the features that matter most might be relatively straightforward, or that the specialized models have not yet been fine-tuned enough to show their full potential in this context.

The study also highlighted the importance of how much data is available. When the researchers limited the training to a smaller group of patients to ensure a fair comparison, the models that combined both image and genetic data still performed better than those using images alone. This consistency held true across all the different types of image analysis tools they tested. The researchers noted that while their results are promising, they are based on a relatively small number of patients, particularly for the combined models. The study serves as a controlled experiment to separate the effects of the data type from the size of the group being studied. The findings suggest that while genetic data is currently the strongest predictor, the future of accurate prognosis likely lies in a hybrid approach that respects the unique information found in both the microscope slide and the genetic sequence. Until larger, independent groups of patients are studied to confirm these results, the medical community must view these findings as a strong suggestion rather than a final rule, but one that points clearly toward the value of looking at the tumor through multiple lenses.

Technical Summary: Comparative Evaluation of Pathology Foundation Models and Transcriptomic Integration for Glioblastoma Survival Prediction

Problem Statement

Glioblastoma Multiforme (GBM) is a highly aggressive brain tumor with a poor prognosis (median survival <15 months) and significant biological heterogeneity. Accurate survival prediction is critical for patient stratification but remains challenging due to the complex interplay of genetic, epigenetic, and histopathological factors. While Whole-Slide Images (WSIs) and RNA-seq profiles capture complementary phenotypic and molecular characteristics, their individual and combined prognostic value specifically within GBM cohorts is not fully characterized.

Furthermore, the emergence of pathology foundation models (pre-trained on massive histological datasets) raises the question of whether they offer superior prognostic representations compared to conventional ImageNet-pretrained Convolutional Neural Networks (CNNs) for GBM survival. Additionally, prior studies often utilized mixed cohorts of lower-grade gliomas (LGG) and GBM, where high performance may stem from distinguishing tumor grades rather than stratifying survival within GBM itself. This study addresses these gaps by evaluating deep survival models on a GBM-specific cohort (TCGA-GBM) to determine:

Whether pathology foundation models outperform a ResNet50 baseline for WSI representation.
Whether integrating RNA-seq with WSI data improves performance over WSI-only modeling under matched training conditions.
Which modality (histopathology, transcriptomics, or multimodal) provides the strongest prognostic signal.

Methodology

Dataset and Cohort

The study utilized a retrospective analysis of the TCGA-GBM cohort, comprising H&E-stained WSIs, bulk RNA-seq gene expression profiles, and clinical survival data.

Cohort Selection: From an initial pool of 573 patients, exclusions were made for missing data or non-primary GBM histology, retaining 439 patients.
Data Splitting: To ensure independent evaluation, a center-based held-out test set was created using patients from Tissue Source Sites (TSS) 27 and 41. The final multimodal test set consisted of 17 patients (27 WSIs).
Training Subsets:
- Full-Budget: WSI-only models trained on 356 patients; RNA-seq-only on 137 patients.
- Size-Matched: All models (WSI-only, RNA-seq-only, and Multimodal) trained on the same subset of 80 patients with both modalities to isolate modality effects from sample size differences.

Feature Extraction and Modeling

WSI Processing: WSIs were tiled into 512×512 patches at 20× magnification.
Encoders: Six WSI encoders were evaluated (frozen weights):
- Baseline: ImageNet-pretrained ResNet50.
- Foundation Models: TITAN, Prov-GigaPath (slide and patch level), UNI2-h, and CONCH v1.5.
- Aggregation: Patch-level embeddings were aggregated using Attention-Based Multiple Instance Learning (ABMIL) to generate slide-level representations.
RNA-seq Processing: Gene expression profiles were normalized (VST, log2) resulting in 60,660 features. A DeepHazard network (neural extension of Cox proportional hazards) was trained to extract a fixed patient-level embedding ( $z_{RNA}$ ). The optimal embedding dimensionality was determined to be 500.
Survival Architecture: A DeepHazard model was used for all tasks.
- Unimodal: Input from either WSI embeddings or RNA embeddings.
- Multimodal: Intermediate fusion via concatenation of WSI and RNA embeddings before the survival prediction head.
Validation: A 4×5 repeated cross-validation scheme was employed on the development cohort, with final performance evaluated on the held-out test set using the Concordance Index (C-index). Statistical significance was assessed using repeated-measures ANOVA with Nadeau-Bengio variance correction and Holm-Bonferroni adjustment.

Key Results

1. RNA-seq as the Strongest Unimodal Signal

RNA-seq models demonstrated the most stable and highest unimodal performance.

Performance: Median C-index of 0.629 (full-budget) and 0.632 (size-matched).
Comparison: The RNA-seq-only model trained on the smaller size-matched subset numerically outperformed the best full-budget WSI-only model (0.632 vs. 0.587), suggesting the molecular signal is dominant and learnable even with fewer samples.

2. Multimodal Integration Consistently Improves Performance

Integrating RNA-seq with WSI data consistently outperformed WSI-only modeling across all six encoders under size-matched conditions.

Performance Gain: Multimodal models achieved median C-indexes ranging from 0.614 to 0.698.
Best Configuration: The CONCH v1.5-based multimodal model achieved the highest overall performance with a median C-index of 0.698.
Statistical Significance: A strong main effect of modality was observed ( $p < 0.01$ ), with multimodal models significantly outperforming WSI-only models for ResNet50, Prov-GigaPath (slide), and CONCH v1.5 after correction.

3. Pathology Foundation Models vs. Baseline

Contrary to the hypothesis that domain-specific pretraining would yield consistent advantages, pathology foundation models did not systematically outperform the ImageNet-pretrained ResNet50 baseline in this specific setting.

WSI-Only: In the full-budget setting, ResNet50 and CONCH v1.5 tied for the highest median C-index (0.587). No significant encoder effect was found.
Multimodal: ResNet50 ranked second (0.683) only behind CONCH v1.5 (0.698), outperforming other foundation models like UNI2-h and TITAN.
Conclusion: In this limited-size, frozen-encoder setting, the choice of WSI encoder did not significantly alter full-budget unimodal performance, and domain-specific pretraining did not guarantee superior prognostic representations.

4. Impact of Training Set Size

Increasing the WSI training set from the size-matched subset (80 patients) to the full cohort (356 patients) improved performance for five of the six encoders (e.g., UNI2-h gained +0.064). However, even with the larger dataset, WSI-only models remained below the performance of RNA-seq-only and the strongest multimodal configurations.

Significance and Claims

The authors position this work as a controlled, GBM-specific exploratory assessment rather than a definitive clinical solution. The study claims the following:

Modality Composition is Critical: In this experimental setting, the composition of data modalities (integrating transcriptomics) had a larger and more consistent influence on prognostic performance than the selection of the WSI encoder or the size of the WSI-only training set.
Complementary Value: The results suggest that while RNA-seq provides the dominant prognostic signal, histopathological features (when encoded effectively, e.g., by CONCH v1.5 or ResNet50) provide complementary information that further refines survival prediction when integrated.
Context-Dependence of Foundation Models: The study challenges the assumption that pathology foundation models are universally superior to general-purpose encoders for survival tasks, noting that their benefits may be context-dependent on cohort size, task specificity, and downstream architecture.
Need for Validation: The authors explicitly state that the positive trend of multimodal integration over RNA-seq alone is "unconfirmed" due to the small sample size and lack of external validation. They emphasize that larger, independent cohorts are required to confirm the incremental value of multimodal integration and to determine if foundation models offer advantages under fine-tuning or in larger datasets.

In summary, the paper provides evidence that for GBM survival prediction, integrating transcriptomic data with histopathology is beneficial, but the specific choice of pathology foundation model may be less critical than the integration strategy itself, at least within the constraints of the current TCGA-GBM cohort and frozen-encoder methodology.