Direct pathway enrichment prediction from histopathological whole slide images and comparison with gene expression mediated models

This study demonstrates that directly predicting pathway enrichment profiles from histopathological whole-slide images outperforms the conventional two-step approach of first predicting gene expression and then inferring pathways, offering a more efficient method for biological interpretation in cancer diagnostics.

The Gist

Imagine you have a massive library of books (the human body's cells) that tell the story of a disease. Usually, to understand the plot, you have to read every single page of the book. In the medical world, this is like RNA sequencing: it's incredibly accurate but expensive, slow, and requires a lot of tissue.

Now, imagine you have a photograph of the book's cover and its binding. This is a histopathology slide (a glass slide with a tiny piece of tissue stained pink and purple). Doctors have used these photos for over a century to diagnose cancer, but they can only see the "cover art"—the shape and color of the cells. They can't easily read the "story" (the molecular activity) inside just by looking.

Recently, scientists have taught computers (AI) to look at these photos and guess the story. But there's a debate: What is the best way for the computer to guess the story?

This paper by Arfa Jabin and Shandar Ahmad compares two different strategies for teaching the AI to read the "molecular story" from the "photo."

The Two Strategies

1. The Indirect Route (The "Translator" Method)

Think of this as a two-step translation process.

• Step 1: The AI looks at the photo and tries to guess the exact words of the story (predicting the activity of thousands of individual genes).
• Step 2: The AI takes those guessed words and tries to summarize them into a main theme (predicting if a specific biological "pathway" or process is active).

The Problem: This is like trying to translate a book from English to French, and then from French to German. Every time you translate, you lose a little bit of meaning. The "noise" from the first guess gets amplified in the second step, making the final summary less accurate.

2. The Direct Route (The "Intuitive" Method)

This is the shortcut.

• The AI looks at the photo and skips the middleman. It goes straight from the image to the main theme. It asks, "Does this picture look like a story where the immune system is fighting?" or "Does this look like a story where cell growth is out of control?"

The Advantage: It doesn't get bogged down in guessing every single word. It focuses directly on the big picture patterns that the photo actually shows.

The Experiment: Breast Cancer

The researchers tested these two methods on 987 breast cancer patients. They had both the photos (WSIs) and the actual "story" (RNA data) for all of them, so they could see which method was right.

They focused on 40 different biological pathways (like "Cell Cycle," "Immune Response," or "Hormone Signaling").

The Results: Who Won?

The Direct Method won.

• The Score: The Direct method was much better at correctly identifying which pathways were active. It achieved a high accuracy score (MCC of ~0.73), while the Indirect method struggled more (MCC of ~0.64).
• The Analogy: Imagine trying to guess if a house is on fire.
  • The Indirect method tries to guess the temperature of every single brick, then the humidity of every room, and then decides if there's a fire. It gets confused by the details.
  • The Direct method just looks at the smoke and the flames and says, "Yes, that's a fire." It's faster and more accurate because it focuses on the obvious clues.

Why Did This Happen?

The researchers found that some things are very easy to see in a photo, while others are hidden.

• Easy to see (High Success): Pathways related to the immune system or the structure of the tissue. If a lot of immune cells are invading the tumor, the photo looks crowded and chaotic. The AI can "see" this chaos directly.
• Hard to see (Lower Success): Pathways related to hormones or internal chemical signals. These happen inside the cell's tiny machinery and don't change the "look" of the tissue much. The AI struggled here, which makes sense—you can't always see the internal wiring just by looking at the house's exterior.

The Big Takeaway

This study tells us that we don't always need to try to reconstruct the entire "molecular library" (gene expression) to understand the disease. Sometimes, it's smarter to train the AI to look directly for the specific "themes" (pathways) that matter.

In simple terms: If you want to know if a tumor is aggressive, you don't necessarily need to read every single gene. You can often tell by looking at the "shape" of the tumor in a standard microscope slide, provided you ask the AI the right question directly. This could lead to faster, cheaper, and more accurate cancer diagnoses in the future, using just the routine slides hospitals already take.

Technical Summary

1. Problem Statement

While RNA sequencing (RNA-seq) provides clinically actionable molecular stratification for tumors, it is costly, tissue-intensive, and time-consuming. Conversely, routine Hematoxylin and Eosin (H&E) Whole Slide Images (WSIs) are the clinical gold standard but lack inherent molecular resolution.

Recent deep learning approaches have attempted to predict transcriptomic states from WSIs. However, a critical gap exists in the optimal strategy for inferring biological pathway activity (coordinated gene networks) from these images:

• Indirect Approach: Predict gene expression levels first, then computationally infer pathway enrichment from the predicted expression.
• Direct Approach: Train a model to predict pathway enrichment states directly from image features.

The authors investigate whether the indirect, two-step approach (Morphology $\to$ Gene Expression $\to$ Pathway) is superior to a direct, end-to-end approach (Morphology $\to$ Pathway) for predicting pathway enrichment in Breast Invasive Carcinoma (BRCA).

2. Methodology

Data Cohort

• Dataset: TCGA Breast Invasive Carcinoma (TCGA-BRCA).
• Sample Size: 987 patients with paired, high-quality WSI and RNA-seq data (filtered from an initial pool of 1,231).
• Pathway Definition: Used KEGG 2021 Human database. After filtering for ubiquitous (>90%) and rare (<10%) pathways, a target set of 40 pathways was selected for binary classification (Active/Inactive).

Image Preprocessing Pipeline

1. Tissue Masking: Used OpenSlide and PIL to identify tissue vs. background.
2. Artifact Removal: Applied color deconvolution (HED), contrast enhancement (CLAHE), Otsu thresholding, and morphological operations to remove pen marks, glass, and noise.
3. Tiling: Images were partitioned into non-overlapping 224 $\times$ 224 pixel tiles.
   • Tiles with <20% tissue area were discarded.
   • A maximum of 8,000 tiles per WSI were retained for training.

Feature Extraction

• Backbone: ResNet50 (pre-trained on ImageNet).
• Embedding: Features were extracted from the Global Average Pooling (GAP) layer, resulting in 2,048-dimensional vectors per tile.
• Aggregation: A multi-instance learning approach was used to aggregate tile-level embeddings into a single slide-level representation.

Modeling Strategies

The study trained parallel models to compare two paradigms:

1. Indirect (Gene Expression-Mediated) Model:
   • Stage I: A Multi-Layer Perceptron (MLP) with SiLU activations and Layer Normalization was trained to predict continuous gene expression values (regression) from WSI features.
   • Stage II: The predicted gene expression values were aggregated using Over-Representation Analysis (ORA) to derive pathway activity scores, which were then binarized for classification.
2. Direct (Pathway Prediction) Model:
   • An MLP classifier (dubbed "PathwayNet") was trained directly on slide-level embeddings to predict the 40-dimensional binary pathway status vector.
   • Architecture: Input (2048-dim) $\to$ Hidden Layer (SiLU, Dropout, Normalization) $\to$ Output (Sigmoid activation for 40 pathways).
   • Class Imbalance: Addressed using SMOTE oversampling for minority pathway classes.

Training Protocol

• Optimizer: AdamW with decoupled weight decay.
• Loss: Mean Squared Error (for regression) and Binary Cross-Entropy (for classification).
• Validation: Stratified splitting, early stopping based on validation AUC, and consensus predictions across multiple random seeds.
• Metrics: Pearson/Spearman correlation, MSE, Matthews Correlation Coefficient (MCC), and AUROC.

3. Key Results

• Performance Superiority of Direct Prediction:
  • The Direct Model achieved a mean AUROC of 0.931 and a mean MCC of 0.729.
  • The Indirect (GE-mediated) Model performed significantly worse, with a mean MCC of only 0.64.
  • The direct approach demonstrated robust prediction scores across individual pathways, with some reaching an MCC of 1.0.
• Pathway-Specific Variability:
  • High Predictability: Pathways related to the immune system, inflammation, and the tumor microenvironment (ECM) showed the highest prediction accuracy. These biological processes manifest as distinct, spatially explicit morphological features (e.g., lymphocyte infiltration, stromal remodeling) visible in H&E stains.
  • Low Predictability: Pathways driven by intracellular signaling (e.g., hormone signaling/estrogen) showed lower predictability, likely because their morphological signatures are less distinct or require higher-resolution sub-cellular features not captured by standard WSI analysis.
• Indirect Model Limitations: The indirect model suffered from error propagation; inaccuracies in predicting individual gene expression levels compounded when aggregating into pathway scores, leading to lower overall classification performance.

4. Key Contributions

1. Paradigm Shift in Computational Pathology: The study provides empirical evidence that direct pathway prediction from histology images outperforms the traditional "predict genes then infer pathways" pipeline.
2. Validation of "Virtual Molecular Profiling": It demonstrates that routine H&E slides contain sufficient morphological information to infer complex systems-level biological states (pathway enrichment) without intermediate gene-level reconstruction.
3. Biological Insight: The results highlight that the tumor microenvironment is strongly encoded in tissue morphology, whereas intracellular signaling pathways are less directly observable via standard histology.
4. Methodological Benchmark: The paper establishes a rigorous framework (TCGA-BRCA, 987 cases, 40 pathways) for comparing image-to-omics strategies, serving as a benchmark for future multimodal AI research.

5. Significance and Conclusion

This research suggests that for clinical applications aiming to stratify patients based on pathway activity (e.g., immune checkpoint inhibitor response), direct prediction models are more efficient and accurate than indirect gene-expression-based models.

• Clinical Impact: This approach could enable "virtual" molecular profiling using only standard, low-cost H&E slides, reducing the need for expensive RNA-seq and accelerating treatment decisions.
• Future Directions: The authors suggest that while direct prediction is superior for microenvironmental pathways, predicting intracellular signaling pathways may require multimodal frameworks that integrate specific weak supervision or higher-resolution imaging modalities.

In summary, the paper argues that pathway-level biology is a more direct target for deep learning on histology images than gene-level biology, offering a more robust path toward AI-driven precision oncology.