Advancing Cancer Prognosis with Hierarchical Fusion of Genomic, Proteomic and Pathology Imaging Data from a Systems Biology Perspective

1. Problem Statement

Current multimodal cancer survival prediction methods primarily integrate genomic data (gene expression) and histopathology images (Whole Slide Images, WSIs). However, these approaches suffer from two critical limitations:

1. Omission of the Proteome: Existing models overlook the proteome, which serves as the essential intermediate layer bridging genomic alterations and histopathological phenotypes. Genomic signatures alone cannot account for post-transcriptional and translational regulatory mechanisms that ultimately determine cellular behavior and tissue architecture (e.g., HER2 protein overexpression driving specific staining patterns, regardless of mRNA levels).
2. Flat Fusion Architecture: Current integrative studies fuse heterogeneous data sources in a "flat" manner, treating all modalities as equal peers. This fails to capture the inherent biological hierarchy of the central dogma (Genes $\to$ Proteins $\to$ Phenotypes/Morphology), resulting in suboptimal integration that misses mechanistic pathways linking molecular aberrations to morphological outcomes.

2. Methodology: HFGPI Framework

The authors propose HFGPI (Hierarchical Fusion of Genomic, Proteomic, and Pathology Imaging), a framework designed to model biological progression from a systems biology perspective. The architecture consists of four main stages:

A. Molecular Tokenizer (Feature Extraction)

Instead of treating expression profiles as isolated vectors, the authors introduce a strategy to encode both identity and expression:

• Genomic Embeddings: Uses Gene2Vec to generate identity embeddings capturing functional relationships and co-expression patterns, which are then modulated by patient-specific expression levels via element-wise multiplication.
• Proteomic Embeddings: Leverages a Large Language Model (LLM) (specifically GPT-5) to generate textual descriptions of protein functions and their potential morphological correlates in H&E stained images. These descriptions are encoded using a Vision-Language Model (VLM) text encoder (CONCH) to create protein identity embeddings, which are then combined with expression data. This aligns the protein feature space with the histopathology image space.

B. Gene-Regulated Protein Fusion (GRPF)

This module explicitly models the directional regulatory flow from genes to proteins:

• Graph-Aware Cross-Attention: Constructs gene and protein graphs using k-Nearest Neighbors (k-NN) based on functional similarity. It employs Graph Convolutional Networks (GCNs) to refine embeddings.
• Regulatory Modeling: Uses cross-attention where proteins "query" genes to capture regulatory influences.
• Structure-Preserving Alignment: Introduces a constraint ($L_{struct}$) to ensure the learned attention matrix respects the intrinsic topologies of both gene and protein networks, enforcing biological coherence.

C. Protein-Guided Hypergraph Learning (PGHL)

This module captures the complex, many-to-many relationships between proteins and tissue morphology:

• Hypergraph Construction: Proteins act as hyperedges connecting multiple semantically related image patches. A protein defines a hyperedge by linking its top-$k$ most relevant patches based on semantic similarity.
• Hypergraph Convolution: Applies hypergraph convolution to propagate messages, allowing patches to aggregate contextual information from other patches sharing common protein associations.
• Feature Fusion: Aggregates patch features into protein-driven morphological representations, which are then fused with gene-regulated protein embeddings to create a unified multimodal representation.

D. Survival Prediction

The hierarchically fused features are processed by a Transformer encoder to capture global dependencies, followed by gated attention pooling to generate patient-level embeddings. The model is optimized using a negative log-likelihood (NLL) loss for survival analysis, combined with the structure-preserving alignment loss.

3. Key Contributions

1. Proteome Integration: First to systematically integrate proteomic data as an intermediate layer in multimodal cancer prognosis, addressing the gap between genotype and phenotype.
2. Hierarchical Fusion Strategy: Proposes a biologically grounded architecture that explicitly models the cascade (Genes $\to$ Proteins $\to$ Morphology) rather than using flat fusion, significantly improving the capture of mechanistic pathways.
3. Novel Encoding & Learning Modules:
   • Molecular Tokenizer: Combines identity embeddings (Gene2Vec/LLM-generated descriptions) with expression profiles.
   • GRPF: Models directional gene-protein regulation with structure-preserving constraints.
   • PGHL: Uses hypergraphs to model higher-order protein-morphology associations, moving beyond simple pairwise attention.
4. Interpretability: The framework provides interpretable insights into regulatory cascades (e.g., identifying specific genes regulating a protein and the corresponding histological patches).

4. Experimental Results

The method was evaluated on five TCGA cancer datasets (BLCA, BRCA, GBMLGG, LUAD, UCEC) using 5-fold cross-validation.

• Performance: HFGPI achieved a state-of-the-art average C-index of 0.753, significantly outperforming:
  • Unimodal baselines (Genomics/Proteomics/Images alone).
  • Two-modality methods (Genomics + Images).
  • Three-modality flat fusion methods (e.g., PS3, ICFNet), showing improvements of 1.8% to 2.4% over the best existing three-modality competitors.
• Ablation Studies:
  • Removing the proteomic modality dropped performance by ~4.5%.
  • Replacing the Molecular Tokenizer with standard gene families or pathways reduced performance.
  • Removing GRPF (replacing with standard cross-attention) caused a 2.3% drop, highlighting the importance of modeling regulatory directionality.
  • Removing PGHL (replacing with standard attention) caused a 1.8% drop, validating the need for hypergraphs to capture higher-order relationships.
• Efficiency: HFGPI is computationally efficient, requiring only 2.03M parameters and 0.017s inference time per slide, outperforming ICFNet significantly in speed and parameter count.

5. Significance

This work represents a paradigm shift in computational pathology by moving from data-driven correlation to biology-driven integration. By explicitly modeling the systems biology hierarchy, HFGPI not only achieves superior prognostic accuracy but also offers a more interpretable framework for understanding how molecular alterations translate into observable tissue phenotypes. This approach bridges the gap between "omics" data and clinical pathology, potentially enabling more precise risk stratification and personalized treatment strategies in oncology.