Cross-Slice Knowledge Transfer via Masked Multi-Modal Heterogeneous Graph Contrastive Learning for Spatial Gene Expression Inference

1. Problem Statement

Spatial Transcriptomics (ST) allows for the quantification of gene expression within the spatial context of tissue sections, offering crucial insights into tissue function and disease mechanisms. However, ST technologies suffer from high experimental costs, technical complexity, and data sparsity/noise, limiting their large-scale clinical and research application.

In contrast, H&E-stained pathology images are inexpensive, widely available, and routinely acquired in clinical workflows. The core challenge addressed by this paper is predicting spatial gene expression profiles directly from histology images. Existing methods face two main limitations:

1. Single-Slice Focus: Most models only capture intra-slice spatial relationships, ignoring shared expression patterns and morphological similarities across different tissue slides (inter-slice).
2. Data Quality Issues: ST data often contains missing values, noise, and sparsity, which existing models struggle to handle robustly.

2. Methodology: SpaHGC

The authors propose SpaHGC, a novel framework that integrates Multi-Modal Heterogeneous Graph Learning with Masked Contrastive Learning to model both intra-slice and inter-slice dependencies.

A. Multi-Modal Heterogeneous Graph Construction

The model constructs a heterogeneous graph $\mathbf{G} = (\mathbf{V}, \mathbf{E})$ comprising three distinct subgraphs to capture different levels of relationships:

1. Target-slice Spot-Spot Graph ($E_{TS}$): Connects neighboring spots within the target slide based on Euclidean distance to capture local spatial continuity and tissue context.
2. Cross-slice Spot-Spot Graph ($E_{CS}$): Connects target nodes to reference nodes from other slides. These connections are established by finding the Top-K most similar reference patches based on image embeddings (extracted via the UNI foundation model). This enables knowledge transfer from reference to target slides.
3. Reference-slice Spot-Spot Graph ($E_{RS}$): Connects reference nodes to their most similar peers within the reference cohort (using concatenated image and gene features). This captures global expression patterns and reinforces semantic consistency across the reference set.

Node Features:

• Target Nodes: Visual embeddings ($\mathbf{z}_t$) from the UNI model.
• Reference Nodes: Concatenated vectors of visual embeddings ($\mathbf{z}_r$) and ground-truth gene expression ($\mathbf{y}_r$), allowing the model to learn from known expression patterns.

B. Key Architectural Modules

1. Cross Node Dual Attention (CNDA):
   • A bidirectional attention mechanism that facilitates dynamic information exchange between target and reference nodes.
   • Target nodes attend to reference nodes to acquire morphological and transcriptomic knowledge.
   • Reference nodes attend to target nodes to update their representations, suppressing inter-slide variability while preserving shared patterns.
2. Cross Node Attention Pool (CNAP):
   • A multi-head unidirectional attention module that adaptively integrates auxiliary information from reference nodes into target node representations.
   • It uses a residual connection to ensure the model relies primarily on the target's own representation while refining it with relevant cross-slice context.
3. Complementary Masking Strategy:
   • To enhance robustness against ST noise (dropouts, sparsity), the model generates two augmented graph views.
   • It applies node-type-specific feature masking with complementary masks (if a feature is masked in View 1, it is retained in View 2).
   • This forces the model to learn stable representations even when specific features are missing.

C. Training Objective

The model is trained using a Masked Graph Contrastive Learning framework:

• Contrastive Loss: Encourages the embeddings of the same target node from the two augmented views to be close in the semantic space (asymmetric design with stop-gradient).
• Regression Loss: Minimizes Mean Squared Error (MSE) and maximizes Pearson Correlation Coefficient (PCC) between predicted and ground-truth gene expression.

3. Key Contributions

1. Novel Framework: Introduction of SpaHGC, the first method to jointly model intra-slice spatial structures and inter-slice shared expression patterns using a multi-modal heterogeneous graph.
2. Cross-Slice Transfer Mechanism: Design of CNDA and CNAP modules that enable effective knowledge transfer from reference slides to the target slide, addressing the limitation of single-slice analysis.
3. Robustness via Masking: Implementation of a complementary masking strategy that simulates real-world ST data noise, significantly improving model generalization and robustness.
4. State-of-the-Art Performance: Comprehensive benchmarking across seven diverse datasets (different platforms, tissues, and cancer subtypes) showing significant improvements over existing methods.

4. Experimental Results

The authors evaluated SpaHGC on seven public ST datasets (including cSCC, Breast Cancer, Lymph Node, and Pancreas) against nine state-of-the-art baselines (e.g., STNet, HisToGene, BLEEP, M2OST).

• Quantitative Performance: SpaHGC achieved the highest Pearson Correlation Coefficient (PCC) and lowest RMSE across all datasets.
  • PCC improvements ranged from 7.3% to 27.1% compared to the best baseline.
  • Statistical significance was confirmed via paired Wilcoxon signed-rank tests ($p < 0.001$).
• Qualitative Analysis:
  • Pattern Reconstruction: Visualizations of marker genes (e.g., SPINK5, CEACAM5) showed SpaHGC better preserved original spatial expression patterns compared to baselines.
  • Tumor Localization: Clustering based on predicted expression yielded the highest Adjusted Rand Index (ARI), accurately identifying tumor vs. non-tumor regions.
  • Biological Relevance: Downstream pathway enrichment analysis (KEGG/GO) on predicted data revealed significant enrichment in cancer-related pathways (e.g., B Cell Receptor Signaling), often with higher statistical significance than results from original noisy data.
• Ablation Studies:
  • Removing cross-slice edges ($E_{CS}$) caused the largest performance drop, confirming the critical role of inter-slice knowledge transfer.
  • The UNI pathology foundation model outperformed other encoders (ResNet, ViT, Virchow).
  • The complementary masking strategy outperformed random masking and no masking.

5. Significance

This work represents a significant advancement in computational pathology by:

• Bridging the Cost Gap: Providing a cost-effective alternative to expensive ST experiments by accurately inferring gene expression from standard H&E slides.
• Leveraging Collective Intelligence: Moving beyond single-slide analysis to utilize the collective information of multiple tissue samples, mimicking how pathologists compare cases.
• Handling Data Scarcity: Demonstrating that contrastive learning with masking can effectively mitigate the sparsity and noise inherent in spatial transcriptomics data.
• Clinical Potential: The model's ability to accurately reconstruct gene expression patterns and identify tumor regions suggests strong potential for clinical applications in diagnosis, prognosis, and treatment planning where ST data is unavailable.

The code and data are publicly available, facilitating further research and reproducibility.