FEAST: Fully Connected Expressive Attention for Spatial Transcriptomics

1. Problem Statement

Spatial Transcriptomics (ST) provides gene expression data with spatial context, crucial for understanding tissue architecture and disease mechanisms (e.g., tumor microenvironments). However, acquiring ST data is prohibitively expensive, limiting its widespread adoption. Consequently, researchers aim to infer ST data from readily available Hematoxylin and Eosin (H&E)-stained Whole Slide Images (WSIs).

Current deep learning approaches for this task face three primary limitations:

• Sparse Graph Constraints: Most methods use Graph Neural Networks (GNNs) built on pre-defined sparse graphs (e.g., connecting only spatial neighbors or morphologically similar spots). This structural limitation prevents the model from capturing potential interactions between non-adjacent spots, ignoring complex, long-range biological relationships.
• Positive-Only Attention: Standard attention mechanisms constrain weights to non-negative values (via Softmax), modeling only excitatory (positive) correlations. They fail to capture inhibitory (negative) biological interactions, such as immune suppression pathways (e.g., PD-L1/PD-1), which are critical in biology.
• Information Loss in Patch Extraction: Standard methods extract fixed-size patches centered on spot coordinates. This leads to the truncation of morphological structures at grid boundaries and ignores the "off-grid" tissue regions between spots, resulting in significant loss of morphological context.

2. Methodology: FEAST Framework

The authors propose FEAST (Fully connected Expressive Attention for Spatial Transcriptomics), an attention-based framework designed to overcome these limitations through three core innovations:

A. Fully Connected Graph via Attention

Instead of relying on pre-defined sparse edges, FEAST models the tissue as a fully connected graph. It leverages the self-attention mechanism to dynamically learn the strength of interactions between all pairs of spots.

• Positional Bias: To address the permutation-invariance of standard attention (which ignores spatial distance), FEAST incorporates a static, non-learned positional bias ($B_h$) based on Euclidean distance (inspired by ALiBi). This allows different attention heads to specialize in local interactions (strong distance penalty) or global long-range relationships.

B. Negative-Aware Attention

To model the dual nature of biological interactions (excitatory and inhibitory), FEAST introduces a novel attention mechanism that allows for negative weights.

• Mechanism: It computes two scores: a positive score ($S_{pos}$) derived from the standard dot product and a negative score ($S_{neg}$) derived from the sign-inverted dot product.
• Formulation: After applying Softmax to both scores, the final attention weight is calculated as:
  $$A_{final} = A_{pos} - \beta \cdot A_{neg}$$
  where $\beta$ is a scaling factor. This enables the model to explicitly assign negative weights to spots that inhibit the target spot's expression, capturing relationships that standard attention would treat merely as "irrelevant."

C. Off-Grid Sampling Strategy

To recover morphological context lost between standard spot grids, FEAST introduces pseudo-spots.

• Sampling: Additional patches are extracted from intermediate, off-grid locations between original spots. These pseudo-spots capture truncated structures and fill information gaps.
• Hierarchical Attention: Since adding pseudo-spots increases the computational cost of the $O(N^2)$ attention mechanism, FEAST employs a two-stage hierarchical approach:
  1. Local k-NN Attention: Both original and pseudo-spots attend only to their $k$-nearest neighbors. This allows original spots to absorb rich local context from pseudo-spots efficiently.
  2. Global Self-Attention: Only the original spots (now enriched with local context) undergo global self-attention. This maintains computational efficiency while preserving the benefits of the fully connected global view.

3. Key Contributions

1. FEAST Framework: A novel attention-based architecture that replaces sparse GNNs with a fully connected graph, enabling the modeling of all-pairwise tissue interactions.
2. Negative-Aware Attention: A mechanism that explicitly models both excitatory and inhibitory biological relationships, enhancing both predictive performance and biological interpretability.
3. Off-Grid Sampling: A strategy to mitigate morphological information loss by introducing pseudo-spots, coupled with a hierarchical attention mechanism to manage computational costs.
4. State-of-the-Art Performance: Comprehensive experiments demonstrating superior performance over existing methods in gene expression prediction.

4. Experimental Results

The authors evaluated FEAST on three public ST datasets: ST-Net (breast cancer), Her2ST (breast cancer), and SCC (skin cancer).

• Quantitative Performance: FEAST achieved State-of-the-Art (SOTA) results across all datasets.
  • It outperformed the previous best model (MERGE) in 7 out of 9 evaluation metrics (MSE, MAE, PCC).
  • Notable improvements were seen in the Her2ST dataset, where FEAST reduced MSE to 0.5761 (vs. 0.6422 for MERGE) and increased PCC to 0.5524 (vs. 0.5037).
• Qualitative Analysis:
  • Heatmaps: FEAST produced gene expression heatmaps (e.g., for genes FASN and GNAS) that were visually closer to ground truth, capturing sharp boundaries and fine-grained spatial patterns that MERGE smoothed over.
  • Attention Maps: Visualizations confirmed that negative-aware attention correctly identified inhibitory relationships (e.g., assigning negative weights to adipose tissue or distinct cell types), whereas standard attention failed to distinguish these negative correlations.
• Ablation Studies:
  • Removing either the negative-aware attention or the off-grid sampling strategy resulted in performance degradation, confirming both components are essential.
  • The optimal number of neighbors ($k$) for local attention was found to be 32.
  • The temperature parameter for negative attention ($\tau_{neg}$) performed best at 0.6, sharpening the focus on strong inhibitory relationships.

5. Significance

FEAST represents a significant advancement in computational pathology and spatial transcriptomics:

• Biological Fidelity: By modeling inhibitory interactions, FEAST aligns more closely with real biological systems where gene regulation involves both activation and suppression.
• Data Efficiency: It maximizes the utility of existing H&E images, reducing the reliance on expensive ST data acquisition.
• Interpretability: The ability to visualize positive and negative attention maps provides researchers with a new tool to hypothesize and validate biological interactions (e.g., tumor-immune cell crosstalk) directly from histology images.
• Generalizability: The framework's success across different cancer types and datasets suggests it is a robust, generalizable solution for spatial gene expression prediction.

The code for FEAST is publicly available, facilitating further research and biological validation.