Technical Summary: scGTN - Deep Siamese Graph Transformer Network for Single-cell RNA Sequencing Clustering

1. Problem Definition

Single-cell RNA sequencing (scRNA-seq) is a pivotal technology for profiling gene expression at the cellular level, enabling the identification of cell types and the understanding of cellular heterogeneity. A central task in this domain is cell clustering, which organizes cells into biologically meaningful groups based on expression patterns.

Despite progress, existing methods face two primary challenges:

1. Sparsity and Noise: scRNA-seq data suffers from extensive dropout effects and measurement noise, resulting in high-dimensional, sparse matrices. This complicates the reliable construction of cell relationships, often leading to graphs that are highly sparse and noisy.
2. Limited Structural Exploitation: Current Graph Neural Network (GNN) approaches primarily focus on local neighborhood aggregation. They often overlook richer structural cues embedded within the cell graph, such as shortest paths and node-wise distances. Consequently, they fail to fully utilize the graph's inherent information, leading to cell similarities that are highly homogeneous and difficult to distinguish.

2. Methodology: scGTN Framework

The authors propose scGTN (Deep Siamese Graph Transformer Network), a framework that explicitly integrates gene expression profiles and intercellular structural dependencies. The architecture consists of three main modules:

2.1 Dual Augmentation Module

To mitigate sparsity and noise, the method generates two complementary views of the data:

• Gene Expression Perturbation: Gaussian noise is added to the gene expression profiles ($X$) to simulate natural variability. This is formulated as $\tilde{X} = \{x_i \odot m\}$, where $m$ is a random mask from a Gaussian distribution.
• Intercellular Graph Structure Augmentation: Two distinct graph views are constructed from the initial $k$-nearest neighbor (kNN) graph:
  • View 1 (Refinement): Spurious edges are removed based on pairwise cosine similarity to preserve biologically relevant connections.
  • View 2 (Diffusion): Graph diffusion is applied to propagate information across the graph, strengthening meaningful connections.

2.2 Siamese Graph Transformer Network Fusion

A Siamese architecture with weight-sharing encoders processes the two augmented views. Unlike standard GNNs, this module explicitly incorporates structural information:

• Gene Expression Embedding: A Siamese graph encoder processes the perturbed gene matrices ($\tilde{X}$) and adjacency matrices ($\tilde{A}$) to produce initial embeddings.
• Explicit Structural Embedding: To capture complex relationships, the model integrates:
  • Position Embeddings: Based on the intimacy of neighbors to the central node.
  • Shortest Path Embeddings: Explicitly modeling the connection structure between nodes.
• Transformer Refinement: These embeddings (Gene + Position + Shortest Path) are fused and processed through a multi-head self-attention mechanism with feed-forward networks (FFN) and residual connections.
• Correlation Loss: A specific loss ($L_{cor}$) is introduced to prevent feature collapse and redundancy between the two views by maximizing the correlation of diagonal elements and minimizing off-diagonal correlations.

2.3 Optimal Transport Clustering

The clustering task is formulated as a distribution alignment problem using a self-supervised strategy:

• Cluster Assignment: Similarity between cell embeddings and cluster centers is modeled using a Student's t-distribution.
• Target Distribution: The assignment probabilities are sharpened to create a target distribution.
• Optimal Transport: An optimal transport strategy aligns the predicted cluster distribution with mixing proportions to ensure robust clustering and prevent degenerate solutions (e.g., all cells mapping to a single label). This is optimized using the Sinkhorn distance method with an entropy constraint.

2.4 Overall Optimization

The final objective function combines four components:
$$L = L_{clu} + \alpha L_{cor} + \beta L_{rec} + \gamma L_{ZINB}$$
Where $L_{clu}$ is the clustering loss, $L_{cor}$ is the correlation loss, $L_{rec}$ is the reconstruction loss (MSE) to preserve graph structure, and $L_{ZINB}$ is the Zero-Inflated Negative Binomial loss to model data sparsity and overdispersion.

3. Key Contributions

The paper outlines three primary contributions:

1. New Perspective: The study addresses scRNA-seq clustering by jointly considering gene expression profiles and cell graph structures, exploiting complementary intercellular information beyond simple feature-level similarities.
2. Novel Methodology: The introduction of scGTN, a framework utilizing a Siamese Graph Transformer Network. This explicitly incorporates richer structural information, specifically shortest paths and node-wise distances, to learn discriminative cell representations.
3. Extensive Experiments: Comprehensive evaluation on multiple benchmark datasets demonstrates the framework's superior performance compared to existing methods.

4. Experimental Results

The authors evaluated scGTN on seven public benchmark datasets (covering human and mouse pancreas, liver, and immune cells) against ten baselines, including traditional methods (k-means, hierarchical), deep learning methods (DEC, scDeepCluster), and graph-based methods (scGNN, scDSC).

• Quantitative Performance: scGTN achieved the best performance across all three metrics (Accuracy, NMI, and ARI) on all seven datasets. On average, it outperformed the second-best method by 3.00% in ACC, 2.42% in NMI, and 2.30% in ARI.
• Visualization: UMAP visualizations on the Human Liver dataset showed that scGTN produced highly compact clusters with clear boundaries, whereas baseline methods exhibited subtype mixing or diffuse distributions.
• Biological Validation:
  • Differential Gene Expression (DEG): Clusters generated by scGTN showed high overlap (>90%) with ground-truth cell types based on top-100 DEGs. The top-5 marker genes identified by scGTN (e.g., PPY for Pancreatic PP cells) matched the gold standard rankings precisely.
  • KEGG Pathway Enrichment: The method successfully identified distinct functional subpopulations, such as "Insulin secretion" pathways for $\beta$ cells and "Ribosome" pathways for acinar cells, confirming biological interpretability.

5. Significance and Claims

The paper claims that scGTN addresses the critical limitations of current methods by moving beyond local neighborhood aggregation to capture global structural dependencies. By explicitly integrating shortest-path information and node-wise distances via a Siamese Graph Transformer, the model learns more discriminative and robust embeddings. The authors assert that this approach effectively handles the inherent sparsity and noise of scRNA-seq data, providing a more accurate tool for cell type identification and biological interpretation compared to existing graph-based and deep learning clustering frameworks.