🧬 biology

Single-Cell Cross-Modal Transfer by Adversarial Fine-Tuning of Foundation Models

This paper proposes a method that leverages adversarial fine-tuning of a single-cell foundation model to perform cross-modal translation between unpaired spatial transcriptomics and single-cell RNA sequencing data, effectively recovering in situ neighborhood information from dissociated cells.

Original authors: Joseph Boyd, Matthew Lyon, Martino Mansoldo, Christian Hurry, Finnian Firth

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Joseph Boyd, Matthew Lyon, Martino Mansoldo, Christian Hurry, Finnian Firth

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to understand the layout of a bustling city. You have two very different sources of information:

The "Dissociated" List (scRNA-seq): Imagine you took every single person in the city, put them in a giant blender, and then sorted them into piles based on their job titles and personality traits. You know exactly who they are and what they do, but you have lost all memory of where they lived or who their neighbors were.
The "Spatial" Map (ST): Imagine you have a high-resolution map of the city showing exactly where people are standing and what the general "vibe" or activity level is in each neighborhood. However, this map doesn't tell you the specific job or personality of every single individual; it just gives you a "bulk" average of the area.

The Problem: Scientists want to combine these two. They want to take the detailed personality list (the blended people) and figure out where they likely lived in the city, effectively reconstructing the map from the list alone. The catch? The list and the map come from completely different cities (different patients or tissue samples), so there is no direct "matching" to guide them.

The Solution: SCXM (The "Adversarial Translator")
The authors of this paper built a new AI tool called SCXM to solve this puzzle. Here is how it works, using simple analogies:

1. The Foundation Model (The "Expert Librarian")

Instead of building a tool from scratch, they started with a pre-trained "Foundation Model" (called Geneformer). Think of this as a super-smart librarian who has already read millions of books about biology. This librarian already knows how genes usually behave together. The researchers didn't teach the librarian everything from scratch; they just gave them a specific new task: "Learn to guess the neighborhood vibe based on a person's profile."

2. The Adversarial Game (The "Teacher and the Student")

To teach the librarian without having the actual city map to compare against, they used a technique called Adversarial Fine-Tuning. Imagine a game between two characters:

The Student (The Encoder): Tries to look at a person's profile and guess their neighborhood vibe.
The Teacher (The Discriminator): Tries to spot the difference between a real neighborhood vibe (from the actual map) and the Student's guess.

The Student keeps trying to fool the Teacher. The Teacher keeps getting sharper at spotting fakes. Eventually, the Student gets so good at guessing that the Teacher can no longer tell the difference between a real neighborhood and a predicted one. This forces the Student to learn the hidden rules of how cell profiles relate to their surroundings.

3. The "Consistency Check" (The "Reality Anchor")

Since the Student is guessing, they might start making wild, impossible predictions. To stop this, the researchers added a "consistency check."

They take the Student's guess of the neighborhood vibe.
They run it through a "Decoder" (a reverse-engineering tool) to see if it turns back into a realistic list of genes.
If the list looks weird, the Student gets a penalty. This ensures the guesses stay grounded in biological reality.

What Did They Find?

The team tested this on three different "cities" (datasets from human lungs, brains, and breast tissue).

Better than the competition: Their method (SCXM) was much better at guessing the neighborhood vibe than previous methods. It successfully predicted that certain genes would be "crowded" in specific areas, just like they are in real tissue.
Finding "Neighborhoods": They discovered that the tool could identify specific "neighborhoods" within the tissue. For example, in the lung data, it could spot areas where B-cells (a type of immune cell) were clustered together versus where they were isolated. This is like identifying a "downtown" vs. a "suburb" just by looking at a list of residents.
What it captures best: The tool was very good at predicting large-scale patterns, like how blood vessels form (angiogenesis) or how the lung develops its structure. It was less successful at predicting very fast, local interactions like cell-to-cell signaling or the cell cycle, suggesting that some biological processes are harder to "guess" from a profile alone.

The Bottom Line

The paper demonstrates that even when you lose the map (spatial data), the list of residents (single-cell data) still holds clues about where they used to live. By using a smart AI that plays a guessing game against a critic, they can reconstruct the "neighborhoods" of the body's tissues, helping scientists understand how structure and function are linked in biology.

Technical Summary: Single-Cell Cross-Modal Transfer by Adversarial Fine-Tuning of Foundation Models

Problem Statement
Spatial transcriptomics (ST) offers gene expression readouts with spatial resolution but is often limited in transcriptome coverage and subcellular scale. Conversely, single-cell RNA sequencing (scRNA-seq) provides whole-transcriptome data but lacks spatial context due to tissue dissociation. While these modalities are complementary, acquiring paired ST and scRNA-seq data from the same samples is often cost-prohibitive or impossible. Consequently, there is a need for computational methods capable of translating unpaired scRNA-seq data into spatial context features. The core challenge addressed is learning this translation from fully unpaired data (where cells in each modality originate from different tissue samples or patients) while leveraging the fact that both modalities manifest as count data.

Methodology: SCXM
The authors propose SCXM (Single-Cell Cross-Modal), a framework that utilizes adversarial fine-tuning of a pre-trained single-cell foundation model to translate scRNA-seq gene expression into spatially bulked expression features.

Foundation Model Backbone: The encoder ( $G_E$ ) is built upon a pre-trained GeneformerV2 transformer. To adapt this foundation model, the authors employ a "fine-tuning" strategy where all but the final four self-attention blocks are frozen. A two-layer Multi-Layer Perceptron (MLP) is appended to the <cls> token to project the representation to the target spatial dimensionality.
Spatial Context Definition: Spatial context ( $Z$ ) is defined via graph convolution on a spatial graph derived from ST data. For a given cell $i$ and gene $g$ , the context is computed as a weighted sum of neighboring cells' expression values ( $z_{ig} = \sum_{j \in N(i)} k_{ij} y_{jg}$ ), using a Gaussian kernel with a 25µm radius.
Learning Framework (WGAN-GP): The training utilizes a Wasserstein GAN with Gradient Penalty (WGAN-GP).
- Generator ( $G_E$ ): Maps scRNA-seq input ( $X$ ) to a latent spatial representation ( $Z$ ).
- Discriminator ( $D$ ): Trained to estimate the Earth Mover's Distance (EMD) between the generated distribution and the ground truth spatial distribution, enforcing the $K$ -Lipschitz continuity via a gradient penalty.
- Decoder ( $G_D$ ): A frozen decoder reconstructs ST gene expression from precomputed spatial features $Z$ derived from ST data.
Objective Function: The model is trained with a semi-supervised approach combining four terms:
1. WGAN Loss ( $L_{WGAN}$ ): Adversarial loss to align distributions.
2. Gradient Penalty ( $L_{GP}$ ): Enforces Lipschitz continuity.
3. ST Reconstruction Loss ( $L_{ST}$ ): Ensures the decoder can reconstruct ST data from its own spatial features.
4. Spatial Consistency Loss ( $L_{SC}$ ): A consistency penalty where the encoder's prediction for scRNA-seq data is decoded by the frozen decoder and compared against the original scRNA-seq input (sliced to the spatial gene panel). This exploits the interchangeability of count data across modalities.

Key Contributions

Unpaired Translation: Demonstrated that a single-cell foundation model can perform cross-modal translation from scRNA-seq to ST context features using fully unpaired data, without requiring paired samples for training.
Adversarial Fine-Tuning: Introduced a specific adversarial fine-tuning protocol for foundation models in the omics domain, utilizing consistency penalties to guide the encoder toward a consistent posterior.
Biological Validation: Showed that dissociated cells retain clues regarding their former spatial neighborhoods, allowing the model to infer spatially bulked expression patterns (e.g., distinguishing clustered vs. isolated gene expression).

Results
The method was evaluated on three datasets: Human Lung, Mouse Brain, and Human Breast Cancer, using Xenium Prime data as a high-plex paired validation set.

Performance: SCXM outperformed existing unpaired translation methods (scConfluence and scACT) across all datasets.
- Lung: Median Pearson Correlation Coefficient (PCC) of 0.4291 (SCXM) vs. 0.1750 (scConfluence) and 0.0065 (scACT).
- Breast: Median PCC of 0.4551 (SCXM) vs. 0.1964 (scConfluence) and 0.0284 (scACT).
- Brain: Median PCC of 0.1700 (SCXM) vs. 0.0573 (scConfluence) and 0.0151 (scACT). The authors note performance was lower on the smaller brain dataset but highlighted that the best-performing gene still achieved a PCC of 0.7085.
Niche Characterization: The model successfully identified regions of high spatially bulked expression (e.g., CXCR4), which correlated with the enrichment of specific cell types (B cells) and allowed for the segmentation of isolated vs. clustered B cells. Differential analysis revealed significant upregulation of niche-related genes (e.g., POU2AF1, MS4A1) in clustered groups.
Pathway Enrichment: Gene Set Enrichment Analysis (GSEA) on the lung dataset showed that SCXM predictions best recapitulated pathways related to lung lineage identity and angiogenesis (large-scale spatial biology), followed by immune response pathways. Pathways related to cell-cell signaling and cell cycle were less successfully recapitulated.

Significance and Claims
The paper claims that SCXM demonstrates preliminary success in translating between scRNA-seq and ST data from completely unpaired sources. The authors posit that the superior performance over baseline methods is attributed to the use of consistency penalties (Equations 3 and 4), which effectively introduce semi-supervision by exploiting the count data relationship between modalities, contrasting with the purely unsupervised cycle consistency or optimal transport plans of other methods.

The work suggests that the primary strength of SCXM lies in capturing prominent spatial contexts (such as tissue lineage and angiogenesis) rather than fine-grained, highly localized signaling processes. The authors modestly conclude that while the method is promising, future work is needed to understand its limits, particularly regarding domain shifts across different tissue types and technologies, and to explore the integration of spatial cell type composition for a more universal representation of spatial biology.