🧬 biology

A vision foundation model for single-cell biology via spatial gene cartography

The paper introduces scVision, a vision foundation model that transforms single-cell transcriptomes into continuous images by spatially arranging genes based on co-expression, enabling state-of-the-art zero-shot cell-type annotation and gene program recovery without fine-tuning by leveraging the biological signal inherent in gene layout rather than just the neural network architecture.

Original authors: Ridvan Yesiloglu, Sakib Mostafa, James Zou, Ash Alizadeh, Jiajun Wu, Lei Xing, Ehsan Adeli, Md Tauhidul Islam

Published 2026-07-17

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Ridvan Yesiloglu, Sakib Mostafa, James Zou, Ash Alizadeh, Jiajun Wu, Lei Xing, Ehsan Adeli, Md Tauhidul Islam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine biology as a massive, bustling library where every single book is a living cell. For a long time, scientists could only read the "average" story of a whole shelf of books mixed together, which meant they missed the unique, tiny details of individual volumes. But thanks to a technology called single-cell sequencing, we can now read every single book in the library, revealing millions of unique stories about how our bodies work, how diseases start, and how cells change. However, there's a problem: these libraries are growing so fast that they have billions of pages, and trying to read them all is like trying to drink from a firehose. To make sense of this, scientists are building "foundation models"—super-smart computer brains that learn the general rules of biology so they can help us understand new cells without needing to be taught from scratch every time. The big question is: how do we teach these computers to read a cell's story most effectively?

The paper you're about to read introduces a new way to teach these computer brains, called scVision. Instead of treating a cell like a sentence made of random words (which is how most current models work), the researchers decided to treat a cell like a picture. They realized that genes (the "words" inside a cell) don't just act alone; they work in teams, like characters in a scene. By arranging these genes into a specific, organized grid—like placing related characters next to each other on a stage—they turned the cell's data into a continuous image. They then taught a computer vision model (the kind that usually recognizes cats and dogs) to understand these "cell pictures." The result is a model that is incredibly fast, needs very few examples to learn new things, and can spot hidden biological patterns that other models miss. It's like upgrading from a text-based translator to a visual artist who can instantly see the whole picture of what a cell is doing.

Turning Cells into Pictures

Imagine you have a giant bag of Lego bricks, and each brick represents a gene. In most computer models, scientists dump these bricks into a pile and ask the computer to guess what the structure looks like. The computer has to figure out which bricks belong together just by looking at the pile. It's a bit like trying to solve a puzzle with the pieces scattered on the floor.

The authors of this paper, led by researchers at Stanford, decided to try something different. They asked: What if we built the puzzle first?

They took the most important genes in a cell and arranged them on a fixed grid, like a 104-by-104 pixel image. They used a clever mathematical trick called "optimal transport" to decide where each gene goes. The rule was simple: genes that usually work together (like a team of firefighters) get placed right next to each other on the grid. Genes that don't talk to each other get placed far apart. When they "paint" the activity of a cell onto this grid, the result is a unique image for every cell. If a cell is busy fighting an infection, a specific region of the image lights up with bright colors. If a cell is resting, a different pattern emerges.

This turns the problem of understanding a cell from a text-reading task into a vision task. Instead of a computer reading a list of words, it's now looking at a picture. This allows them to use powerful "vision transformers"—the same type of AI that helps self-driving cars see the road or helps your phone recognize your face—to understand biology.

The Super-Student: scVision

The researchers built a model they call scVision. They trained it on a massive dataset of 72 million human cells from various parts of the body. They didn't tell the model what specific cell types to look for; instead, they used a technique called "masked image modeling." Imagine showing the model a picture of a cell but covering up 75% of it with a black box. The model's job is to guess what the hidden parts look like based on the visible parts. By doing this billions of times, the model learned the deep, underlying rules of how cells are built and how they behave.

Once trained, the model became "frozen." This means they didn't need to re-teach it for every new experiment. They could just take a new cell, turn it into an image, and ask the model, "What kind of cell is this?" without any extra training.

Why This Matters: The Zero-Shot Magic

The real test of a foundation model is how well it works on things it has never seen before. The researchers tested scVision on six completely different datasets that were never part of its training. These included cells from the kidney, ovary, brain, gut, and even a complex mix of many organs.

Here is where the magic happens:

It's a Label-Efficient Wizard: In the world of AI, usually, you need thousands of labeled examples (like showing a computer 1,000 pictures of a cat and saying "this is a cat") to teach it a new task. scVision is different. In many tests, the model could identify new cell types with just one labeled example. In one experiment, scVision with just one example per cell type performed better than other models that had 50 examples each. It's like a student who can learn a new subject by reading just one page of the textbook, while others need to read the whole chapter.
It Sees the Big Picture: When the researchers looked at how the model organized the cells, they found that scVision created the clearest, most distinct groups. Other models sometimes got confused, mixing up similar cell types. scVision kept them neatly separated, making it much easier to tell them apart.
It's Fast: Because it treats cells as images, scVision is incredibly efficient. It can process 528 cells per second on a standard computer chip. Compare that to other models, which might only manage 1 to 14 cells per second. It's the difference between a sprinter and a snail.

Reading the Mind of a Cell

One of the coolest features of scVision is that it's not just a "black box" that gives answers; it can explain why it made a decision. Because the genes are arranged in a picture, the model's "attention" (what it focuses on) can be mapped back to specific regions of the image.

The researchers found that when the model looked at a specific cell type, it would focus on a small, local patch of the image. When they translated that patch back into genes, they found it corresponded to real biological programs. For example:

In kidney cells, the model focused on genes related to injury and stress.
In ovarian cells, it highlighted genes involved in stress responses.
In brain cells, it found a specific set of genes that were active in both healthy brains and diseased brains, showing that the model had learned a universal "immune cell" identity that works across different organs.

This suggests that scVision isn't just memorizing data; it's actually learning the biological "grammar" of how genes work together.

What It's Not (and What It Rules Out)

The paper is careful to point out what this model is not.

It's not just a better classifier: The researchers tested if the model's success was just because they used a specific type of math to guess the answers. They tried many different methods, and scVision still won. The advantage comes from the image representation itself, not just the guessing game.
It's not magic for everything: While scVision is great at identifying cell types and finding patterns, it does have limits. For instance, if the data is very "noisy" (like if the sequencing was very shallow), the model's performance drops a bit more than some other models. However, it is very robust to missing genes, which is a common problem in real-world data.
It's not a cure-all yet: The model was trained on human data, so we don't know yet how well it works on other animals. Also, while it can find patterns, it doesn't replace the need for scientists to do experiments to prove what those patterns mean.

The Takeaway

This paper suggests that the way we represent biological data matters just as much as the size of the model. By turning a cell's genetic code into a picture and arranging the genes in a way that respects their natural relationships, the researchers created a tool that is faster, more accurate, and easier to understand than previous methods.

It's a shift from thinking of a cell as a list of ingredients to seeing it as a complex, organized landscape. And just like a good map helps you navigate a new city, scVision helps scientists navigate the vast, uncharted territory of human biology, finding the hidden streets and landmarks that define who we are at a cellular level. The authors suggest that this "vision-based" approach could be the key to unlocking the next generation of discoveries in medicine and biology, turning the overwhelming flood of data into a clear, beautiful picture.

Technical Summary: scVision – A Vision Foundation Model for Single-Cell Biology

Problem Statement
Current single-cell foundation models largely adapt architectures from natural language processing (NLP), treating cells as sequences of gene tokens. This approach imposes two primary limitations on biological representation:

Loss of Gene Relationships: It treats genes as largely unordered inputs, failing to capture the coordinated nature of gene programs where genes in the same pathway or regulon act together.
Loss of Quantitative Information: Many token-based models require expression values to be discretized, ranked, or binned to fit token vocabularies, discarding the continuous magnitude of gene expression which is critical for defining cellular identity.

These simplifications may hinder the learning of underlying biological relationships, leading to suboptimal performance in zero-shot tasks such as cell-type annotation and cross-study integration.

Methodology
The authors introduce scVision, a vision foundation model that reframes single-cell representation learning as a computer vision problem. The core innovation is the scImage, a continuous-valued image representation of a cell's transcriptome.

scImage Construction via Optimal Transport:
- Instead of tokenization, the model selects the top 10,816 most informative genes (highly variable genes) to fill a fixed $104 \times 104$ lattice.
- Gene Layout: A Gromov–Wasserstein optimal transport algorithm aligns the relational geometry of genes (based on co-expression correlations) to the spatial geometry of the lattice pixels. This ensures that genes with similar expression patterns across cells are placed as spatial neighbors.
- Rendering: For each cell, expression values are z-scored and projected onto this fixed layout. The resulting image has one pixel per gene, where pixel intensity corresponds to the continuous expression level. Co-regulated genes form local image regions (textures), and cellular identity manifests as global patterns.
Model Architecture and Pretraining:
- Architecture: scVision utilizes a Vision Transformer (ViT-base) with approximately 86 million parameters (12 blocks, 12 attention heads, 768 embedding dimension) and a lightweight decoder.
- Pretraining Strategy: The model is pretrained as a Masked Autoencoder (MAE) on 72 million human cells drawn from the CZ CELLxGENE Census.
- Objective: During training, 75% of the image patches are masked. The model learns to reconstruct the masked patches from the visible tokens, forcing it to capture both local gene-program structures and global cellular states without task-specific supervision.
Zero-Shot Evaluation:
- The pretrained encoder is frozen. Downstream tasks (annotation, integration, perturbation) are performed using the learned embeddings without fine-tuning.
- Readout: Cell embeddings are generated by mean-pooling patch tokens. Cell-type annotation is performed via a 20-nearest-neighbor (20-NN) probe in cosine space against a reference bank.

Key Results
The authors evaluated scVision across six independent, held-out atlases (kidney cortex, ovary, focal cortical dysplasia cortex, Crohn's disease ileum, human retina, and a multi-organ single-nucleus reference) using a strict cross-study holdout paradigm.

Annotation Accuracy: scVision achieved the highest balanced accuracy and present-class macro-F1 scores on all six atlases, outperforming existing foundation models (scFoundation, scGPT, Geneformer) and classical baselines (logistic regression, random forests, etc.).
Label Efficiency: The model demonstrated exceptional label efficiency. In several cases, using only one labeled cell per type with scVision outperformed other foundation models using 50 labeled cells. For instance, in the kidney cortex task, scVision with one label per class surpassed scFoundation, scGPT, and Geneformer with 50 labels per class.
Multi-Study Integration: In an scIB benchmark across eight tissues, scVision preserved biological structure better than all other methods (including scVI, which is explicitly trained for integration). While scVI achieved stronger batch mixing, scVision matched the combined total score of scVI and scGPT, demonstrating that a frozen representation can integrate studies effectively without explicit batch labels.
Interpretability and Biological Recovery:
- Attention Maps: The model's attention mechanisms naturally map to spatial regions of the scImage, corresponding to groups of co-expressed genes. Without pathway supervision, scVision recovered known biological programs (e.g., unfolded protein response in ovarian pericytes, TREM2-DAP12 signaling in Crohn's disease stroma).
- Cross-Tissue Recurrence: Attention-derived programs for specific cell types (e.g., a four-gene myeloid program: IRF8, PIM1, TRAF1, CCND2) were found to recur across multiple organs (brain, kidney, breast, blood) with significantly higher overlap than random chance, suggesting the model learns reusable cell-type identity programs.
Robustness: The spatial representation proved robust to random gene dropout, maintaining performance even when 70% of genes were masked. However, it showed sensitivity to sequencing depth (count downsampling) compared to rank-based token models, as intensity encoding is affected by global signal reduction.

Significance and Claims
The paper claims that scVision demonstrates that the representation of the transcriptome is a critical factor in foundation model performance, potentially more so than model scale alone. By assigning genes biologically meaningful spatial positions and preserving continuous expression values, scVision:

Improves Transferability: It achieves superior zero-shot transfer across studies compared to token-based models, suggesting that the spatial organization of the transcriptome is essential for generalizing to unseen biological contexts.
Enables New Operations: The image formulation enables operations difficult in token models, such as spatial masking of gene neighborhoods to perturb co-regulated modules in silico as a single unit.
Bridges Disciplines: It opens a direct path for applying modern computer vision methods (local feature extraction, spatial attention) to single-cell biology, retaining more of the original transcriptomic signal than discretization-based approaches.

The authors conclude that progress in single-cell foundation models may depend not just on larger datasets or models, but on representations that better encode the relational structure of the transcriptome.