🧬 biology

scpFormer: A Foundation Model for Unified Representation and Integration of the Single-Cell Proteomics

The paper introduces scpFormer, a transformer-based foundation model pre-trained on over 390 million single-cell proteomic cells that utilizes a continuous, sequence-anchored approach to unify fragmented antibody panel data into a shared semantic space, thereby enabling robust batch integration, unsupervised clustering, in silico panel expansion, and transferable insights for precision oncology.

Original authors: Qifeng Zhou, Lei Yu, Yuzhi Guo, Yuwei Miao, Hehuan Ma, Wenliang Zhong, Lin Xu, Junzhou Huang

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qifeng Zhou, Lei Yu, Yuzhi Guo, Yuwei Miao, Hehuan Ma, Wenliang Zhong, Lin Xu, Junzhou Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your body is a massive, bustling city. For years, scientists have been trying to understand how this city works by reading the "blueprints" (DNA) and the "construction orders" (RNA). But there's a problem: just because a blueprint says "build a bridge" doesn't mean the bridge is actually there, or that it's holding traffic. The real work happens when the construction crew (proteins) actually shows up, builds the bridge, and starts driving cars across it.

The Problem: The "Missing Pieces" Puzzle
Until now, studying these construction crews (proteins) in individual cells has been like trying to solve a giant jigsaw puzzle where every box has a different set of pieces.

The Fragmentation: Some researchers only look at the "traffic lights" (antibodies for specific proteins), while others look at "street signs" (different proteins). They rarely overlap.
The Missing Data: Because proteins are hard to catch, many spots in the puzzle are just empty holes (missing data).
The Old Way: Previous computer programs tried to force these different puzzle boxes into one shape by throwing away any piece that didn't fit the standard template. This meant scientists were throwing away unique, important clues.

The Solution: scpFormer (The "Universal Translator")
The authors of this paper built a new AI called scpFormer. Think of it as a super-smart, universal translator that doesn't just read the blueprints; it understands the language of the construction crew itself.

Here is how it works, using some simple analogies:

1. It Doesn't Use a Dictionary; It Uses a "Vibe Check"

Old AI models treat every protein like a word in a fixed dictionary. If the AI hasn't seen the word "Fluorescent Protein X" before, it can't understand it.
scpFormer is different. It uses a system called ESM (Evolutionary Scale Modeling). Imagine that instead of looking up a word in a dictionary, the AI looks at the protein's "DNA fingerprint" (its amino acid sequence).

The Analogy: If you meet a stranger who speaks a language you don't know, but they have the same accent and body language as your friend, you can guess what they are saying. scpFormer does this with proteins. It understands that Protein A and Protein B are "cousins" because they look and act alike, even if the AI has never seen Protein B in its training data before. This allows it to handle any set of proteins, no matter how weird or unique the experiment is.

2. It Reads the "Volume," Not Just the "Name"

In the old days, computers treated protein levels like a light switch: either it's "on" (1) or "off" (0). But in biology, it's more like a dimmer switch. A protein can be glowing faintly or blindingly bright, and that difference matters.
scpFormer keeps the "dimmer switch" intact. It doesn't round numbers off. It understands the subtle gradients of how much protein is actually there, preserving the fine details of the cell's mood.

3. The "Group Chat" Brain

The model is trained on 390 million cells. That's like reading every book ever written in a library.

The Training: It learns the "group chat" logic of cells. It learns that if Protein A is high, Protein B usually goes down, and Protein C spikes.
The Magic: When it sees a cell with missing data (holes in the puzzle), it doesn't just guess randomly. It looks at the proteins it does see, remembers the "group chat" rules it learned, and fills in the blanks with high accuracy. It's like a detective who can reconstruct a crime scene just by looking at a few scattered clues because they know how the suspects usually behave.

Why Does This Matter? (The Real-World Impact)

Fixing the "Batch Effect" (The Noise Problem):
Imagine taking photos of the same city with three different cameras. One is blurry, one is too bright, and one has a weird filter. Old tools try to force the photos to look identical, often blurring out the actual buildings. scpFormer learns to ignore the camera filters (technical noise) and sees the actual city (biology) clearly, no matter which camera took the picture.
Predicting Cancer Drug Responses:
This is the big one. The researchers tested if this AI could predict how cancer cells would react to drugs.
- The Result: When they fed the AI's "understanding" of the protein world into a drug prediction system, it was more accurate than using standard genetic data.
- The Analogy: It's like predicting how a car will drive. Looking at the engine manual (RNA) tells you what the car should do. Looking at the actual engine running (Proteins via scpFormer) tells you what the car is actually doing. The AI realized that the protein level is the truest sign of how a cancer cell will react to a medicine.

The Bottom Line

scpFormer is a "Foundation Model" for single-cell proteomics. It's a tool that finally allows scientists to mix and match different experiments, fill in the missing data, and understand the true "functional state" of a cell.

Instead of throwing away unique data because it didn't fit the mold, this AI embraces the chaos, learns the underlying language of life, and helps us build better treatments for diseases like cancer. It turns a pile of scattered, broken puzzle pieces into a clear, complete picture of human health.

1. Problem Statement

Single-cell proteomics (SCP) is critical for understanding cellular function, as proteins represent the direct effectors of biological activity, often decoupled from mRNA levels. However, the field faces three fundamental challenges that prevent the development of a unified foundation model:

Feature Fragmentation & Discontinuity: Unlike single-cell RNA sequencing (scRNA-seq), which has a universal set of ~20,000 genes, SCP datasets rely on highly variable, bespoke antibody panels or stochastic mass spectrometry sampling. There is no universal reference vocabulary, leading to severe data silos where non-overlapping markers are discarded.
Structured Missingness: Proteins cannot be amplified like nucleic acids. SCP data suffers from systematic, non-random missingness (dropouts) due to physical detection limits and stochastic sampling, rather than random noise.
Limitations of Standard Transformers: Traditional Transformer architectures rely on fixed-vocabulary, discrete tokenization (index-based). This approach fails in SCP because it cannot process proteins outside a pre-defined training set and struggles with the continuous, variable-length nature of proteomic data.

2. Methodology: scpFormer Architecture

scpFormer is a transformer-based generative foundation model pre-trained on 391 million single cells across 544 proteins from diverse platforms (CITE-seq, CyTOF, mass cytometry, multiplexed imaging). It introduces two key innovations to overcome the limitations of standard architectures:

A. Open-World Protein Tokenization (Continuous Semantic Embedding)

Instead of mapping proteins to discrete indices, scpFormer uses a continuous, sequence-anchored approach:

Identity Embedding: For every protein, the model retrieves its canonical amino acid sequence and processes it through a pre-trained Evolutionary Scale Modeling (ESM) encoder. This projects the protein into a shared semantic manifold based on evolutionary and structural similarity, allowing the model to generalize to unseen proteins (zero-shot capability).
Value-Aware Embedding: Raw, continuous expression magnitudes are projected into a high-dimensional space via a Multi-Layer Perceptron (MLP) rather than being discretized.
Fusion: The final input token for a protein is the sum of its semantic identity (ESM) and its continuous expression value. This allows the model to handle variable-length inputs where each cell is a unique set of detected proteins.

B. Hierarchical Pre-training Framework

The model is trained using a Context-Target masking strategy (simulating panel expansion) and optimized via three synergistic loss functions:

Expression Self-Decoder ( $L_{self}$ ): Imputes missing target proteins based only on the observed context (visible proteins), capturing local micro-level protein-protein interactions.
Global Expression Decoder ( $L_{global}$ ): Forces the global classification token ([CLS]) to compress the entire cellular state. It predicts missing proteins using only the global cell embedding and the protein's identity, ensuring the [CLS] token is "expression-aware."
Joint Decoder ( $L_{joint}$ ): Refines local imputations by injecting the mature global cell state back into the sequence. This acts as a structural regularizer, ensuring local predictions align with the holistic physiological state of the cell.

3. Key Contributions

First SCP Foundation Model: Establishes the first generative foundation model specifically designed for single-cell proteomics, moving beyond ad-hoc statistical workflows.
Panel-Agnostic Integration: By using continuous ESM-based tokenization, scpFormer eliminates the need for a fixed vocabulary, enabling the integration of datasets with disjoint antibody panels and zero-shot generalization to new proteins.
Robust Representation Learning: The model learns batch-invariant, intrinsic protein co-expression landscapes, effectively decoupling technical noise from biological signals.
Cross-Scale Transferability: Demonstrates that single-cell proteomic embeddings can be transferred to bulk-omics tasks, improving drug response prediction.

4. Results

The authors benchmarked scpFormer against state-of-the-art methods (Seurat, CellTypist, Harmony, ComBat, scFoundation) across multiple tasks:

Cell Type Annotation:
- Intra-dataset: Achieved a macro F1-score of 0.978 (vs. 0.868 for Seurat), particularly excelling at identifying rare progenitor cells and phenotypically similar subtypes.
- Cross-batch: In an out-of-distribution test (fine-tuned on Batch 1, tested on Batch 2), scpFormer achieved 0.918 accuracy and 0.789 macro F1, significantly outperforming Seurat (0.642) and CellTypist (0.577), demonstrating robustness to technical variations.
Unsupervised Clustering & Batch Integration:
- On multi-batch CITE-seq and mass cytometry datasets, scpFormer achieved superior biological variance conservation (AvgBIO ~0.72–0.79) and batch removal (AvgBATCH ~0.91–0.96) compared to Harmony and ComBat, which suffered from over-correction or failure to bridge technical gaps.
Protein Imputation:
- In Silico: Achieved a mean Pearson correlation of 0.751 for imputing masked proteins, outperforming Random Forest, k-NN, and Linear Regression.
- Zero-Shot Clinical: Applied to a sparse MIS-C clinical dataset without fine-tuning, it restored biological manifolds with the highest biological conservation scores, rescuing signatures of specialized immune subsets blurred by dropouts.
Cancer Drug Response Prediction:
- When integrated into the DeepCDR framework, scpFormer embeddings outperformed transcriptomic-based embeddings (scFoundation) in predicting IC50 values (Pearson $r = 0.921$ vs. $0.888$), with lower Mean Absolute Error (MAE) for 183 out of 223 tested drugs.

5. Significance

Paradigm Shift: scpFormer shifts single-cell analysis from discrete, index-based modeling to a continuous, evolutionary-aware framework, addressing the fundamental fragmentation of proteomic data.
Precision Oncology: By capturing the functional state of the proteome more accurately than transcriptomics, the model provides a stable signal for predicting drug responses, bridging the gap between single-cell biology and clinical outcomes.
Experimental Design: The ability to perform in silico panel expansion suggests that researchers can computationally enrich sparse or targeted experimental panels, reducing the need for costly wet-lab re-measurements.
Scalability: The model provides a scalable, panel-agnostic substrate for future biomarker discovery, capable of integrating heterogeneous data from different technologies, tissues, and experimental designs.