Phenotypic reversion and target prioritization for cellular inflammation via representation learning with foundation models

This study presents a proof-of-concept framework using single-cell foundation models and a large-scale Perturb-seq dataset to successfully prioritize genetic targets for reversing cellular inflammation, demonstrating that incorporating disease-relevant proinflammatory conditions significantly improves the identification of biologically relevant therapeutic targets compared to basal conditions alone.

The Gist

Imagine your body is a bustling city. Normally, the streets are quiet, traffic flows smoothly, and everyone is happy. This is your healthy state.

But sometimes, troublemakers (like viruses or injury) show up and start shouting, "Fire! Fire!" This is inflammation. The city goes into panic mode: sirens blare, traffic jams form, and buildings start to burn. In the context of this paper, the "troublemakers" are chemicals called IL-1β and TNF-α, and the "city" is a specific type of cell in your blood vessels (endothelial cells) that, when inflamed, can lead to heart disease (atherosclerosis).

The scientists at Pfizer wanted to find a way to stop the shouting and calm the city down. They asked: "If we could turn off specific switches (genes) inside these cells, which ones would make the city go back to being peaceful?"

Here is how they solved this puzzle, explained simply:

1. The Massive Experiment (The "What If" Game)

The researchers took a huge number of these cells (over 860,000!) and played a giant game of "What If."

• They created a "chaos scenario" by adding the shouting chemicals (inflammation).
• Then, they systematically turned off (knocked out) 1,740 different genes in these cells, one by one.
• They asked: "If we turn off Gene A, does the city calm down? What about Gene B?"

They did this in two worlds:

1. The Quiet World: Cells with no shouting chemicals.
2. The Chaotic World: Cells screaming with inflammation.

2. The Old Way vs. The New Way

To figure out which genes worked, they tried three different methods to rank the "best" genes to turn off.

Method A: The Accountant (Differential Expression)

This is the old-school way. They looked at the data like a spreadsheet. They compared the "Chaotic World" to the "Quiet World" and counted how many words (genes) changed.

• The Flaw: It's like trying to fix a noisy room by just counting how many people are talking. It misses the vibe of the room. Also, if you only look at the "Quiet World," you might miss the genes that only matter when things are chaotic.

Method B: The Librarian (ChatGPT)

They asked an AI (ChatGPT) to guess the best genes. They told the AI, "Here is the problem: inflammation. Here is a list of 1,740 genes. Which ones should we turn off?"

• The Result: The Librarian did pretty well! It knew the answer because it had read millions of biology books before. But it's biased—it only knows what humans have already written down. It can't discover something totally new that no one has ever thought of.

Method C: The Magic Mirror (Foundation Models / scFMs)

This is the star of the show. They used a special type of AI called a Single-Cell Foundation Model (scFM).

• The Analogy: Imagine taking a photo of every single cell and turning it into a unique "fingerprint" in a giant, high-dimensional virtual space.
• In this space, "Chaotic Cells" are far away from "Quiet Cells."
• The AI's job was to find the genes that, when turned off, would make the "Chaotic Fingerprint" move closer to the "Quiet Fingerprint."
• The Magic: This AI didn't read any biology books. It didn't know what "inflammation" was. It just looked at the patterns in the data and said, "Hey, when you turn off this gene, the fingerprint looks exactly like a calm cell."

3. The Big Discovery

The scientists found that the Magic Mirror (scFM) was the best detective.

• It successfully identified the genes that experts already knew were important (like the "TNF receptor").
• Crucially, it also found new candidates that the Accountant (Method A) missed and that the Librarian (Method B) hadn't explicitly been told about.
• They also realized that you need the "Chaotic World" to find the right answers. If they only looked at the "Quiet World," the AI got confused and couldn't find the right switches to fix the problem.

4. Why This Matters

Think of drug discovery like finding a key to a locked door.

• Old way: You try every key in a giant box, hoping one fits. It's slow and expensive.
• This new way: You use a smart scanner (the AI) that looks at the lock and instantly tells you which 10 keys out of 1,000 are most likely to work.

The Takeaway:
This paper proves that we can use advanced AI to "reverse engineer" disease. Instead of just guessing which genes cause inflammation, we can use AI to find the specific switches that turn the inflammation off and bring the cell back to a healthy state.

It's like having a GPS that doesn't just tell you where you are, but tells you exactly which turns to take to get back home, even if you've never been there before. And the best part? They are giving away the map (the dataset) to the whole world so other scientists can use it to cure diseases faster.

Technical Summary

1. Problem Statement

In early drug discovery, a central challenge is identifying genetic perturbations (e.g., gene knockdowns) that can reverse a disease-associated cellular phenotype back to a healthy state. Specifically, the authors address the problem of phenotypic reversion in the context of cellular inflammation driving atherosclerosis.

• The Gap: Traditional methods for target nomination often rely on Differential Expression (DE) analysis combined with Gene Set Enrichment Analysis (GSEA). These methods depend heavily on pre-existing pathway annotations, which can be sparse, biased toward well-studied genes, and may not capture state-specific regulatory effects.
• The Opportunity: The rise of single-cell foundation models (scFMs) offers a data-driven approach to learn meaningful latent representations of cells without relying on manual pathway curation. The authors aim to determine if scFMs can effectively prioritize targets that revert an inflammatory transcriptomic profile to a basal (healthy) state, outperforming or complementing traditional DE and Large Language Model (LLM) approaches.

2. Methodology

A. Dataset Generation (Perturb-seq)

The authors generated a large-scale, high-quality Perturb-seq dataset using hTERT-immortalized human arterial endothelial cells (TeloHAEC).

• Scale: 864,115 cells, 38,606 genes, and 1,740 unique genetic perturbations (CRISPRi knockdowns).
• Experimental Conditions:
  1. Basal (Untreated): Cells grown without cytokine stimulation.
  2. Inflammatory (Treated): Cells stimulated with IL-1β and TNF-α (1 ng/mL each) for 24 hours to mimic the inflammatory environment of atherosclerotic plaques.
• Quality Control: High knockdown efficiency (median 80%) and deep sequencing (median 16,373 UMIs per cell).

B. Target Prioritization Approaches

The study compared four distinct methods for ranking genetic targets based on their ability to revert the inflammatory state to the basal state:

1. DE Approach (Classical):

   • DE (Basal): Ranked targets based on the downregulation of inflammatory pathways in the basal condition only.
   • DE (Inflammatory): Ranked targets based on pathway downregulation specific to the inflammatory condition (prioritizing targets significant in inflammation but not basal).
   • Mechanism: Wilcoxon rank-sum test followed by GSEA using MSigDB canonical pathways.

2. Latent Similarity Approach (scFMs):

   • Concept: Embed single-cell transcriptomes into high-dimensional latent spaces using pre-trained foundation models.
   • Models Used:
     • scGPT: A transformer-based model pre-trained on large-scale scRNA-seq data.
     • STATE: A self-supervised model.
     • SCimilarity: A model for cell similarity.
     • Raw Counts: Using raw integer counts as features (non-ML baseline).
     • UMAP: 2D dimensionality reduction (baseline).
   • Ranking Logic: Compute the cosine similarity between the latent representation of a treated perturbation and the untreated safe-targeting control. Targets with the highest similarity (closest to the healthy state) are ranked highest. This was done zero-shot (no fine-tuning on the specific dataset).

3. LLM Approach (ChatGPT):

   • Concept: Prompted an LLM with the biological context and a list of targets to rank them based on its pre-trained knowledge of TNF-α/IL-1β signaling, without access to the experimental transcriptomic data.

C. Evaluation Metrics

• Positive Control Enrichment: Measured the Area Under the Curve (AUC) for the enrichment of a pre-defined set of 6 known anti-inflammatory targets (e.g., TNFRSF1A, TRADD, NFKB1, NFKB2) in the top-ranked lists.
• Pathway Enrichment: Used Enrichr to test if top-ranked targets collectively enriched for relevant biological pathways (both the specific 6 used for DE and broad libraries like KEGG, Reactome, GO).

3. Key Results

A. Impact of Cellular State

• State-Dependent Effects: 59% of gene targets exhibited differential effects depending on the cellular state (significant in one condition but not the other).
• Condition Necessity: The DE (Inflammatory) approach significantly outperformed DE (Basal). The basal-only approach failed to identify inflammation-specific targets, highlighting the necessity of including disease-relevant stimulation in perturbation screens.

B. Performance of Ranking Methods

• scGPT Dominance: The scGPT latent similarity approach achieved the highest performance in positive-control enrichment (AUC = 0.79), outperforming all other methods.
• Comparison:
  • scGPT (0.79) > Raw Counts (0.73) > ChatGPT (0.70) > DE (Inflammatory) (0.69).
  • STATE and SCimilarity performed poorly in this specific task, indicating that not all foundation models generalize equally well to this phenotype reversion task.
  • Surprisingly, the simple Raw Counts embedding outperformed STATE and SCimilarity, suggesting that for this specific task, the complexity of some scFMs did not yield better representations than raw data.
• ChatGPT: Performed well (AUC 0.70) despite having no access to the data, confirming that human literature knowledge aligns with the biological reality of the dataset. However, it was outperformed by data-driven scGPT.

C. Pathway Recovery

• scGPT recovered 100% of the relevant pre-defined pathways in its top 30–100 targets.
• Unlike the DE approach, which relies on pre-selected pathways, scGPT identified relevant pathways (e.g., TNF signaling, IL-1 signaling) purely through unbiased transcriptomic similarity, without any prior knowledge of these pathways.
• The DE approach yielded few significantly enriched pathways when tested against broad libraries, whereas scGPT yielded biologically sensible pathways directly related to the stimulation.

4. Key Contributions

1. Novel Dataset: Release of a massive, high-quality Perturb-seq dataset (864k cells) with matched basal and inflammatory conditions, serving as a benchmark for zero-shot generalization in ML.
2. Proof-of-Concept for scFMs: Demonstrated that foundation models (specifically scGPT) can perform phenotypic reversion target nomination in a zero-shot setting, outperforming classical DE analysis and even human-knowledge-based LLMs.
3. Methodological Insight: Showed that latent similarity is a robust, model-agnostic framework for target triaging that does not rely on the biases of curated pathway annotations.
4. State-Dependency Validation: Provided empirical evidence that disease-relevant stimulation (IL-1β/TNF-α) is critical for identifying therapeutic targets, as many targets behave differently in basal vs. inflammatory states.

5. Significance and Limitations

Significance

• Accelerated Discovery: This approach offers a scalable, data-driven pipeline for early target discovery that reduces reliance on manual pathway curation and human bias.
• Novelty: By identifying targets based solely on transcriptomic similarity to a healthy state, the method can surface novel candidates that might be missed by traditional methods focused on known pathways.
• Community Resource: The release of the dataset and code facilitates further research into "virtual cells" and the development of next-generation foundation models.

Limitations

• Validation: The study relies on enrichment of known targets (positive controls) rather than in vitro or in vivo functional validation of the top-ranked novel targets.
• Druggability: The study prioritizes targets based on phenotype reversion but does not account for druggability, toxicity, or pleiotropic effects, which are critical for clinical progression.
• Model Bias: While scFMs reduce human pathway bias, they are trained on existing data and may inherit biases from their training sets.
• Sample Size: The positive control set (6 genes) is small, limiting the statistical power of the enrichment analysis.

Conclusion

The paper successfully demonstrates that foundation models (scFMs) can be effectively leveraged to prioritize genetic targets for reversing cellular inflammation. The latent similarity approach using scGPT proved superior to traditional differential expression analysis and even outperformed an LLM relying on human literature, validating the potential of data-driven, model-agnostic frameworks in early drug discovery. The authors emphasize that combining disease-relevant stimulation with advanced representation learning is key to uncovering state-dependent therapeutic targets.