Self-supervised learning for a gene program-centric view of cell states

The paper introduces Tripso, a self-supervised transformer model that moves beyond single latent representations to learn interpretable, gene program-centric embeddings of cell states, thereby enabling the discovery of biologically meaningful patterns and actionable hypotheses across development, disease, and experimental systems.

The Gist

Imagine you are trying to understand a massive, chaotic orchestra playing a symphony.

The Old Way:
In the past, scientists looking at single cells (the tiny building blocks of our bodies) tried to understand them by listening to the entire orchestra at once. They would take a "snapshot" of the cell and compress all that noise into a single, blurry summary. It's like trying to describe a complex song by saying, "It sounds happy," or "It sounds sad." You lose the details. You can't tell if the violins are playing a specific melody or if the drums are beating a new rhythm. This made it hard to understand why a cell was acting a certain way, especially when comparing healthy cells to sick ones, or cells from a baby to cells from an elderly person.

The New Tool: Tripso
The researchers in this paper built a new tool called Tripso. Think of Tripso as a super-smart conductor who doesn't just listen to the whole orchestra, but instead focuses on specific sections: the strings, the brass, the woodwinds, and the percussion.

Instead of giving the cell one blurry summary, Tripso breaks the cell's activity down into Gene Programs (GPs).

• What is a Gene Program? Imagine a "Gene Program" is a specific musical theme or a coordinated group of musicians playing together. For example, there might be a "Stress Response" theme (where the cell is panicking) or a "Growth" theme (where the cell is building new parts).
• How Tripso works: Tripso uses a type of AI (called a Transformer, the same tech behind advanced chatbots) to listen to the cell and say, "Okay, the 'Stress' section is playing very loudly right now, but the 'Growth' section is quiet." It gives a detailed score for every single theme happening inside the cell simultaneously.

What Did They Discover?
Using this new "conductor," the team looked at three different biological "concerts":

1. The Aging Orchestra (Blood Cells):
   They looked at blood cells from babies, adults, and the elderly.

   • The Discovery: They found that baby blood cells have a very loud "JAK-STAT" theme (a specific signal for growth and defense) that fades as we get older. They also saw that as we age, the "B-cell" section of the orchestra changes its tune completely, shifting from a "rapid expansion" song to a "diversity" song.
   • Why it matters: This helps us understand how our immune system changes as we grow up and get old, which is crucial for treating diseases in children versus the elderly.

2. The Broken Rehearsal (Lab-Grown Cells):
   Scientists often grow stem cells in a lab to study them, but these cells often forget how to act like real stem cells. It's like a rehearsal where the musicians are playing the wrong notes.

   • The Discovery: Tripso compared the "lab rehearsal" to the "real concert" (cells inside the human body). It noticed that a specific part of the cell's machinery (called the SEC61 translocon) was playing too loudly in the lab cells, causing them to lose their "stem-ness."
   • The Fix: The team tested this by adding a chemical to "turn down the volume" on that specific machinery. It worked! The lab-grown cells stayed healthy and acted more like real stem cells. This is a huge win for making better medicines and therapies.

3. The Skin's Secret Neighborhoods (Skin Disease):
   They looked at skin from people with eczema (atopic dermatitis).

   • The Discovery: Tripso found a hidden "theme" in the skin cells that wasn't in any existing database. This theme was active in a specific neighborhood of the skin right next to the oil glands (sebaceous glands). It turned out this was a special group of immune cells (memory T-cells) hiding there, waiting to cause a flare-up even after the skin looked healed.
   • Why it matters: This explains why eczema keeps coming back. The "bad actors" are hiding in a specific spot, and now we know exactly what they look like so we can target them better.

The Big Picture
Before Tripso, scientists were trying to understand a complex story by reading a single-word summary. Tripso lets them read the whole chapter, paragraph by paragraph, highlighting the specific themes that matter.

By focusing on these Gene Programs instead of just the whole cell, Tripso helps doctors and scientists:

• Understand how diseases develop over a lifetime.
• Fix lab experiments so they work better for making drugs.
• Find hidden causes of chronic diseases that were previously invisible.

In short, Tripso turns the chaotic noise of biology into a clear, readable score, allowing us to conduct the orchestra of life much more effectively.

Technical Summary

1. Problem Statement

Single-cell omics technologies have enabled the profiling of tens of thousands of genes per cell, yet extracting biological insights from this high-dimensional data remains challenging.

• Limitations of Current Methods: Conventional workflows often compress cell states into a single latent representation (e.g., via PCA or Variational Autoencoders) or reduce gene programs (GPs) to scalar scores.
• The Core Issue: These approaches "entangle" multiple sources of biological variation. A single embedding obscures the structure of underlying, coordinated gene sets (GPs), such as signaling pathways or transcription factor targets. This makes it difficult to:
  • Disentangle concurrent biological processes within the same cell.
  • Compare cell states across diverse conditions (e.g., in vivo vs. in vitro, different ages, or disease states) in a principled manner.
  • Generate interpretable, mechanistic hypotheses for experimental validation.
• Goal: To develop a framework that represents cellular states as a composition of interpretable, multi-dimensional gene program embeddings rather than a single scalar or entangled vector.

2. Methodology: Tripso

The authors introduce Tripso (Transformers for learning Representations of Interpretable gene Programs in Single-cell transcriptOmics), a self-supervised deep learning framework based on Transformer architecture.

Architecture Overview

Tripso models cell states hierarchically through three stages:

1. Gene Encoder (Base Model):

   • Input: Tokenized scRNA-seq count data (rank-based tokenization).
   • Mechanism: A transformer encoder generates contextualized gene embeddings via self-attention among genes within individual cells.
   • Training: Uses a Masked Language Modeling (MLM) objective (predicting masked gene tokens) to learn context-dependent gene-gene relationships.
   • Initialization: Gene embeddings are initialized using pretrained weights from Geneformer.

2. Gene Program (GP) Transformers:

   • Input: Predefined sets of biologically related genes (e.g., from PROGENy, CollecTRI) are routed to specific, independent transformer blocks.
   • Mechanism: Each GP block contains a special CLS token that attends to its constituent gene embeddings. This CLS token acts as a non-linear, context-aware summary of the GP's activity in a specific cell.
   • Training: Trained via bidirectional masking (similar to BERT) to predict masked genes within the specific program.
   • Note: This allows the model to capture up/down-regulation patterns and context-dependent weighting of genes within a program.

3. Global Cell Representation:

   • Input: The CLS embeddings from all GP blocks are aggregated.
   • Mechanism: A global cell transformer block attends to the GP embeddings to produce a unified cell-level embedding.
   • Training: A decoder reconstructs original gene expression counts using a Negative Binomial loss, ensuring the cell representation retains biological fidelity.

Key Capabilities

• GP Discovery: Tripso includes a module to discover novel, data-driven GPs without predefined annotations. It clusters genes based on attention patterns (gene-gene relationships) learned by the transformer, identifying context-specific modules.
• Quantitative Metrics:
  • Gene Importance: Cosine similarity between a gene embedding and the GP CLS token quantifies a gene's contribution to a specific program.
  • GP Importance: Systematic ablation (setting a GP embedding to zero) measures how much a specific program contributes to the overall cell identity.
• Downstream Analysis: The GP embeddings are compatible with standard single-cell tools (e.g., UMAP, Optimal Transport for mapping in vivo to in vitro trajectories, differential abundance testing).

3. Key Contributions

1. Novel Architecture: The first self-supervised transformer framework that explicitly models cellular states as a hierarchy of GP-specific embeddings, moving beyond single-scalar pathway scores.
2. Benchmarking: Demonstrated superior performance over existing methods (Spectra, Expimap, DeepGSEA) in distinguishing cell states based on pathway activity and genetic perturbations.
3. Biological Discovery:
   • Resolved age-specific shifts in hematopoietic differentiation.
   • Identified actionable targets for improving in vitro stem cell maintenance.
   • Discovered a novel, spatially restricted tissue-resident memory T cell (TRM) program in atopic dermatitis.

4. Key Results

A. Benchmarking Performance

• Perturb-seq Dataset: Tripso outperformed matrix factorization (Spectra) and interpretable VAEs (Expimap) in distinguishing cells based on cytokine stimulation (TGFβ vs. TNFα) and genetic knockouts.
• Recovery of Signatures: In GP discovery tasks, Tripso recovered significantly more ground-truth pathway-response genes (48% relative improvement over Spectra) and produced biologically structured gene sets rather than large, uninterpretable latent factors.
• Real-world Tissue: Applied to a human endometrial atlas, Tripso better discriminated cell states driven by WNT and TGFβ signaling compared to baselines.

B. Application 1: Age-Specific Patterns in Human Hematopoiesis

• Dataset: Integrated 498k+ cells from prenatal development to aging (yolk sac, fetal liver, bone marrow) and in vitro cultures.
• Findings:
  • JAK-STAT Activity: Elevated in pediatric hematopoietic stem/progenitor cells (HSPCs), overlapping with type I interferon response genes, suggesting a conserved mechanism for postnatal expansion.
  • IKZF1 Shift: Identified a developmental shift in B-cell differentiation. Prenatal Pro-B cells favored proliferation (high DTX1, BCL2L1), while postnatal cells emphasized BCR diversification (high IGLL1, DNTT). This divergence was only visible in the IKZF1 GP embedding space, not in global embeddings or unrelated programs (e.g., WNT).

C. Application 2: Optimizing In Vitro Hematopoietic Stem Cell (HSC) Maintenance

• Hypothesis: Tripso mapped in vitro HSCs to in vivo references to find genes whose modulation could improve stemness.
• Discovery: The SEC61 translocon machinery (specifically SSR1 and SEC61G) was identified as upregulated in differentiated states within the PI3K GP.
• Validation: Treating cord blood-derived CD34+ cells with a SEC61 inhibitor (SEC61-IN-1) significantly increased the frequency of immunophenotypic HSCs (CD34+ CD45RA− CD90+ EPCR+) in culture, particularly in media containing UM171 and SR-1. This validated the model's ability to guide experimental protocol optimization.

D. Application 3: Disease-Associated GPs in Skin Inflammation

• Dataset: 1.7 million cells from human skin biopsies (healthy, atopic dermatitis [AD], psoriasis) integrated with spatial transcriptomics (Xenium) and proteomics.
• Discovery: Tripso discovered GP23, a novel program enriched in IL13+ tissue-resident memory T cells (TRM) specifically in AD.
  • Composition: Includes inflammatory mediators, metabolic adaptation genes (mitochondrial/fatty acid oxidation), and vesicle trafficking components.
  • Spatial Validation: GP23 activity was spatially co-localized with sebaceous gland-associated immune niches and T-cell aggregates in AD lesions.
  • Persistence: GP23-high niches persisted in skin samples even after clinical lesion clearance (relapse samples), suggesting these transcriptional programs drive disease recurrence.

5. Significance and Impact

• Interpretability: Tripso bridges the gap between the predictive power of foundation models and biological interpretability. By anchoring representations in GPs, it allows researchers to trace changes back to specific biological programs and individual genes.
• Hypothesis Generation: The framework successfully generated testable hypotheses (e.g., SEC61 inhibition) that led to experimental validation, demonstrating its utility in drug discovery and protocol optimization.
• Spatial Context: The integration of Tripso with spatial data revealed that disease-associated programs are not just cell-type specific but niche-restricted, providing a deeper understanding of tissue-level pathology.
• Future Directions: The authors propose extending Tripso to multimodal data (chromatin, protein) and cross-species comparisons, positioning GP-centric modeling as a foundational step toward "virtual cell" models.

In summary, Tripso represents a paradigm shift from "black box" cell embeddings to transparent, program-centric representations, enabling more precise dissection of cellular states across development, disease, and engineered systems.