Benchmarking Static Gene Regulatory Network Reconstruction and Dynamic Transition Probing in Single-Cell Foundation Models.

This paper introduces a unified benchmark demonstrating that single-cell foundation models encode transferable gene regulatory and dynamical priors, with specific components like scGPT's token embeddings and scFoundation's reconstruction head outperforming classical methods in static network reconstruction and dynamic transition probing under zero-shot settings.

The Gist

Imagine your body is a massive, bustling city, and every cell is a tiny apartment building. Inside each building, thousands of switches (genes) control the lights, the heating, and the security systems. A Gene Regulatory Network (GRN) is essentially the master blueprint or the "wiring diagram" that shows which switches control which other switches.

For a long time, scientists have tried to draw this wiring diagram by looking at snapshots of the city. But recently, a new type of super-smart computer program called a Single-Cell Foundation Model has been trained on millions of these snapshots. These models are like "city experts" who have read every blueprint ever made.

This paper asks a simple but tricky question: Do these "city expert" programs actually understand the wiring diagram, and if so, how do we get that knowledge out of them?

Here is what the researchers did, explained through a few analogies:

1. The Great Detective Contest

The researchers set up a "contest" to see who could draw the best wiring diagram. They pitted six of the newest, most advanced AI models (the "Foundation Models") against three older, traditional methods (the "Classical Baselines").

They tested them on six different "neighborhoods" (datasets) and compared their drawings against four different "gold standard" maps (reference networks).

2. Where is the Secret Knowledge Hidden?

The researchers realized that these AI models are like giant, complex libraries. They wanted to know exactly where the knowledge about the wiring was hiding inside the library. They looked at three specific places:

• The Book Covers (Token Embeddings): The basic labels the model learned when it first started reading.
• The Final Chapter (Hidden States): The deep understanding the model has after processing all the information.
• The Highlighter Marks (Attention Scores): The parts the model focused on most when making a decision.

The Winner: In a "zero-shot" test (meaning the AI had to guess without being specifically taught the wiring diagram first), the scGPT model was the champion. When the researchers looked at its "Book Covers" (token embeddings), they found it was better at guessing the wiring than the old methods. It correctly identified the most important "switches" (transcription factors) and drew a map that looked most like the real gold-standard maps.

3. The Time-Travel Test (Dynamic Transition Probing)

Knowing the wiring diagram is great, but does it help you predict what happens when the city changes? For example, does the model understand how a "construction site" cell turns into a "finished building" cell?

Static maps can't answer this. So, the researchers invented a new test called Dynamic Transition Probing.

Think of it like this: Imagine you have a photo of a caterpillar (an early cell). You ask the AI to use its internal logic to "rewrite" that photo step-by-step until it looks like a butterfly (a late cell). The AI isn't told how to do this; it just has to use its internal knowledge of how cells grow.

The Result: The AI models could actually do this! They successfully "rewrote" early cell profiles to look like late ones, proving they understand the flow of time and development. The model called scFoundation was the best at this time-travel simulation.

The Bottom Line

The paper concludes that these new AI models are not just memorizing data; they have actually learned the "rules of the game" for how genes talk to each other and how cells change over time.

However, just because the knowledge is inside the model doesn't mean it's easy to find. Getting the best results depends on:

1. Which model you use (some are better architects than others).
2. How it was trained (what kind of books it read).
3. How you ask for the answer (which part of the library you look in).

In short, these AI models have built a powerful internal map of the cell's wiring and its future, but we need the right tools to read that map correctly.

Technical Summary

Technical Summary: Benchmarking Static Gene Regulatory Network Reconstruction and Dynamic Transition Probing in Single-Cell Foundation Models

Problem Statement
While single-cell foundation models are hypothesized to encode gene regulatory information, the specific mechanisms by which they capture these signals remain unclear. Furthermore, there is a lack of consensus on how these deep learning models compare to conventional inference methods in reconstructing Gene Regulatory Networks (GRNs). A critical gap exists in determining whether the learned gene-gene relationships within these models are merely static correlations or if they possess predictive power over expression dynamics and developmental transitions.

Methodology
The authors introduce a unified benchmarking framework designed to evaluate six single-cell foundation models and three classical baselines across six datasets and four distinct reference network types. The methodology is structured around two primary investigative axes:

1. Static GRN Reconstruction: The study disentangles three specific sources of regulatory signal within each foundation model to determine their efficacy:

   • Pretrained token embeddings.
   • Final-layer hidden states.
   • Attention-derived scores.
     These components are evaluated under a strict zero-shot setting, meaning no fine-tuning on the target datasets is performed.

2. Dynamic Transition Probing: To address the limitation that static GRNs cannot validate the predictive nature of learned relationships, the authors introduce a novel "dynamic transition probing" technique. This method iteratively applies a model's reconstruction head to drive early-cell profiles toward late-cell states. Crucially, this process operates without temporal supervision, testing the model's intrinsic ability to simulate developmental trajectories.

Key Contributions

• Unified Benchmark: The establishment of a comprehensive evaluation framework comparing foundation models against classical baselines for GRN reconstruction.
• Signal Disentanglement: A systematic analysis separating regulatory signals derived from embeddings, hidden states, and attention mechanisms.
• Dynamic Probing Protocol: The development of a zero-shot method to probe whether foundation models encode meaningful developmental dynamics.

Key Results

• Static Performance: Under zero-shot conditions, scGPT utilizing token-embedding similarity demonstrated superior performance compared to classical baselines. Specifically, it outperformed them on STRING and ChIP-seq reference networks, successfully recovered core transcription factors, and best preserved the topology of reference networks.
• Dynamic Performance: In the dynamic transition probing experiments, pretrained models were found to capture meaningful developmental transitions. Among the models tested, scFoundation exhibited the strongest overall performance in driving early-cell profiles toward late-cell states.
• Dependency Factors: The efficacy of recovering regulatory and dynamical priors was shown to be highly dependent on specific variables: the model architecture, the pretraining design, and the specific extraction strategy employed.

Significance
The paper concludes that single-cell foundation models do encode transferable regulatory and dynamical priors. However, the utility of these priors is not uniform; it is contingent upon how the model is architected, how it was pretrained, and how the information is extracted. This work provides a necessary foundation for understanding the internal representations of single-cell foundation models and establishes rigorous standards for their application in regulatory network inference and dynamic modeling.