CardamomOT: a mechanistic optimal transport-based framework for gene regulatory network inference, trajectory reconstruction and generative modeling

CardamomOT is a unified mechanistic optimal transport framework that overcomes previous limitations in inferring gene regulatory networks and reconstructing unobserved protein trajectories from single-cell RNA sequencing time series, thereby enabling accurate generative modeling of cellular responses to perturbations.

The Gist

Imagine you are trying to figure out how a complex machine works—like a car engine—but you can only take snapshots of the car at different moments in time. You can't see the engine running, you can't hear the pistons firing, and you can't touch the moving parts. You only have a series of photos showing the car parked in different spots.

This is exactly the challenge biologists face when studying how cells change. They have "snapshots" of cells (from a technology called scRNA-seq) taken at different times, but they can't watch a single cell evolve in real-time because the process destroys the cell. They need to figure out the "engine" (the Gene Regulatory Network) that drives these changes, even though they can't see the "fuel" (proteins) directly.

Enter CardamomOT, a new computer tool that acts like a master detective and a time-traveling simulator rolled into one.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Missing Link"

In the old days, scientists tried to guess the rules of the cell by looking at the photos (mRNA data) and assuming the car moved in a straight, predictable line. But cells are messy. They don't move in straight lines; they jump, stutter, and react to invisible forces.

The biggest missing piece was proteins. Proteins are the actual workers that tell genes what to do. But in these snapshots, proteins are invisible. Previous tools tried to guess the protein levels by assuming they were "frozen" or static, which is like trying to understand a movie by looking at a single, blurry frame.

2. The Solution: CardamomOT's "Magic Bridge"

CardamomOT uses a mathematical concept called Optimal Transport. Think of this as a logistics company trying to move boxes from Warehouse A (Time 1) to Warehouse B (Time 2) in the most efficient way possible.

• The Old Way: The logistics company just moved boxes in a straight line, ignoring the terrain.
• CardamomOT's Way: It builds a mechanistic bridge. It knows the "laws of physics" for the cell (how genes and proteins interact). It asks: "If I move this cell from Time 1 to Time 2, what invisible protein changes must have happened to make that journey possible?"

It doesn't just guess the path; it reconstructs the hidden movie of the proteins moving, even though it never saw them.

3. The "Iterative Dance" (How it learns)

CardamomOT doesn't get the answer right away. It performs a clever dance with itself:

1. Step 1 (The Guess): It guesses the path the proteins took to get from one snapshot to the next.
2. Step 2 (The Rulebook): Based on that path, it writes a new "Rulebook" (the Gene Regulatory Network) that explains why the proteins moved that way.
3. Step 3 (The Refinement): It checks the Rulebook against the photos. If the Rulebook says the car should have turned left, but the photo shows it turned right, it tweaks the Rulebook and tries the path again.

It repeats this dance over and over until the Rulebook and the Path perfectly match the photos. This is why it's called "Iterative"—it keeps polishing the answer until it shines.

4. The Superpower: The "Digital Twin"

Once CardamomOT has figured out the rules and the hidden protein paths, it doesn't just stop. It builds a Digital Twin of the cell system.

Think of this as a video game engine. Because it understands the mechanics of the cell, you can now play "What If?" scenarios:

• "What if we remove this specific gene (a knockout)?"
• "What if we add too much of this protein (overexpression)?"

You can run these experiments on the computer. The tool simulates the cell's reaction and tells you: "If you do this, the cell will turn into a skin cell instead of a nerve cell."

5. Why This Matters

The authors tested this on real biological data, like how stem cells turn into different body parts.

• The Proof: They simulated a scenario where a specific gene was turned on. The computer predicted the cell would become more efficient at reprogramming. When scientists actually did this in a real lab, the experiment matched the computer prediction perfectly.

The Bottom Line

CardamomOT is like a time-traveling mechanic.

• It takes a pile of static photos of cells.
• It figures out the invisible engine (proteins) driving them.
• It writes the instruction manual (the Gene Network) for how the engine works.
• And finally, it lets you run simulations to predict how the cell will react to new drugs or genetic changes, saving time and money in the lab.

It turns a blurry, static puzzle into a clear, moving movie, allowing scientists to not just watch life happen, but to predict how it will change.

Technical Summary

1. Problem Statement

The inference of Gene Regulatory Networks (GRNs) and cellular trajectories from single-cell RNA sequencing (scRNA-seq) time-series data faces two primary challenges:

1. The Destructive Nature of scRNA-seq: Standard protocols destroy cells, preventing longitudinal tracking of individual cells. Researchers only have "snapshots" of cell populations at discrete time points, making it difficult to establish causal relationships or reconstruct continuous trajectories.
2. The Hidden Protein Variable: Gene expression dynamics are driven by protein levels (transcription factors), which are rarely measured in scRNA-seq. mRNA levels are highly stochastic due to transcriptional bursting, and the relationship between mRNA and protein is non-linear and time-delayed. Existing methods often ignore protein dynamics or rely on oversimplified assumptions (e.g., Brownian motion or linear systems) that fail to capture the mechanistic reality of gene regulation.

Previous work by the authors (CARDAMOM) addressed some issues but relied on quasi-stationary approximations for protein dynamics, could not utilize exact time labels effectively, and required complex hyperparameter tuning.

2. Methodology: CardamomOT

CardamomOT introduces a unified mechanistic optimal transport (OT) framework that jointly reconstructs GRNs and unobserved protein trajectories. It operates within an Expectation-Maximization (EM)-like iterative loop.

A. Biological Model

The method is grounded in a hybrid two-state gene expression model:

• Promoter Dynamics: Genes switch stochastically between inactive and active states.
• Transcriptional Bursting: When active, genes produce mRNA in bursts.
• Protein Dynamics: Proteins are produced from mRNA and degrade over time.
• Regulation: The activation rate of a gene is a sigmoidal function of the protein levels of its regulators (the GRN).
• Metastability: The model assumes cells spend most time in stable "basins" (attractors) with rare stochastic transitions between them.

B. The Algorithmic Pipeline

The method takes raw scRNA-seq count matrices and time labels as input and performs the following steps:

1. Preprocessing (NB Mixture Calibration):

   • Fits a Negative Binomial (NB) mixture model to the data to infer mechanistic parameters (burst frequencies, burst sizes).
   • Assigns each cell to a discrete "basin" (regulatory state) based on likelihood.

2. Iterative Refinement (EM-like Loop):

   • Step 1: Protein Trajectory Reconstruction (Mechanistic OT):
     • Given a current GRN estimate, the method solves an Optimal Transport problem to link cells across time points.
     • Unlike standard OT (which uses Euclidean distance), CardamomOT uses a mechanistic cost function. It integrates the deterministic protein ODEs between time points to find the most likely protein trajectory connecting a cell at $t_j$ to a descendant at $t_{j+1}$.
     • This allows the reconstruction of hidden protein trajectories $\{P_c(t)\}$ without assuming quasi-stationarity.
   • Step 2: GRN Calibration:
     • Using the reconstructed protein trajectories, the GRN parameters ($\theta$) are updated via regression to minimize the loss between predicted burst rates and observed basin modes.
     • This step incorporates exact time labels and prior knowledge on kinetic rates.
   • Step 3: Basin Label Refinement:
     • Basin assignments are refined to ensure consistency between the inferred protein states and the observed mRNA data, balancing likelihood with mechanistic constraints.

3. Post-processing (Generative Calibration):

   • Kinetic rates (degradation rates) are recalibrated using NeuralODEs to ensure the simulated dynamics match the observed timescales and stochasticity.
   • The final output is a calibrated generative model capable of simulating new datasets and predicting perturbation effects.

3. Key Contributions

• Joint Inference: Unlike previous methods that treat GRN inference and trajectory reconstruction as separate tasks, CardamomOT solves them simultaneously, using the GRN to inform trajectories and vice versa.
• Mechanistic Optimal Transport: It replaces the standard Euclidean/Brownian cost in OT with a cost derived from a non-linear, GRN-driven mechanistic model. This allows for the reconstruction of hidden protein trajectories rather than just mRNA velocities.
• Generative Capability: The calibrated model functions as a "digital twin" of the biological system, capable of simulating realistic temporal snapshots and predicting the effects of unseen genetic perturbations (knockouts/overexpression) in silico.
• Robustness: The method explicitly incorporates exact time labels and literature-based priors on kinetic rates, reducing the need for hyperparameter tuning and improving robustness to noise.

4. Results

The authors validated CardamomOT on both in silico benchmarks and three experimental datasets (mouse embryonic stem cell differentiation, sympathoadrenal differentiation, and fibroblast reprogramming to iPSCs).

• GRN Inference Accuracy: CardamomOT consistently outperformed state-of-the-art methods (including CARDAMOM, Reference Fitting, GENIE3, and SINCERITIES) in terms of Area Under the Precision-Recall curve (AUPR), particularly for complex network topologies.
• Trajectory Reconstruction:
  • It achieved high cosine similarity (>0.8) with ground-truth velocity fields in protein space, significantly outperforming linear methods (Reference Fitting) which showed poor agreement (<0.4).
  • It successfully reconstructed hidden protein trajectories that were never directly observed.
• Generative Modeling: The calibrated model could regenerate experimental data distributions (marginals, correlations, and cell type proportions) with high fidelity.
• Perturbation Prediction:
  • Dnmt3a Overexpression: Correctly predicted a shift toward neuroectoderm and away from pluripotency, consistent with its known repressive epigenetic role.
  • Obox6 and Zfp42 Overexpression: Successfully predicted the experimentally verified enhancement of iPSC reprogramming efficiency, despite having no prior knowledge of these factors' functions.

5. Significance

CardamomOT represents a significant advancement in single-cell analysis by bridging the gap between data-driven trajectory inference and biophysically grounded mechanistic modeling.

• Causal Insight: By explicitly modeling protein dynamics and regulatory logic, it moves beyond correlation to infer causal regulatory structures.
• Predictive Power: Its ability to function as a generative model allows researchers to perform in silico experiments, predicting cellular responses to genetic perturbations before conducting wet-lab experiments.
• Scalability: The method is computationally efficient, scaling to networks of 100 genes and tens of thousands of cells in under an hour.
• Unified Framework: It is one of the first methods to integrate GRN inference, trajectory reconstruction, and generative simulation into a single, coherent framework based on optimal transport theory.

In summary, CardamomOT provides a robust, interpretable, and predictive tool for decoding the regulatory programs governing cellular differentiation and reprogramming, offering a "digital twin" approach to understanding complex biological systems.