🤖 machine learning

Towards Identifiability of Interventional Stochastic Differential Equations

This paper establishes the first provable bounds for uniquely recovering parameters of stochastic differential equations from stationary distribution samples under multiple interventions, providing theoretical guarantees for both linear and small-noise nonlinear cases while demonstrating practical advantages in modeling gene regulatory dynamics.

Original authors: Aaron Zweig, Zaikang Lin, Elham Azizi, David Knowles

Published 2026-07-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Aaron Zweig, Zaikang Lin, Elham Azizi, David Knowles

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out the secret recipe for a giant, bubbling pot of soup. But there's a catch: you can never see the pot while it's cooking. You can't stir it, you can't taste the broth as it simmers, and you can't watch the ingredients swirl around. All you get to do is peek at the pot after it has settled down and stopped moving.

This is the problem scientists face with Stochastic Differential Equations (SDEs). These are fancy math models used to describe how things change over time in a noisy, unpredictable world—like how genes interact in a cell or how proteins signal in your body. Usually, to understand the recipe (the parameters), you need to watch the whole cooking process. But in real life, especially in biology, the "cooking" is often destructive. You can't watch a single cell change over time without killing it. So, researchers are stuck with just the final, settled state of the soup.

The big question this paper asks is: Can we figure out the exact recipe just by looking at the final, settled soup, if we get to peek at the pot under different conditions?

The "Intervention" Trick

To solve this, the scientists use a clever trick called an intervention. Imagine you have a pot of soup. You can't see the recipe, but you can add different spices (interventions) to the pot and see how the final settled soup changes.

The Setup: They assume the soup follows a specific mathematical rule (an SDE).
The Action: They "intervene" by shifting the system (adding a specific vector $c_i$ ) and then wait for it to settle into a new, steady state.
The Goal: By comparing the final states of many different interventions, can we reverse-engineer the original recipe?

The Main Discovery: How Many Spices Do You Need?

The authors found that you don't need to try every possible spice combination. You just need a specific number of them, and that number depends on how complex the "hidden" part of the recipe is.

1. The Simple Soup (Linear SDEs)
If the soup's behavior is simple and straight-line (linear), the paper proves mathematically that you need a number of interventions equal to the hidden rank ( $r$ ) of the system.

Think of the system as having a hidden "core" of $r$ ingredients driving the flavor, even if the pot holds $n$ total ingredients.
The Proof: The authors showed that if you have $r$ interventions, you can uniquely figure out the recipe.
The Warning: If you only have $r-2$ interventions, it's impossible. You simply don't have enough clues. It's like trying to guess a 3-ingredient secret sauce with only one taste test; you'll get stuck with too many possibilities.
The Catch: This works best if you already know how fast the ingredients naturally decay (the "decay term $D$ "). If you don't know that, it gets harder, but in their computer simulations, they still managed to find the recipe.

2. The Complex Soup (Nonlinear SDEs)
Real life is rarely simple. Genes and proteins interact in messy, curved ways (nonlinear). Here, the math gets much trickier because the "settled state" doesn't have a neat, closed-form formula anymore.

The Small Noise Assumption: The authors focused on a "small noise" regime. Imagine the soup is so calm that the bubbles are tiny. In this quiet state, the complex soup behaves almost like the simple one, but with a twist.
The Magic of Activation: They discovered that if the "recipe" includes a special, learnable function (an activation function, like a switch that turns genes on or off), you can actually figure out the whole thing!
The Number: For these complex systems, you need roughly $r^2$ interventions (the square of the hidden rank).
The Surprise: Even if you don't know the exact shape of the switch (the activation function $\sigma$ ) beforehand, the math says you can still recover the recipe as the noise gets smaller and smaller. The system "learns" the shape of the switch just by looking at how the soup settles under different interventions.

What They Ruled Out

The paper is very clear about what doesn't work:

No "Zero Noise" Magic: If you try to solve this with absolutely no noise (a perfectly still pot), you actually need $n-r$ interventions (almost as many as the total number of ingredients). The noise is actually helpful because it gives you information about the system's curvature (the second moments) that a perfectly still pot hides.
No "Adversarial" Cheating: If someone maliciously chooses the interventions to hide the recipe (like adding a spice that does absolutely nothing to the flavor), you can't solve it. But the paper assumes the interventions are chosen randomly and naturally, which is how real experiments work.
No "Cycle" Confusion: Unlike some older methods that force the system to be a simple tree (no loops), this method handles cycles. It doesn't assume the system is a straight line from A to B; it handles the messy loops found in real biology.

How Sure Are They?

The Math: For the simple (linear) case, they have proven mathematical bounds. They showed exactly how many interventions are needed and proved that fewer won't work.
The Complex Case: For the nonlinear case, they have proven that it works in the limit of very small noise. They didn't just guess; they used advanced perturbation theory to show that as the noise vanishes, the solution becomes unique.
The Experiments: They didn't just stop at theory. They ran simulations on synthetic data.
- In the linear tests, they showed that with $r$ interventions, the error dropped to near zero, while with $r-2$ , it was a mess.
- In the nonlinear tests, they used "learnable" activation functions (neural networks) and showed they could recover the true gene networks better than fixed methods.
- They even tested this on "semi-synthetic" gene data (simulated to look like real biology) and found that using learnable activations helped predict how genes would behave under new, unseen interventions.

The Takeaway

This paper is like a master chef saying, "You don't need to watch the whole cooking process to know the recipe. If you have a calm kitchen (small noise) and you taste the soup after adding just the right number of different spices (interventions), you can mathematically prove you've found the exact ingredients and how they mix."

It's a big step forward for understanding complex systems like gene regulation, where we can't watch the clock tick but can only see the final result. The authors suggest that by using these "learnable" mathematical tools, we might finally be able to map out the tangled web of life, one intervention at a time.

Technical Summary: Towards Identifiability of Interventional Stochastic Differential Equations

Problem Statement

The paper addresses the fundamental challenge of identifiability in stochastic dynamical systems modeled by Stochastic Differential Equations (SDEs). Specifically, it investigates whether the underlying parameters of an SDE can be uniquely recovered given only samples from the system's stationary distribution under multiple shift interventions.

This setting is motivated by practical limitations in fields like single-cell genomics, where destructive sequencing prevents the observation of continuous trajectories. Instead, researchers only have access to "snapshots" of the system at steady state under various perturbations (e.g., CRISPR knockouts). While causal disentanglement methods exist for Structural Causal Models (SCMs), they often rely on Directed Acyclic Graphs (DAGs) and struggle with cycles or continuous-time dynamics. The authors aim to determine the theoretical conditions under which SDE parameters are identifiable without trajectory data, focusing on the number of interventions required.

Methodology and Theoretical Framework

The authors analyze two primary parameterizations of the drift term $v(x)$ in the SDE $dX_t = v(X_t)dt + \sqrt{\epsilon}dB_t$ :

1. Linear SDEs

The drift is parameterized as $v(x) = (AB - D)x$, where $A \in \mathbb{R}^{n \times r}$ and $B \in \mathbb{R}^{r \times n}$ represent a low-rank interaction structure ( $r \ll n$ ), and $D$ is a decay matrix.

Assumptions: The matrices $A$ and $B$ are drawn from densities ensuring full rank and rotational invariance. The decay matrix $D$ is assumed known (or learnable under specific conditions). The interventions $C$ are shift vectors.
Key Insight: The authors prove that the stationary mean and covariance provide sufficient information to recover the drift matrix $AB - D$.
Intervention Bounds: They establish that $r$ interventions are necessary and sufficient for identifiability (almost surely), whereas $r-2$ interventions are insufficient. This is a tight bound, contrasting with the $n-r$ interventions required in adversarial settings or noiseless ODE limits.

2. Nonlinear SDEs (Small Noise Regime)

The drift is parameterized as a two-layer neural network: $v(x) = A\sigma(Bx) - x$ , where $\sigma$ is an elementwise activation function.

Assumptions: The analysis focuses on the small noise regime ( $\epsilon \to 0$ ). The activation function $\sigma$ is assumed to be odd, $C^2$ , with bounded derivatives, and non-linear (ruling out purely linear functions).
Perturbation Theory: The authors utilize perturbation theory to show that as $\epsilon \to 0$ , the stationary distribution's second-order moments (scaled by $\epsilon$ ) converge to a solution of a Lyapunov equation involving the Jacobian of the drift at the fixed points.
Identifiability Mechanism: Unlike the linear case, the stationary covariance varies across interventions due to the nonlinearity of $\sigma$ . This variability allows for the identification of $A$ and $B$ even without knowing the specific form of $\sigma$ (in the limit).
Intervention Bounds: Theoretical results suggest that $k \gtrsim r^2$ interventions are sufficient to identify the parameters up to signed permutation invariance.

Key Contributions

First Provable Bounds for SDE Identifiability: The paper provides the first theoretical guarantees for the unique recovery of SDE parameters from stationary distributions under interventions, a setting previously open.
Tight Bounds for Linear Systems: It characterizes the exact number of interventions ( $r$ ) required to identify low-rank linear SDEs, proving that fewer interventions ( $r-2$ ) lead to unidentifiability.
Nonlinear Identifiability in Small Noise: It extends these results to nonlinear SDEs, demonstrating that identifiability is possible even if the activation function is unknown, provided the noise is sufficiently small and the number of interventions scales with $r^2$ .
Learnable Activations: The authors propose and validate the use of learnable activation functions (MLPs) in SDE parameterizations. This improves expressiveness for modeling complex gene regulatory networks without sacrificing the structural simplicity required for identifiability.

Experimental Results

The authors validate their theoretical findings through synthetic and semi-synthetic experiments:

Linear Recovery: In linear settings, experiments confirm that $r$ interventions successfully recover the drift parameters with low error, while $r-2$ interventions fail, exhibiting high variance and poor performance.
Nonlinear Recovery: For nonlinear SDEs with small noise ( $\epsilon = 10^{-5}$ ), the model successfully recovers the underlying matrices $A$ and $B$ using $r^2$ interventions, even when the activation function is known.
Generalization with Learnable Activations: In experiments involving synthetic gene regulatory networks (GRNs), models with learnable activation functions (parameterized as MLPs) significantly outperform fixed activation functions (e.g., sigmoid) in predicting unseen interventions, particularly in low-noise regimes.
GRN Recovery: Applying the method to semi-synthetic genomic data (simulated via the BEELINE simulator), the nonlinear SDE approach with learnable activations showed substantial improvement in recovering the ground-truth gene regulatory network compared to linear SDE baselines (e.g., the Bicycle method).

Significance and Claims

The paper claims to offer a foundational step toward causal inference in dynamical systems where temporal data is unavailable. By establishing that SDE parameters are identifiable from stationary distributions under specific intervention counts, the work suggests that SDEs are a viable and theoretically grounded alternative to SCM-based methods for modeling cyclic and continuous-time biological processes.

The authors emphasize that their results are modest regarding the noise regime, noting that the strong theoretical guarantees hold primarily in the small noise limit and for specific drift parameterizations. They acknowledge that generalizing to larger noise regimes or more complex drift functions remains an open challenge. However, the empirical success with learnable activations in genomic applications suggests a practical pathway for improving the inference of complex regulatory networks from high-throughput, steady-state data.