🤖 machine learning

Partially Observed Structural Causal Models

This contribution introduces Partially Observed Structural Causal Models (POSCMs), an extended framework that formalizes causal systems with latent contexts influencing both interaction structures and mechanisms, and validates its identifiability theory and the edge-functional decomposition approach through a biophysically detailed virtual human retina simulator.

Original authors: Turan Orujlu, Jordan Matelsky, Martin V. Butz, Charley M. Wu, Konrad P. Kording

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Turan Orujlu, Jordan Matelsky, Martin V. Butz, Charley M. Wu, Konrad P. Kording

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how a complex machine works, like a huge, living city. In the old way of thinking (called Structural Causal Models or SCM), scientists assumed the city's layout was fixed. They knew exactly which streets connected to which buildings and only had to figure out what happened when people walked along those streets.

In the real world, however, it is more chaotic. Sometimes the "streets" themselves appear or disappear, depending on hidden factors we cannot see. Perhaps a new road is only built if the neighborhood is wealthy, or a bridge collapses when the weather is stormy. The "context" (the hidden factor) decides both where the streets lead and how traffic flows on them.

This article introduces a new framework called Partially Observed Structural Causal Models (POSCM) to deal with this chaos. Here is the breakdown in simple words:

1. The Problem: The "Invisible Architect"

In many real systems (such as the human brain or social networks), there is an "invisible architect" (called latent context) that we cannot see directly.

The old view: We assume the map is fixed. We only observe what happens.
The new view (POSCM): The map itself is drawn in real time by this invisible architect. The architect decides:
1. Structure: Which connections exist (e.g., does Neuron A talk to Neuron B?).
2. Mechanism: How these connections function (e.g., is the signal loud or quiet? Is it excitatory or inhibitory?).

The tricky part is that we often cannot see the architect, cannot clearly recognize the map, and can only see the final traffic (the data).

2. The Solution: A "Surgical" Toolkit

To understand this system, the authors developed a new "surgical" toolkit. They realized that to repair or understand the system, one must be able to cut specific wires, not just shut down entire machines.

Node Interventions: This is like turning off an entire light bulb (a node). You can see what happens downstream, but you do not know which wire carried the energy.
Edge Interventions: This is the major innovation of the article. It is like cutting a single wire between two specific light bulbs without touching the bulbs themselves.
- The Analogy: Imagine a recipe.
  - Node Intervention: "Do not use eggs." (You change the whole dish).
  - Edge Intervention: "Do not use the egg in the cake base, but keep it in the frosting." (You change exactly how one ingredient interacts with a specific part).

To make this "surgery" mathematically possible, they used a clever trick (based on a famous mathematical theorem) to resolve every complex interaction into simple, one-to-one conversations between pairs of variables. This allows them to isolate and test individual connections.

3. The Rules of the Game (Identifiability)

The article asks: "Can we actually figure out the hidden rules if we cannot see the architect?" They found that the answer depends on which tools you have:

Scenario A: You cannot see the architect, and you cannot touch them.
- Result: You are stuck. You cannot distinguish whether the map changed because the architect changed their mind, or whether only the traffic rules shifted. It is a "fog of war" where different hidden realities look exactly the same from the outside.
Scenario B: You cannot see the architect, but you can touch the connections (edges).
- Result: You can still be confused. If you only see the traffic, you might think a street is busy because it is a popular route, or because the traffic lights are broken. You cannot distinguish between a "bad street" and a "bad traffic rule."
Scenario C: You can touch the architect (or at least change their mind) AND you can touch the traffic.
- Result: Success! If you can force the architect to change the map (e.g., "Build a road here!") and you can also control the traffic lights, you can finally reverse-engineer the entire system. You can figure out exactly how the architect thinks and how the machines work.

4. The Proof: The Virtual Retina

To prove this works, the authors did not just do mathematics; they built a virtual human eye (a retina simulator).

The Setup: In a real eye, you can see the firing neurons (traffic), but often not simultaneously the cell types (the architect) or the exact wiring (the map).
The Test:
1. They tried to figure out the eye's wiring without knowing the cell types. Result: Failed (exactly like Scenario A).
2. They tried to figure out the wiring by only turning lights on and off, without cutting wires. Result: Failed (exactly like Scenario B).
3. They used a combination of changing cell types (simulating architect changes) and turning lights on and off. Result: They successfully reconstructed the exact wiring and the rules of how signals travel.

Summary

This article says: "The world is too complex for old maps where streets are fixed. We need a new way to model systems where streets are built by hidden forces. By using a new kind of 'surgical' mathematics that allows us to cut individual wires, and by knowing exactly what kind of experiments we need to conduct (changing both the hidden forces and the visible traffic), we can finally understand these complex, partially hidden systems."

They proved this works by successfully reverse-engineering a virtual human eye and showing that with the right mix of interventions, we can see through the fog.

Technical Summary: Partially Observed Structural Causal Models (POSCMs)

Problem Statement

Standard Structural Causal Models (SCMs) typically treat causal graphs as fixed, exogenous structures where mechanisms operate on observed variables. However, many real-world systems—particularly in biology, social sciences, and engineering—exhibit more complex dynamics: the interaction structure (edges) and downstream mechanisms are jointly determined by latent contexts. For instance, in neuroscience, a neuron's cell type (context) determines both its synaptic connectivity (structure) and its physiological response (mechanism), yet these contexts are often only partially observable or inferred indirectly.

Current causal discovery methods struggle in this regime because they assume fixed graphs and fully observed variables. When structure formation is endogenous to latent processes and only partial observations of variables, contexts, and edges are available, standard SCMs fail to disentangle "who is connected to whom" from "how they interact." This work bridges the gap between fully observed SCMs and the reality of partially observed systems where structure and mechanism are entangled with latent contexts.

Methodology

1. Formalism: Partially Observed SCMs (POSCMs)

The authors introduce Partially Observed Structural Causal Models (POSCMs), a framework formalizing causal systems where latent contexts ( $\beta$ ) govern both:

Structure Formation: The stochastic generation of directed edges ( $A$ ).
Value Mechanisms: The generation of observed variables ( $V$ ) along the realized graph.

The framework is based on an ordered generation process (an extension of Simon's causal ordering and Barabási-Albert's sequential node arrival). The generation process proceeds in two phases according to a topological order $\tau$ :

Phase I (Structure & Context): For each node $i$ , incoming edges $A_{ji}$ are drawn from a structural kernel $\alpha$ conditioned on the source context $\beta_j$ . Subsequently, the context $\beta_i$ is generated based on the contexts of its realized parent nodes. This nested process ensures the overall graph remains acyclic over $(\beta, A, V)$ .
Phase II (Mechanisms & Variables): Mechanisms $f_i$ are assigned based on the local context $\beta_i$ and the realized parent set. Finally, endogenous variables $V_i$ are generated.

An observation model $O$ explicitly defines what is observed (e.g., noisy $V$ , latent $\beta$ , or unobserved $A$ ), thereby separating the causal process from the observation process.

2. Message-Augmented Representation & KAS Decomposition

To enable "surgical" interventions on individual edges, the authors adopt a Kolmogorov-Arnold-Speaker (KAS) edge function decomposition.

Theorem: Any continuous mechanism can be represented as a sum of univariate functions of its parents.
Application: This allows the authors to define message variables ( $M_{i \leftarrow j}$ ) for each dyad. The mechanism at node $i$ is viewed as an aggregation of messages received from parent nodes.
Significance: This decomposition provides an explicit parameterization of dyadic functional contributions, making it possible to define interventions that target a single edge channel without altering the source node or other edges.

3. Intervention Hierarchy

POSCMs define a hierarchy of interventions comprising four primitive types:

$\beta$ -Node Interventions: Setting the latent context of a node (e.g., changing cell type).
$V$ -Node Interventions: Setting the value of an observed variable (standard do-operator).
$\beta$ -Edge Interventions: Replacing the context-message function on a specific channel.
$V$ -Edge Interventions: Replacing the value-message function on a specific channel.

This hierarchy enables "surgical" manipulation of structure and mechanism, distinguishing POSCMs from standard SCMs where edge interventions are often conceptual rather than operational.

Identifiability Theory

The work establishes rigorous conditions under which the components of a POSCM can be identified, highlighting both barriers and positive results.

Negative Results (Barriers)

Latent Contexts: If contexts are unobserved and no $\beta$ -interventions are available, the structural kernel, context propagation mechanisms, and mechanism assignment operators are not identifiable beyond a context reparameterization (Proposition 5.2). Node-level data ( $V$ ) alone cannot resolve the symmetry between different context labelings.
Latent Edges: If edges are unobserved, $V$ -node interventions are insufficient to predict the effects of $V$ -edge interventions. This creates a confounding of structure and mechanism, where a model with frequent weak edges is observationally equivalent to a model with rare strong edges under node-level interventions, yet they differ under edge-level interventions (Lemma 5.3).

Positive Results

Kernel Identifiability: Under specific assumptions (context/value readout, joint coverage by $\beta$ -node and $V$ -node interventions, and a probing-based structure readout), the population kernels (structural kernel $\alpha$ , context propagation $\phi$ , and mechanism operator $\Gamma$ ) are identifiable (Theorem 5.4).
Dyadic Message Identifiability: With richer access (including edge-message interventions or replay in paired worlds), specific dyadic message primitives can be identified up to a representation gauge (Theorem C.3).

Experimental Validation

The authors validate their theoretical predictions using a biophysically detailed virtual human retina simulator (NEURON). The retina serves as a natural POSCM where cell types (latent $\beta$ ) determine connectivity and synaptic physiology.

Experiment 1 (Non-Identifiability without $\beta$ -Interventions): The authors constructed "twin" retina models where cell type labels and synaptic rules were swapped. Under observational data and $V$ -node interventions (voltage clamps), the firing rate distributions were indistinguishable (KS-test $p > 0.9$ ), empirically confirming the reparameterization symmetry predicted by Proposition 5.2.
Experiment 2 (Confounding of Structure and Mechanism): Two models were calibrated to exhibit identical mean synaptic drives under node-level clamps (one with high density/low conductance, one with low density/high conductance). While node-level interventions yielded similar responses, edge-level interventions (perturbing specific synapses) revealed significant discrepancies (higher MMD), validating Lemma 5.3.
Experiment 3 (Kernel Identifiability): By traversing contexts (varying retinal eccentricity to alter cell type composition) and values (voltage clamps), the authors successfully recovered the context-dependent population composition and the sigmoidal synaptic transfer curve (BC $\to$ RGC), consistent with Theorem 5.4.

Main Contributions & Significance

Formal Framework: The work provides the first formal framework (POSCMs) treating structure formation as an endogenous causal mechanism driven by latent contexts, accounting for partial observability of both structure and context.
Intervention Vocabulary: It introduces a novel intervention hierarchy including edge-level operations, enabled by a KAS-based, message-augmented representation. This allows "surgical" causal manipulations previously undefined in standard SCMs.
Identifiability Diagnostics: The work provides not only positive guarantees but explicitly diagnoses failure modes (symmetries of latent contexts and confounding of latent edges). It clarifies precisely which families of interventions are necessary to disentangle structure from mechanism.
Empirical Proof-of-Concept: The virtual retina experiments demonstrate that these theoretical barriers and solutions occur in realistic biophysical systems and that kernel-level recovery is achievable under the proposed readout and coverage assumptions.

The authors conclude that POSCMs generalize SCMs to a world closer to reality, where interaction structures are not fixed scaffolds but dynamic, context-dependent outcomes that can be reasoned about and intervened upon.