🤖 AI

Decentralized Causal Discovery using Judo Calculus

This paper presents a decentralized, intuitionistic framework for causal discovery called "judo calculus," which utilizes sheaf theory and Lawvere-Tierney modal operators to formalize context-dependent causal claims as locally true across regimes, demonstrating improved computational efficiency and performance over classical methods in diverse real-world applications.

Original authors: Sridhar Mahadevan

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Sridhar Mahadevan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective's Dilemma: Finding Truth in a Crowded Room

Imagine you are a detective trying to solve a mystery: what causes what? In the world of data science, this is called causal discovery. Usually, detectives gather all their clues into one giant pile to find patterns. But what if those clues come from different places with different rules? Maybe you have medical data from a hospital in Tokyo, a lab in Berlin, and a study on mice in a cage. If you just mash them all together, you might miss the fact that the "rules of the game" changed in each location. A medicine might work in one environment but fail in another, or a cause might disappear entirely if someone interfered with the process.

To solve this, scientists use a concept called stability. The idea is simple: if a cause-and-effect relationship is real, it should show up consistently across different settings, like a song that sounds the same whether played on a piano, a guitar, or a synthesizer. However, figuring out which relationships are truly stable and which are just lucky coincidences is incredibly hard, especially when some experiments actively change the system (like turning off a switch). This paper tackles the problem of how to combine these different "local" discoveries into one reliable picture without getting confused by the noise or the changes.

The "Judo" Solution: Throwing Your Weight Around

This paper introduces a clever new way to solve this puzzle, which the authors call "Judo calculus" (or formally, J-stable discovery). The name comes from the martial art Judo, where you use an opponent's force against them. In this case, the "opponent" is the messy, conflicting data from different environments. Instead of trying to force all the data to agree perfectly, the method uses the differences to filter out the weak links.

Here is how the "Judo" move works:

The Local Scouts: Imagine you send a team of detectives (called "base learners") to different neighborhoods (regimes) to draw their own maps of how things are connected. One detective works in a quiet library, another in a noisy construction site, and a third in a lab where they are actively smashing things (interventions).
The Cover: The paper calls the group of neighborhoods a "cover." Each detective draws a graph showing which variables point to which others.
The Stability Filter: Instead of just averaging all the maps, the system looks for edges (connections) that appear in enough of the maps to be considered "stable." If a connection shows up in 9 out of 10 neighborhoods, it's likely real. If it only shows up in 1, it's probably a fluke.
The Judo Twist (Intervention Awareness): This is the most important part. Sometimes, a detective is in a neighborhood where they broke a specific machine (an intervention). If the machine is broken, the detective won't see the wires connecting to it. A normal method might say, "Oh, that wire doesn't exist!" But the Judo method knows better. It uses a special "mask" to say, "Ignore the fact that this detective didn't see the wire; they were busy breaking the machine." This prevents the system from deleting real connections just because an experiment changed the rules.

What the Paper Actually Found

The authors tested this "Judo" method using three different types of detective tools:

Score-based (GES): Tools that look for the "best fit" map.
Constraint-based (ψ-FCI): Tools that look for rules about what cannot be connected.
Gradient-based (DCDI): Tools that use math to slowly improve the map.

They ran these tools on synthetic data (computer-generated worlds where they knew the true answer), the Sachs protein-signaling data (real biological data from cells), LINCS (drug perturbation data), and PISA (student test scores).

The Good News:
In the computer simulations, the Judo method was very good at cleaning up the mess. When they used a strict rule (like requiring an edge to appear in almost all environments), the method successfully removed "brittle" edges—connections that looked real but were actually just noise. In some cases, this made the final map much more accurate than just looking at all the data mixed together. It also showed that this process can be done in parallel, meaning different computers can work on different neighborhoods at the same time, which is great for speed.

The Bad News (and the Reality Check):
The paper is very honest about what this method cannot do.

Stability is not Proof: Just because a connection is stable across many environments doesn't automatically prove it is a true cause. A stable lie is still a lie. If the local detectives are all making the same mistake, the Judo method will happily agree with them.
It Depends on the Choices: The result depends heavily on which neighborhoods you choose to include and how you define an intervention. If you pick the wrong groups of data, the final map will be wrong.
It's an Approximation: The authors emphasize that this is a statistical shortcut, not a magical mathematical proof. It helps find good candidates, but it doesn't replace the deep theoretical work needed to prove causality.

The Verdict

The paper concludes that "decentralized stability filtering" is a useful tool for cleaning up causal maps and removing weak, unreliable connections. It works well when you have data from many different places and you want to find the common threads. However, it is not a magic wand. It cannot turn bad data into good data, and it cannot prove causality on its own. The "Judo" move helps you throw out the noise, but you still have to be careful about what you keep.

In short, the paper suggests that by letting local experts do their own work and then carefully combining their results while respecting the rules of their specific environments, we can build better, more robust maps of cause and effect. But as the authors warn, the final picture is only as good as the choices we make about which data to include and how we handle the experiments that change the game.

Based on the provided preprint, here is a detailed technical summary of the paper "Decentralized Causal Discovery Using Judo Calculus."

Problem Statement

Causal discovery is frequently performed on pooled data from distinct regimes (e.g., different environments, laboratories, or interventions). While pooling increases sample size, it risks mixing heterogeneous mechanisms and obscuring which causal conclusions persist across regimes. Conversely, fitting models independently for each regime exposes heterogeneity but leaves the challenge of combining noisy local graphs into a coherent global structure.

The paper addresses the problem of decentralized causal discovery: how to aggregate local causal estimates (graphs, conditional independences, or mechanisms) from a pre-specified family of regimes (a "cover") without assuming a universal causal structure a priori.

Methodology: J-Stable Discovery ("Judo Calculus")

The authors propose a method-agnostic framework called J-stable discovery (informally "judo calculus"). This procedure operates on a "cover" of regimes $E = \{e_1, \dots, e_m\}$ and follows these steps:

Local Learning: A base causal learner (score-based, constraint-based, or gradient-based) is applied independently to each regime $e \in E$ .
Support Aggregation: Local outputs (e.g., edge selections $A(e)_{ij}$ $A (e)_{ij}$ ) are aggregated. The paper distinguishes between:
- Unweighted Support: The frequency of an edge $i \to j$ across the cover.
- Intervention-Aware Support: A critical modification where regimes in which the child mechanism was directly intervened upon are excluded from the support calculation for that specific edge. This prevents the "mutilation" of the underlying pre-intervention DAG from falsely deleting true edges.
Thresholding: Edges are retained if their aggregated support exceeds a threshold $\pi$ . The authors analyze thresholds corresponding to intersection ( $\pi=1$ ), union ( $\pi=1/m$ ), and intermediate values.

Theoretical Context:
The terminology is motivated by the companion theory of j-do-calculus (Mahadevan, 2025), which establishes that if ordinary do-calculus identities hold in compatible local causal models, they descend across a cover. However, the authors explicitly state that their statistical algorithms are approximations to this local-to-global pattern. High empirical edge frequency is evidence of repeatability but does not, without further assumptions, prove d-separation, identify adjustment sets, or orient edges.

Algorithmic Realizations:
The framework is instantiated with three types of learners:

Score-based: Variants of GES (Greedy Equivalence Search), including structural regularization (CGES) and invariance regularization (TCES) that penalizes mechanism variation across environments.
Constraint-based: Variants of FCI/ψ-FCI, aggregating skeletons and orientation marks from local Partial Ancestral Graphs (PAGs).
Gradient-based: Variants of DCDI (Differentiable Causal Discovery via Interventions), utilizing post-hoc aggregation or joint training with regime-specific heads.

Key Contributions

Method-Agnostic Framework: A decentralized procedure for aggregating graph and conditional independence estimates across a cover of regimes.
Intervention-Aware Statistics: A support statistic (Eq. 1) that excludes regimes where a child mechanism was directly replaced, avoiding the erroneous deletion of true edges in the pre-intervention DAG.
Algorithmic Instantiations: Concrete implementations using GES, ψ-FCI, and DCDI, including specific regularizers for invariance and sparsity.
Statistical Guarantees: A finite-sample concentration statement (Proposition 1) for support thresholding under explicit repeated-selection assumptions, clarifying that this guarantees repeatability, not necessarily causal identifiability.
Empirical Analysis: A comprehensive study demonstrating both the successes (improving Structural Hamming Distance in sparse settings) and limitations (loss of recall, sensitivity to regime choice) of the approach.

Experimental Results

The paper evaluates the method on synthetic linear structural causal models and three real-world datasets:

Synthetic Data: Decentralized aggregation (specifically intersection) often yields lower Structural Hamming Distance (SHD) and higher precision compared to pooled learning, particularly in sparse settings. However, this comes at the cost of reduced recall.
Sachs Protein-Signaling Data: The pooled estimator achieved a higher F1 score against the published reference network compared to the intersection method. The intersection method reduced SHD but removed supported edges, illustrating that stability filtering is not uniformly superior and can discard useful signal.
LINCS Perturbation Signatures: Aggregation thresholds aggressively filtered the graph (e.g., reducing 428 union candidates to 14 in the intersection). Without a gold-standard DAG, these results are descriptive.
OECD PISA Variables: Used as a sanity check for recovering component-to-composite relations, confirming that domain guards (forbidding arrows from composites to components) are necessary.
Computational Efficiency: The decentralized approach allows for parallelization across workers. While total compute may equal or exceed a pooled run, wall-clock time can be reduced significantly if workers are available.

Significance and Claims

The paper maintains a modest and precise stance regarding its contributions:

Practical Utility: Decentralized stability filtering is effective for removing brittle edges and enabling parallel computation behind data silos.
Theoretical Limitation: The paper explicitly rejects the claim that high-frequency edges constitute a proof of do-calculus premises. The logical direction is one-way: compatible chartwise causal proofs imply local-to-global identities; the converse (high frequency $\implies$ do-calculus premise) does not hold without additional assumptions.
Context Dependence: The causal interpretation of the results depends heavily on the choice of regimes, intervention masks, local learners, and compatibility assumptions.
Revision Scope: This version deliberately removes previous claims regarding universal Kan-extension constructions and categorical universality, delegating those theoretical aspects to the companion j-do-calculus paper. It focuses instead on a practical, empirically grounded framework for multi-environment discovery.

In conclusion, the paper presents a "practical multi-environment discovery framework" that offers an auditable stability filter and intervention-aware corrections, while honestly acknowledging that stability aggregation does not manufacture causal premises but rather helps propose and assess stable local structures.