🤖 machine learning

TriOpt: A Scalable Algorithm for Linear Causal Discovery

TriOpt is a scalable algorithm for linear causal discovery that integrates ordering-based and continuous optimization methods by first efficiently recovering the topological ordering via Sherman-Morrison updates and then solving a convex structure learning problem without acyclicity constraints, achieving significant speedups over state-of-the-art methods while maintaining high accuracy.

Original authors: Rafat Ashraf Joy, Elena Zheleva

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rafat Ashraf Joy, Elena Zheleva

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out the family tree of a large group of people, but you only have a photo album of them interacting, not a birth certificate. You need to guess who is the parent of whom based on how they look and act together. In the world of data science, this is called Causal Discovery: figuring out cause-and-effect relationships from observational data.

The problem is that as the number of people (variables) grows, the number of possible family trees explodes super-fast. It's like trying to find the one correct path through a maze that gets exponentially more complex with every new turn.

The paper introduces a new tool called TriOpt (Three-Fold Optimization) to solve this maze much faster and more accurately than previous methods, especially when dealing with huge datasets.

Here is how TriOpt works, broken down into simple steps and analogies:

The Problem with Old Methods

Before TriOpt, researchers used two main strategies, both of which had a major flaw:

The "Order-First" Method: Imagine trying to build a family tree by first guessing the order of generations (Grandparents, then Parents, then Kids), and then drawing the lines.
- The Flaw: Every time they guessed a "leaf" (someone with no children) and removed them from the list to check the next person, they had to completely recalculate a giant math chart (a kernel matrix) from scratch. It's like re-reading an entire encyclopedia every time you remove one word from a sentence. This made it incredibly slow for large groups.
The "Continuous Optimization" Method: This approach tries to draw the whole tree at once by sliding a slider until the picture looks right.
- The Flaw: To make sure the tree doesn't have loops (like a child being their own grandparent), the computer has to perform a very heavy, complex calculation (a matrix exponential) at every single step. It's like trying to drive a car while constantly checking if the engine is still running by taking it apart and reassembling it. It's accurate but painfully slow.

The TriOpt Solution: A Three-Step Shortcut

TriOpt combines the best parts of both methods and adds a "magic trick" to make it fast.

Step 1: The "Magic Eraser" (Fast Ordering)

TriOpt still starts by guessing the order of generations. However, instead of recalculating the giant math chart from scratch every time it removes a person, it uses a mathematical trick called the Sherman-Morrison downdate.

The Analogy: Imagine you have a giant spreadsheet. When you delete a row, instead of re-typing the whole sheet, you just make a tiny, specific adjustment to the existing numbers. TriOpt does this mathematically. It realizes that because the relationships are "linear" (straight lines), removing a variable is a simple, low-effort update.
The Result: This turns a task that used to take hours into one that takes minutes, even for thousands of variables.

Step 2: The "One-Way Street" (Convex Optimization)

Once TriOpt has the correct order (e.g., Grandparents $\to$ Parents $\to$ Kids), it knows the rules of the road: Parents can only influence children who come after them in the list.

The Analogy: In the old methods, the computer had to constantly check, "Is this a loop? Is this a dead end?" TriOpt simply draws the map on a piece of paper where only forward movement is allowed. It forces the computer to only look at the "upper triangle" of the data.
The Result: Because the computer no longer has to check for loops, the math problem becomes "convex." In plain English, this means the landscape is a smooth bowl rather than a jagged mountain range. The computer can slide straight to the bottom (the perfect answer) without getting stuck in a local valley.

Step 3: The "No-Loop Guarantee"

Because the computer is forced to only look forward (based on the order found in Step 1), it is mathematically impossible to create a loop.

The Result: The expensive "loop-checking" math is thrown out the window entirely. The computer just solves a standard, fast equation.

Why This Matters (According to the Paper)

The authors tested TriOpt on synthetic data (made-up scenarios), semi-synthetic data (real gene networks), and real-world data (protein signaling in human cells).

Speed: TriOpt is orders of magnitude faster than the current best methods. In some tests with 1,000 variables, it was 95% to 97% faster than its competitors.
Accuracy: Despite being so fast, it is just as accurate, and sometimes even more accurate, than the slower methods.
Scalability: While other methods crash or take forever when the dataset gets big (high-dimensional), TriOpt scales up smoothly.

The One Catch

The paper notes a small limitation: The "Magic Eraser" trick (Sherman-Morrison) works perfectly for most data but can get a little shaky if the data has very specific, weird noise patterns (like Exponential or Gumbel distributions). However, the authors built a safety net into the code to fix this if it happens.

In summary: TriOpt is like upgrading from a car that has to stop and check the map at every intersection to a high-speed train that knows the tracks are one-way. It gets you to the destination (the correct causal graph) much faster without getting lost.

Technical Summary: TriOpt – A Scalable Algorithm for Linear Causal Discovery

Problem Definition

Causal discovery aims to learn Directed Acyclic Graphs (DAGs) representing cause-and-effect relationships from observational data. While randomized controlled trials are the gold standard, they are often infeasible due to cost or ethical constraints. Consequently, learning causal structures from observational data is a fundamental challenge.

The primary bottleneck in this domain is the super-exponential growth of the graph search space with the number of variables ( $d$ ). Existing approaches generally fall into two paradigms, both of which face significant scalability limitations in high-dimensional settings:

Ordering-based methods (e.g., SCORE, CaPS, DiffAN): These first identify a topological ordering and then prune spurious edges. While effective, they often rely on kernelized Stein discrepancy, requiring the recomputation and inversion of kernel matrices ( $O(d^2n^2)$ and $O(dn^3)$ ) at every step of variable removal. Furthermore, the post-processing pruning step (e.g., CAM) can be prohibitively slow in high dimensions.
Continuous optimization methods (e.g., NOTEARS, GOLEM, DAGMA): These cast DAG learning as a differentiable objective with an acyclicity constraint. While they leverage GPU acceleration, enforcing the acyclicity constraint typically involves costly matrix exponentials ( $O(d^3)$ ) and introduces non-convexity, leading to vanishing gradients and susceptibility to local optima.

Methodology: The TriOpt Framework

The paper introduces TriOpt (Three Fold Optimization), a framework that integrates ordering-based and continuous optimization paradigms to achieve scalability without sacrificing accuracy. TriOpt decomposes linear causal discovery into two efficient stages:

Stage 1: Fast Ancestral Ordering via Linear Kernel

Instead of using the standard Radial Basis Function (RBF) kernel which necessitates expensive recomputation, TriOpt employs a linear kernel for score matching.

Additivity and Sherman-Morrison: The linear kernel allows the algorithm to exploit the additive structure of the kernel matrix. When a leaf node is identified and removed, the kernel matrix undergoes a rank-1 modification. TriOpt utilizes the Sherman-Morrison rank-1 downdate formula to update the kernel inverse efficiently.
Complexity Reduction: This approach reduces the computational cost of kernel matrix and inverse computation from $O(d^2n^2)$ and $O(dn^3)$ to $O(dkn^2)$ and $O(dn^2 + kn^3)$ respectively (where $k \ll d$ is the frequency of re-inversion for numerical stability).
Leaf Identification: The algorithm iteratively identifies leaf nodes by maximizing the empirical mean of the Hessian proxy derived from the Stein score estimator, consistent with the identifiability criteria established in CaPS.

Stage 2: Convex Continuous Optimization

Once the topological ordering is recovered, the search space is restricted to strictly upper-triangular matrices (assuming the ordering is correct).

Removal of Acyclicity Constraint: Unlike NOTEARS or DAGMA, TriOpt does not need to enforce a global acyclicity constraint because the hard upper-triangular constraint (enforced via a binary mask) inherently guarantees acyclicity given the correct ordering.
Convexity: By removing the non-convex acyclicity constraint, the optimization problem becomes convex. The objective function consists of a least-squares loss with a smooth $\ell_1$ (Charbonnier) regularization for sparsity.
Theoretical Guarantee: The authors prove that under the true ordering and linear Structural Equation Model (SEM) assumptions, optimizing this convex objective provably recovers the true DAG in the population setting. The unique global minimizer corresponds to the transpose of the true adjacency matrix.

Key Contributions

Novel Formulation: TriOpt tightly integrates ordering-based and continuous optimization methods, addressing the scalability bottlenecks of both.
Efficient Ordering: By leveraging the linear kernel and Sherman-Morrison downdate, TriOpt drastically reduces the runtime of the ordering phase, avoiding the $O(dn^3)$ bottleneck of previous Stein ordering methods.
Convex Optimization: The framework reformulates structure learning as a convex problem by utilizing the recovered ordering to impose a hard upper-triangular constraint, eliminating the need for expensive acyclicity constraints and ensuring convergence to a global minimum.
Theoretical Guarantees: The paper provides a proof that TriOpt exactly recovers the underlying linear DAG under the true ordering in the population setting.

Experimental Results

TriOpt was evaluated on synthetic, semi-synthetic, and real-world datasets against state-of-the-art baselines (NOTEARS, GOLEM, DAGMA, and CaPS).

Scalability: In high-dimensional regimes ( $d=1000$ ), TriOpt achieves orders-of-magnitude speedups. Specifically, TriOpt with Sherman-Morrison is approximately 97% faster than NOTEARS, 96% faster than GOLEM, and 93% faster than DAGMA.
Accuracy: TriOpt maintains comparable or superior accuracy (measured by Normalized SHD, F1 Score, and AID) across various graph types (Erdős–Rényi, Scale-Free) and noise distributions (Gaussian, Gumbel, Exponential).
Robustness:
- TriOpt remains robust even when the linearity assumption is violated (misspecified regime with nonlinear data), outperforming linear baselines in nonlinear settings.
- The method is highly robust to certain types of ordering errors (e.g., adjacent swaps, block reversals) but degrades with structurally significant errors (ancestor swaps, hub swaps), which is consistent with theoretical expectations.
Ablation Studies:
- The use of the linear kernel and Sherman-Morrison downdate significantly reduces memory usage and runtime compared to CaPS, allowing TriOpt to run on dimensions where CaPS fails due to out-of-memory errors.
- The upper-triangular optimization step is significantly faster (up to 664x on 50 nodes) and more accurate than CAM pruning.

Significance and Claims

The paper claims that TriOpt represents a significant step forward in making causal discovery drastically more scalable for high-dimensional settings. By removing the computational overhead of kernel recomputation and the non-convexity of acyclicity constraints, TriOpt enables the application of causal discovery to problems previously considered computationally intractable.

The authors note that while the framework assumes linear relationships, it offers good approximations in many real-world settings and remains robust even when nonlinearity is present. A limitation acknowledged is the potential numerical instability of the Sherman-Morrison downdate under specific skewed noise distributions (Exponential, Gumbel), which is mitigated by periodic re-inversion strategies. The work suggests that TriOpt provides a practical, scalable solution for linear causal discovery that balances computational efficiency with theoretical rigor and empirical accuracy.