Technical Summary: EML-CD – Causal Mechanism Recovery via EML Symbolic Trees

1. Problem Statement

Current causal discovery methods face a fundamental trade-off between structural accuracy and mechanistic interpretability. While continuous optimization methods (e.g., NOTEARS, GOLEM) and neural network-based additive noise models (e.g., GraNDAG, NOTEARS-MLP) achieve high accuracy in recovering Directed Acyclic Graph (DAG) structures, they represent causal mechanisms as black-box neural networks. As argued by Waxman et al. [22], extracting specific functional forms from neural network weights is an ill-posed problem; the weights do not uniquely identify the underlying mechanism. Consequently, existing methods cannot provide closed-form equations to predict intervention effects or analyze causal effect functions quantitatively. Post-hoc explanation tools (SHAP, LIME) offer only local approximations, and spline-based models (CAM) cannot be extracted as symbolic equations. No existing framework jointly recovers causal structure and arbitrary nonlinear closed-form mechanisms within a single learning process.

2. Methodology: EML-CD

EML-CD (EML-based Causal Discovery) addresses this gap by integrating the EML operator [14] into causal structure learning. The EML operator, defined as $eml(x, y) = \exp(x) - \ln(y)$, is functionally complete for elementary functions when composed via a grammar $S \to 1 \mid eml(S, S)$.

2.1 EML Tree Representation

Instead of using neural networks, EML-CD represents each edge mechanism $f_{j,i}(X_i)$ as a gated EML binary tree of bounded depth $D$.

• Structure: The tree is a complete binary tree where internal nodes apply the EML operator or bypass it via a gating mechanism.
• Gating: A depth gate $\sigma(\gamma)$ controls complexity. If $\gamma \ll 0$, the node bypasses the EML operation (degenerating to a linear path); if $\gamma \gg 0$, the EML operation is active. This allows the model to perform data-driven selection between linear and nonlinear mechanisms.
• Training: The model employs a deterministic annealing schedule for the gate temperature $\tau_g$, annealing from 1 to 0.02 during training. This ensures that gradients flow freely early on, while the gates converge to near-binary states, yielding a hard-thresholded closed-form equation that faithfully represents the trained soft function.
• Stability: To prevent numerical divergence (e.g., from $\exp$ or $\ln$), the operator clips inputs and outputs to specific bounds (e.g., $\exp$ input to $[-2, 2]$, $\ln$ argument floored at 0.5).

2.2 Two-Phase Inference

The learning process consists of two distinct phases:

1. Mechanism Pre-training: For all $d(d-1)$ potential pairs $(i, j)$, an EML tree $T_{j,i}$ is trained via Adam to minimize Mean Squared Error (MSE). The best of three random initializations is selected for each pair. This phase yields $R^2$ scores (predictive power) and trained tree parameters.
2. Greedy DAG Construction: Edges are added to the graph in decreasing order of their $R^2$ scores, strictly maintaining acyclicity. Edges with $R^2$ below a threshold $\tau$ are excluded. Optional backward pruning using BIC (based on OLS) can further remove spurious edges.

2.3 Identifiability and Direction

EML-CD orients edges based on fit-quality asymmetry ($R^2_{i \to j}$ vs. $R^2_{j \to i}$) rather than residual independence tests. The restriction of the function class to EML trees makes the anticausal mechanism harder to represent, naturally lowering its $R^2$. This serves as a heuristic proxy for Additive Noise Model (ANM) identifiability.

3. Key Contributions

1. Framework Integration: EML-CD is the first framework to integrate EML symbolic trees directly into causal structure learning, attaching closed-form equations and analytical Jacobians to every detected edge.
2. Interpretability without Sacrifice: It demonstrates that adding symbolic interpretability does not necessarily degrade structure recovery accuracy compared to state-of-the-art baselines.
3. Faithful Mechanism Recovery: In controlled settings, the method recovers the functional form of elementary functions with high fidelity, rather than just fitting a curve.
4. Analytical Causal Effects: By deriving equations directly from the tree structure, EML-CD enables the computation of analytical Jacobians ($\partial X_j / \partial X_i$), providing state-dependent causal effect quantification unavailable from linear coefficients or numerical NN derivatives.

4. Experimental Results

4.1 Real Data: Sachs Protein Signaling ($d=11$)

• Structure Recovery: EML-CD achieved a Structural Hamming Distance (SHD) of 11.2 ± 0.4 (5-seed mean). This is on par with PC and GES (SHD=11) and better than CAM (SHD=12).
• Precision/Recall: It achieved the highest precision (0.756) among all methods, though recall was lower (0.365), indicating a preference for high-confidence edges.
• Mechanisms: The method successfully recovered known pathways (e.g., Raf $\to$ Mek) with closed-form equations. For instance, it identified a nonlinear mechanism for PKC $\to$ P38 with a state-dependent Jacobian ranging from -5.4 to +2.3, and a linear mechanism for Erk $\to$ Akt.

4.2 Controlled Bivariate Recovery

In a test with known ground-truth functions ($N=2000$), EML-CD recovered 10 out of 11 elementary function families (linear, quadratic, cubic, cos, tanh, exp, sqrt, sigmoid, log, abs) with held-out shape correlation $\ge 0.96$. Only high-frequency sine ($\sin(1.5x)$) showed partial recovery (0.84).

4.3 Synthetic Symbolic Benchmark (S-Sym)

• Stability vs. Fixed Dictionaries: On a symbolic SEM benchmark, EML-CD achieved a mean mechanism $f$-MSE of 3.67, significantly lower and more stable than a fixed SINDy dictionary (mean 7644). The SINDy baseline suffered from catastrophic extrapolation on one seed.
• Structure Recovery: The structural SHD (14.0) matched the SINDy dictionary baseline but remained below specialized linear optimizers.
• Limitation: While stable, the method did not reliably recover highly nonlinear shapes (e.g., saturation of tanh) at depth 2 with the given sample size.

4.4 Causal Chambers (Light-Tunnel Subset)

On a physical system governed by Malus's law ($\cos^2(\theta_1 - \theta_2)$), a depth-2 EML-CD model improved F1 (0.444) over linear OLS-BIC (0.273). However, depth-3 models failed in this small-sample setting. Incorporating interaction-aware leaves and intervention metadata improved precision to 1.000, though recall remained limited.

5. Significance and Claims

The paper positions EML-CD as a proof of concept demonstrating that symbolic causal-mechanism interpretability can be integrated into structure learning without sacrificing structural accuracy.

• Core Achievement: It circumvents the ill-posedness of NN-based mechanism extraction by restricting the search space to a finite-dimensional, interpretable EML function class.
• Practical Value: The ability to output analytical Jacobians allows for quantitative prediction of intervention effects and state-dependent causal analysis, which is impossible with standard DAG-only methods.
• Modesty: The authors explicitly acknowledge limitations:
  • The method currently assumes univariate additive mechanisms and cannot natively represent cross-variable interactions (e.g., $\cos^2(\theta_1 - \theta_2)$) without specific extensions.
  • Structural recovery on synthetic benchmarks matches dictionary baselines but does not yet surpass specialized linear optimizers.
  • The stability of the recovered equations across seeds requires further improvement.
  • The EML operator itself is a recent development not yet independently validated at scale.

The work concludes that EML-CD provides initial evidence that attaching closed-form equations to causal graphs is feasible, offering a path toward transparent, analytically tractable causal models.