Technical Summary: SC3D – Stable Causal Dynamic Differentiable Discovery

1. Problem Formulation

The paper addresses the challenge of discovering causal structures from multivariate time series data where interactions occur across multiple time lags and potentially involve instantaneous dependencies. Traditional causal discovery in time series faces two primary hurdles:

1. Combinatorial Search Space: The search space for dynamic graphs involving both lagged (inter-slice) and instantaneous (intra-slice) edges is combinatorial, making exhaustive search infeasible for high-dimensional systems.
2. Numerical Instability: Differentiable causal discovery (DCD) methods, which formulate structure learning as continuous optimization, often suffer from numerical instability when enforcing acyclicity constraints, particularly as the number of variables increases. Existing smooth acyclicity constraints (e.g., trace-based) deteriorate quickly beyond tens of variables.

The authors propose SC3D (Stable Causal Dynamic Differentiable Discovery), a framework designed to jointly learn lag-specific adjacency matrices and an instantaneous Directed Acyclic Graph (DAG) within a nonlinear dynamic Structural Equation Model (SEM).

2. Methodology

SC3D employs a two-stage differentiable framework that bridges structural vector autoregressive (SVAR) models with stable differentiable causal discovery. The method operates on observational multivariate time series $\{X_t\}_{t=1}^T$.

Stage 1: Node-wise Temporal Preselection

The first stage aims to reduce the combinatorial search space by identifying candidate causal parents without enforcing acyclicity constraints.

• Process: For each target variable $X_j^{t+1}$, a conditional predictive model (implemented as a Multilayer Perceptron, MLP) is fitted using a temporal window of candidate parents, including lagged variables up to order $L$ and contemporaneous variables (excluding self-loops).
• Objective: The model maximizes a penalized conditional log-likelihood with an $L_1$ regularization term to enforce sparsity.
• Output: The learned input group magnitudes are thresholded to generate binary masks ($M_{A_\ell}$ for lagged edges and $M_B$ for instantaneous edges).
• Theoretical Guarantee: Theorem 3.1 establishes that, at the population level, this preselection step preserves all true dynamic parents within the chosen temporal window, ensuring no true causal relationships are discarded before the refinement stage.

Stage 2: Constrained Structure Refinement

The second stage refines the causal structure using the masks generated in Stage 1, focusing on enforcing the acyclicity constraint specifically on the instantaneous component.

• Optimization: The model parameters are re-optimized to minimize a regularized negative log-likelihood, restricted to the edges allowed by the Stage 1 masks.
• Acyclicity Enforcement: Unlike previous methods that apply smooth trace-based constraints to the entire graph, SC3D enforces acyclicity only on the instantaneous adjacency matrix $B$. This is achieved via a spectral radius penalty $\rho(|B|)$, which equals zero if and only if the graph is acyclic.
• 2-Cycle Penalty: To address potential instability from reciprocal edges that might satisfy a small spectral radius, an additional $L_1$ penalty on the element-wise product $B \odot B^\top$ is introduced to penalize 2-cycles directly.
• Penalty Scheduling: The spectral penalty coefficient $\gamma$ is gradually increased during training. Once the extracted instantaneous graph satisfies the acyclicity constraint and retains a sufficient number of edges, the penalty is "frozen" to prevent oscillatory behavior and stabilize convergence.
• Theoretical Insight: Proposition 3.2 proves that if the instantaneous block $B$ is acyclic, the entire time-unrolled causal graph is guaranteed to be acyclic, as lagged edges inherently move forward in time and cannot form cycles.

3. Key Contributions

The paper outlines four primary contributions:

1. Novel Formulation: It formulates causal discovery for dynamical systems using lag-specific adjacency matrices and instantaneous DAG components, effectively bridging structural VAR models with stable differentiable causal discovery.
2. Two-Stage Approach: It introduces a screening and constrained refinement strategy. By screening candidate parents in a lagged predictor window before enforcing acyclicity, the method avoids the numerical instability associated with optimizing high-dimensional acyclic graphs directly.
3. Theoretical Guarantees: The authors provide population-level guarantees proving that Stage 1 preserves all true causal parents and that the acyclicity of the instantaneous dependencies is sufficient to guarantee the acyclicity of the time-unrolled graph.
4. Empirical Validation: The framework is evaluated on synthetic SVAR systems, nonlinear and chaotic benchmarks (Lorenz-96, NC8), nonstationary dynamics (TVSEM), and real-world datasets (CausalRivers), demonstrating improved stability and accuracy over existing baselines.

4. Experimental Results

The authors evaluated SC3D against baselines including DYNOTEARS, PCMCI+, NTS-NOTEARS, TECDI, VAR-LiNGAM, and NeuralGC.

• Scalability and Stability: In experiments varying the dimension $d$ from 10 to 50, SC3D demonstrated the lowest total Structural Hamming Distance (SHD). While other methods' errors grew rapidly with dimension, SC3D maintained robust performance. It achieved lower lag-only errors ($SHD_A$) compared to DYNOTEARS and TCDF, indicating superior recovery of temporal dependencies.
• Ranking Performance: Using AUROC and AUPRC metrics, SC3D achieved top-tier performance in ranking lagged causal relationships (AUROC $\approx$ 0.91) and competitive performance in ranking instantaneous edges (AUPRC $\approx$ 0.715), outperforming specialized baselines in sparse network settings.
• Robustness to Lag Order: Sensitivity analysis showed that as the lag order $L$ increased, the instantaneous component of the error remained stable, confirming that the two-stage design effectively controls the search space even with deeper temporal dependencies.
• Dynamical Systems:
  • Lorenz-96 (Chaotic): SC3D achieved the lowest top-$k$ structural error and highest AUROC/AUPRC, outperforming PCMCI+ and TECDI.
  • TVSEM (Nonstationary): In a regime-switching environment, SC3D produced smoother and more temporally consistent score trajectories compared to baselines, which exhibited sharper but less stable transitions.
  • Real-World (CausalRivers): On river discharge data, SC3D achieved competitive structural accuracy and ranking metrics, particularly maintaining performance at higher dimensions ($d=20$) where other methods degraded significantly.
• Ablation Study: Removing Stage 1 preselection caused a catastrophic increase in total SHD (e.g., from ~87 to ~637 at $d=20$), confirming that the preselection step is critical for controlling the search space. The spectral and 2-cycle penalties were also shown to be essential for stable recovery in high dimensions.

5. Significance and Claims

The paper claims that SC3D offers a stable, scalable, and differentiable solution for causal discovery in multivariate dynamical systems. Its significance lies in:

• Overcoming Numerical Instability: By decoupling the search space reduction (Stage 1) from the acyclicity enforcement (Stage 2) and applying spectral constraints only to the instantaneous block, it avoids the numerical pitfalls of smooth trace-based constraints in high dimensions.
• Joint Recovery: It is one of the few methods capable of jointly recovering both lagged and instantaneous causal structures while supporting nonlinear dependencies.
• Theoretical Soundness: The framework provides formal guarantees regarding the preservation of true parents during screening and the sufficiency of instantaneous acyclicity for global acyclicity.

The authors position SC3D as a robust alternative to existing methods that either model lagged and instantaneous dependencies independently or struggle with scalability and stability when learning them simultaneously. The work is presented as a step toward reliable mechanistic intervention and reasoning under distribution shifts in complex dynamical systems.