A Gradient-based Causal Discovery Framework with Applications to Complex Industrial Processes

Technical Summary: Gradient-based Causal Discovery (GCD) Framework

Problem Statement
Modern industrial processes generate large-scale, complex, multi-source data, necessitating robust tools for inferring intrinsic variable interactions. While deep learning-based causal discovery models have advanced the field, existing approaches face three primary limitations when applied to complex industrial systems:

1. Computational Inefficiency: Most state-of-the-art models (e.g., cMLP, cLSTM) utilize a "component-wise" architecture, requiring the construction of a separate neural network for each variable in the time series. This scales poorly with the number of sensors (dimensions), leading to substantial computational costs.
2. Limitations of Weight-Based Inference: Current methods often infer causal relationships by examining the sparsity of the first-layer weights of neural networks. In high-dimensional settings, this strategy can struggle with broad search spaces, potentially converging to local optima and failing to capture complex, nonlinear interactions effectively.
3. Scalability: Intricate network architectures with numerous parameters hinder the practical deployment and scalability of these models in large-scale industrial scenarios.

Methodology
The paper proposes Gradient-based Causal Discovery (GCD), a lightweight framework designed to address these challenges through a unified architecture and a novel inference mechanism.

• Unified Architecture: Unlike component-wise models that train $D$ separate models for $D$ variables, GCD employs a single Multilayer Perceptron (MLP) to predict all variables simultaneously. The model takes past time series values ($x_{t-k:t-1}$) as input and predicts current values ($\hat{x}_t$).
• Gradient-Based Causal Inference: Instead of relying on first-layer weights, GCD infers causality from the input-output gradients of the neural network. The intuition is that the gradient $\frac{\partial \hat{x}_{t,i}}{\partial x_{t-\tau,j}}$ quantifies the sensitivity of the prediction for variable $i$ to the past values of variable $j$.
  • A large gradient magnitude implies a causal influence (Granger causality), while a near-zero gradient suggests independence.
  • The causal strength matrix ($GC$) is computed by averaging the absolute values of these gradients over time steps and lags.
• Sparsity and Regularization: To ensure a sparse causal matrix, an $\ell_1$ regularization term is applied directly to the gradient-based causal matrix ($GC$) within the loss function ($L = L_{pred} + \lambda \|GC\|_1$).
• Statistical Validation (PSST): To address noise and false positives, the authors introduce a Phase-randomization Surrogate Statistical Test (PSST). Unlike traditional time-shift surrogates, PSST uses Fast Fourier Transform (FFT) to randomize phases while preserving amplitude spectra, effectively breaking dependencies even in signals with strong periodic patterns. Empirical null distributions are constructed using surrogate datasets, and p-values are adjusted using the Benjamini-Hochberg False Discovery Rate (FDR) procedure to determine significant causal edges.

Key Contributions

1. Novel Framework: Introduction of GCD, which utilizes a single MLP and leverages $\ell_1$ regularization on input-output gradients rather than network weights to infer causal relationships.
2. Efficiency: The framework eliminates the need for component-wise architectures, significantly reducing the number of trainable parameters and computational overhead.
3. Robust Statistical Testing: The development of the PSST module to more effectively handle periodic signals and multiple comparison issues compared to existing surrogate methods.

Experimental Results
The authors evaluated GCD against seven state-of-the-art baselines (including cMLP, cLSTM, TCDF, eSRU, GVAR, CR-VAE, and CUTS+) across synthetic and real-world datasets.

• Synthetic Benchmarks:
  • Lorenz-96: GCD achieved perfect or near-perfect AUROC and AUPRC scores (1.0 or 0.997) across varying forcing terms, outperforming or matching the best baselines (eSRU, CUTS+).
  • DREAM4: On gene regulatory networks with extremely sparse causal structures, GCD consistently achieved the highest AUROC and AUPRC across all five sub-datasets, demonstrating superior ability to detect sparse relationships.
  • CausalTime: GCD outperformed all baselines on the Traffic and Medical datasets and achieved competitive results on the AQI dataset. Notably, it was the only model to exceed 0.9 on both AUROC and AUPRC for the Medical dataset.
• Industrial Applications:
  • Tennessee-Eastman (TE), Ultra-processed Food (UF), and Debutanizer (DE): GCD demonstrated superior performance in Structural Hamming Distance (SHD), F1-score, and Precision compared to other models. In the UF process, where other models struggled with false positives or missed structures, GCD effectively captured hierarchical dependencies.
• Efficiency Analysis:
  • Complexity: GCD exhibits lower time complexity ($O(e(N+1)\sum H_l H_{l+1})$) compared to component-wise models ($O(eD \dots)$) when the number of variables $D$ exceeds the number of surrogate tests $N$.
  • Training Time: In all three industrial datasets, GCD required significantly fewer parameters and less total training time (e.g., ~72s on TE vs. ~114s for cMLP and ~323s for CR-VAE) while maintaining higher accuracy.

Significance and Claims
The paper claims that GCD offers a scalable, lightweight, and effective solution for causal discovery in complex industrial processes. By shifting from component-wise architectures to a single-model approach and utilizing gradient-based inference, GCD reduces computational costs without sacrificing accuracy. The authors assert that the framework is particularly well-suited for industrial systems with tens or hundreds of sensors, where traditional methods become computationally prohibitive. Furthermore, the integration of the PSST module enhances the reliability of causal discovery in noisy, real-world data. The results validate GCD as a practical tool for fault diagnosis, process optimization, and safety assurance in modern industrial analytics.