Technical Summary: Learning General Causal Structures with Hidden Dynamic Process for Climate Analysis

Problem Statement

Understanding climate dynamics requires moving beyond correlational analysis to uncover underlying causal processes. Climate systems are characterized by two critical complexities often overlooked by existing methods:

1. Latent Drivers: Unobserved atmospheric processes (e.g., pressure systems, precipitation drivers) significantly influence observed variables (e.g., temperature) and evolve stochastically with both instantaneous and time-lagged dependencies.
2. Observable-to-Observable Causality: Geographically proximate observed variables exhibit direct causal influences (e.g., heat transfer between regions) alongside the influence of latent drivers.

Traditional Causal Representation Learning (CRL) focuses on recovering latent factors but typically assumes noise-free, invertible generation from latent to observed spaces, ignoring observable-to-observable causal relations. Conversely, Causal Discovery (CD) methods often rely on parametric assumptions or fail to identify latent variables and their interrelations in the presence of confounding. The paper addresses the need for a unified framework capable of jointly recovering (i) the latent dynamic process and (ii) the causal structure among observed variables in a nonparametric setting with stochastic noise.

Methodology: CaDRe

The authors propose CaDRe (Causal Discovery and Representation learning), a time-series generative model based on a state-space Variational AutoEncoder (VAE) that integrates CRL and causal discovery.

1. Theoretical Foundation

The framework establishes identifiability guarantees under specific assumptions:

• Latent Space Recovery (Theorem 3.2): By leveraging contextual information from adjacent time steps ($x_{t-1}, x_t, x_{t+1}$), the model proves that the latent space $z_t$ is identifiable at the value level (up to an invertible differentiable transformation $h_z$), even with noisy observations. This relies on the injectivity of distributional transformation operators and the "latent drift" assumption (distinct latent states induce distinct conditional distributions of observations).
• SEM-ICA Equivalence (Theorem 3.5): The paper establishes a functional equivalence between the Structural Equation Model (SEM) governing the climate system and Nonlinear Independent Component Analysis (ICA). Specifically, the Jacobian of the mixing structure ($J_m$) and the Jacobian of the observed causal graph ($J_g$) satisfy the relation:
  $$J_g(x_t)J_m(s_t) = J_m(s_t) - D_m(s_t)$$
  where $D_m$ is a diagonal matrix. This allows the causal graph over observations to be derived directly from the mixing structure.
• Causal Graph Identifiability (Theorem 3.6): Under conditions of generation variability (sufficient changes in the generation process) and functional faithfulness, the causal graph over observed variables is identifiable.

2. Estimation Framework

CaDRe instantiates these theories using a VAE architecture with the following components:

• Encoders: A $z$-encoder extracts latent variables ($\hat{z}_t$), and an $s$-encoder extracts non-stationary noise ($\hat{s}_t$) dependent on $z_t$.
• Decoder: Reconstructs observations $\hat{x}_t$ from $\hat{z}_t$ and $\hat{s}_t$.
• Prior Networks: Utilizes normalizing flows to model the transition of latent variables ($z_{t-1} \to z_t$) and the mixing of noise. Learnable inverse transition functions identify which latent variables causally influence others.
• Loss Function: The model optimizes an Evidence Lower Bound (ELBO) augmented with structural penalties:
  $$L_{ALL} = L_{ELBO} + \alpha L_s + \beta L_d$$
  • $L_s$: Sparsity penalty on the Jacobian matrices of latent transitions and observed mixing to enforce sparse causal structures.
  • $L_d$: DAG constraints (based on trace of matrix powers) to ensure acyclicity in both the observed causal graph and the instantaneous latent causal graph.

Key Contributions

1. Unified Framework: A novel approach that simultaneously recovers latent dynamic processes and causal graphs among observed variables, addressing a gap where previous CRL methods ignored observable causality and CD methods ignored latent confounders.
2. Nonparametric Identifiability: Theoretical proofs demonstrating that both the latent space and the causal structure are identifiable from time-series data without assuming noise-free generation or specific parametric forms (e.g., linear or additive noise).
3. CaDRe Algorithm: A practical implementation using flow-based priors and gradient-based structural penalties to learn independent components and causal dependencies.
4. Climate Insights: Application to real-world climate data yielding interpretable causal graphs aligned with domain expertise (e.g., wind patterns, land-atmosphere coupling).

Experimental Results

Synthetic Data

• Latent Recovery: CaDRe outperforms state-of-the-art CRL baselines (e.g., iCITRIS, G-CaRL, TDRL) in recovering latent representations, achieving high MCC and $R^2$ scores across independent, sparse, and dense latent settings.
• Causal Discovery: The method significantly outperforms constraint-based CD methods (FCI, PCMCI, CD-NOD) in recovering causal graphs, particularly in the presence of latent confounders and non-linear dependencies. It achieves lower Structural Hamming Distance (SHD) and higher Precision/Recall.

Real-World Climate Data

• Forecasting: On datasets like CESM2, Weather, and ERSST, CaDRe achieves competitive or superior forecasting accuracy (MSE/MAE) compared to advanced time-series models (e.g., iTransformer, TimesNet, Autoformer), suggesting that learning the underlying causal process improves predictive capability.
• Causal Discovery: Using wind direction as a surrogate for ground truth, CaDRe recovers causal graphs that align closely with physical wind patterns (measured by Wind-SHD and Wind-TPR), outperforming neural and constraint-based baselines.
• Interpretability: The recovered latent variables and causal graphs provide physical interpretations consistent with known phenomena (e.g., precipitation associations with solar radiation, westward flows in equatorial oceans).

Significance and Claims

The paper claims to advance the causal understanding of climate dynamics by providing a unified framework that connects latent driving forces with direct causal interactions among observed variables. Its significance lies in:

• Theoretical Rigor: Establishing nonparametric identifiability guarantees for jointly recovering latent dynamics and causal graphs, a setting previously considered intractable due to the "noise-free" assumption in CRL and the "fully observed" assumption in CD.
• Scientific Utility: Offering interpretable insights into climate systems that complement prevailing correlation-based deep learning pipelines. The recovered structures align with established physical knowledge (e.g., wind systems, land-atmosphere coupling), potentially inspiring new hypotheses in climate science.
• Practical Efficiency: The model achieves low latency through one-step generation, making it suitable for high-dimensional real-world settings.

The authors note limitations, specifically that the current framework captures contemporaneous edges rather than lagged ones among observed variables and has been evaluated primarily on climate data. Future work is suggested to extend the framework to other domains and incorporate lagged causal relations among observations.