📊 statistics

Causal Discovery on Dependent Mixed Data with Applications to Gene Regulatory Network Inference

This paper proposes a de-correlation framework that integrates structural equation modeling with latent variable imputation and covariance estimation to enable accurate causal discovery from dependent mixed data, demonstrating superior performance in gene regulatory network inference compared to standard methods.

Original authors: Alex Chen, Qing Zhou

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alex Chen, Qing Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Untangling the Messy Web of Life

Imagine you are a detective trying to figure out who influenced whom in a crowded room. You have a list of people (variables) and you want to draw a map showing who talked to whom first (causal relationships).

Usually, detectives assume everyone in the room is a stranger who arrived independently. But in real life—like in a family reunion, a social network, or a biology lab—people often arrive in groups, talk to their friends, and influence each other before the detective even starts asking questions. This makes the "who influenced whom" map very hard to draw correctly.

This paper, by Alex Chen and Qing Zhou, introduces a new detective tool designed specifically for messy, crowded rooms where:

People are connected: They aren't independent strangers; they influence each other (dependent data).
People speak different languages: Some speak in numbers (continuous data, like height or gene expression levels), while others speak in categories (discrete data, like "on/off" or "low/medium/high").

The Problem: The "Echo Chamber" Effect

Most standard detective tools (causal discovery algorithms) fail in two ways when applied to modern data:

The Independence Assumption: They assume every data point (every person in the room) is independent. But in biology, cells from the same lineage are like siblings; they share traits not because one caused the other, but because they come from the same family tree. If you ignore this, you might think two siblings are influencing each other when they are just copying their parents.
The Mixed Data Problem: Real-world data is a mix. In a gene network, some genes are "switches" (on/off), while others are "dials" (volume levels). Standard tools struggle to handle a mix of switches and dials, especially when those switches and dials are all echoing each other.

The Solution: The "Noise-Canceling Headphones"

The authors propose a clever two-step strategy they call a "De-correlation Framework." Think of it as putting on noise-canceling headphones to hear the true signal.

Step 1: The "Ghost" Behind the Curtain (Latent Variables)

Imagine that for every person in the room, there is a hidden "Ghost" (a latent variable) that is actually doing the talking.

If a person is a "switch" (discrete), their ghost is a continuous number that got rounded off to become a switch.
If a person is a "dial" (continuous), their ghost is just the number itself.

The authors build a model to guess what these "Ghosts" are doing. They use a special math trick (an EM algorithm) to peek behind the curtain and estimate the hidden continuous values that created the messy mix of switches and dials we see.

Step 2: The "De-Clumping" (De-correlation)

Once they have the "Ghosts," they notice that the ghosts are still clumped together because the people are related.

The Analogy: Imagine a choir where everyone is singing slightly out of tune with their neighbor because they are standing too close. The sound is muddy.
The Fix: The authors use a mathematical "Cholesky factor" (think of it as a specialized filter or a noise-canceling algorithm) to separate the choir members. They mathematically "pull" the data apart so that the influence of the neighbors is removed.

Now, the "Ghosts" are singing independently. The "clumping" is gone.

Step 3: The Final Map

Now that the data is "clean" and the people are acting like independent strangers again, the authors can use standard, off-the-shelf detective tools to draw the causal map. Because the "noise" of the group influence has been removed, the map they draw is much more accurate.

The Real-World Test: The Cellular City

To prove their method works, they applied it to Single-Cell RNA sequencing data.

The Setting: Imagine a city of 859 cells (the "people") trying to decide how to grow into different body parts (like skin, heart, or brain).
The Goal: Figure out which genes (the "mayors") are telling other genes what to do to drive this growth.
The Challenge: Cells from the same family line are very similar (dependent), and gene data is a mix of "on/off" switches and "volume" levels.

The Results:
When they used their "Noise-Canceling" method, the resulting map of gene interactions was:

More Accurate: It predicted future cell behavior much better than standard methods.
Scientifically Valid: Many of the connections they found were already known to biologists (e.g., specific genes known to control stem cells).
Robust: Even when they shuffled the data around (bootstrapping), the most important connections stayed the same.

Why This Matters

In the past, if you tried to map cause-and-effect in complex biological systems (like how cancer develops or how a stem cell becomes a heart cell), you often got a blurry, wrong picture because you ignored the fact that cells are related and data is messy.

This paper provides a universal translator and a noise-canceling filter. It allows scientists to take messy, connected, mixed-type data, clean it up, and then use standard tools to find the true "cause-and-effect" relationships. It's like turning a chaotic, shouting crowd into a clear, orderly conversation so you can finally understand who is leading the discussion.

1. Problem Statement

Causal discovery aims to infer causal relationships among variables from observational data, typically represented by a Directed Acyclic Graph (DAG). Existing methods face two critical limitations in modern applications:

Violation of i.i.d. Assumption: Most standard algorithms assume observations (samples) are Independent and Identically Distributed (i.i.d.). However, in fields like social network analysis and genomics (specifically single-cell RNA sequencing), samples often exhibit dependence (e.g., cells from the same lineage or batch effects).
Mixed Data Types: Real-world datasets often contain a mixture of continuous and discrete variables. Modeling dependence across samples is particularly challenging when discrete variables are involved, as standard correlation-based de-correlation techniques are not directly applicable to discrete data.

The paper addresses the challenge of learning causal DAG structures from dependent mixed data (containing both continuous and discrete variables with correlated samples).

2. Methodology

The authors propose a de-correlation framework based on a Latent Variable Structural Equation Model (SEM). The methodology consists of three main phases:

A. Latent Variable Model Formulation

The authors generalize the SEM to handle mixed data:

Latent Process: Every observed variable $X_j$ (whether continuous or discrete) is generated from an underlying continuous latent variable $Z_j$ .
Discretization: Discrete variables are modeled via a quantization mapping $Q(\cdot)$ where $X_j = Q(Z_j; T_j)$ , determined by thresholds $T_j$ . Continuous variables are observed directly ( $X_j = Z_j$ ).
Dependence Structure: The latent variables follow a linear SEM:
$Z_j = \sum_{k \in PA_j} \beta_{kj} g(Z_k) + \varepsilon_j$
where $g(Z_k)$ is the latent variable itself (if continuous) or the discretized version (if discrete).
Correlated Noise: The error terms $\varepsilon_j$ follow a multivariate normal distribution $\mathcal{N}_n(0, \Sigma)$ , where $\Sigma$ captures the dependence between units (samples). This induces dependence in the observed data.

B. Pre-estimation Phase

Before de-correlation, the authors estimate model parameters:

Thresholds and Coefficients: Using a coordinate ascent algorithm, they estimate the thresholds ( $T$ ) and regression coefficients ( $\beta$ ) for discrete variables, initially assuming independent errors ( $\Sigma = I$ ).
Covariance Estimation: Using a pairwise maximum likelihood estimator (MLE), they estimate the off-diagonal elements of the covariance matrix $\Sigma$ . This approach integrates both discrete and continuous pairs to estimate the correlation structure between units, assuming a block-diagonal structure for $\Sigma$ to reflect natural groupings (e.g., cell clusters).

C. De-correlation and Structure Learning

The core innovation is transforming the dependent data into approximately independent data:

Latent Recovery (EM Algorithm): For discrete variables, the latent continuous values $Z_j$ are imputed using an Expectation-Maximization (EM) algorithm. The E-step samples from a multivariate truncated Gaussian distribution conditional on observed discrete outcomes, and the M-step updates regression coefficients.
De-correlation Transformation: Once the latent data $\hat{Z}$ is recovered, it is transformed using the Cholesky factor $L$ of the precision matrix $\Theta = \Sigma^{-1}$ :
$\tilde{Z} = L^\top \hat{Z}$
This transformation removes the sample-level dependence, resulting in a dataset where units are approximately independent.
Causal Discovery: Standard causal discovery algorithms (e.g., MMHC, PC, Copula-PC) are applied to the de-correlated continuous data $\tilde{Z}$ .
Aggregation: To handle variability from the EM sampling, the authors propose two strategies for the final graph:
- Consensus: Running the discovery algorithm on multiple de-correlated datasets and accepting edges that appear in a majority of runs.
- Average: Averaging the de-correlated datasets before running the discovery algorithm once.

3. Key Contributions

Unified Framework for Mixed Data: The paper extends latent variable SEMs to handle general mixtures of continuous and multi-level discrete variables under sample dependence, overcoming the limitations of previous methods that only handled binary data or purely continuous data.
De-correlation via Pairwise MLE: The development of a pairwise likelihood estimator for the covariance matrix $\Sigma$ allows for robust estimation of dependence structures without requiring a known network of units.
EM-based Imputation: A novel EM algorithm is developed to impute latent continuous variables for discrete observations while accounting for cross-sample correlations.
Application to scRNA-seq: The method is successfully applied to infer Gene Regulatory Networks (GRNs) from single-cell RNA sequencing data, explicitly modeling the non-i.i.d. nature of cells (lineage and batch effects).

4. Results

Simulation Studies

Covariance Estimation: The pairwise MLE approach accurately estimated the covariance matrix $\Sigma$ , with Root Mean Square Error (RMSE) decreasing as the number of variables increased.
De-correlation Effectiveness: After transformation, the correlation between units dropped significantly (from an average of 0.62 in raw data to 0.09 in de-correlated data).
DAG Recovery: Across various DAG learning algorithms (MMHC, PC, Copula-PC) and data settings (random and real benchmark DAGs), the proposed Consensus and Average approaches significantly outperformed the Baseline (applying standard methods directly to dependent mixed data).
- The F1-scores for causal graph recovery improved substantially, particularly in high-dimensional settings ( $p > n$ ) and when using Copula-PC.
- The Consensus approach generally yielded the most stable edge recovery.

Real Data Application (Gene Regulatory Networks)

Dataset: Applied to single-cell RNA-seq data from human embryonic stem cells (differentiating into various lineages).
Model Fit: Cross-validation showed that the proposed method (Consensus) achieved a significantly higher test-data log-likelihood compared to the Baseline and a "Consensus-Ident" (which ignored de-correlation). This confirms that modeling sample dependence improves predictive accuracy.
Biological Validity: The inferred network contained high-confidence edges supported by existing literature. Examples include:
- Interactions among pluripotency regulators (POU5F1, NANOG, ZFP42).
- Feedback loops (NODAL $\leftrightarrow$ LEFTY1).
- Known regulatory links (PRDM14 $\to$ NANOG).
Stability: A bootstrap analysis quantified edge confidence, demonstrating that the inferred network is robust to sampling variations.

5. Significance

This work provides a principled solution to a pervasive problem in modern data science: causal inference on dependent, mixed-type data.

Methodological Impact: It demonstrates that causal discovery algorithms do not need to be fundamentally redesigned for dependent data; instead, the dependence can be "washed out" via a pre-processing de-correlation step based on a latent variable model.
Biological Impact: In the context of single-cell genomics, the method offers a more accurate way to infer Gene Regulatory Networks by acknowledging that cells are not independent samples. This leads to better identification of regulatory mechanisms driving cell differentiation, which is crucial for understanding development and disease.
Future Directions: The authors note that while the current framework assumes linearity, it can be extended to non-linear SEMs, and it currently handles observational data but could be integrated with interventional data for improved identifiability.