🧬 biology

When Does Gene Regulatory Network Inference Break? A Controlled Diagnostic Study of Causal and Correlational Methods on Single-Cell Data

This study introduces a controlled diagnostic framework to isolate specific data pathologies, revealing that while causal methods for Gene Regulatory Network inference outperform correlation-based baselines in ideal conditions, their advantages are selectively neutralized by issues like dropout and latent confounders, offering nuanced guidance on when and why different methods succeed or fail.

Original authors: Miguel Fernandez-de-Retana, Ruben Sanchez-Corcuera, Unai Zulaika, Aritz Bilbao-Jayo, Aitor Almeida

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Miguel Fernandez-de-Retana, Ruben Sanchez-Corcuera, Unai Zulaika, Aritz Bilbao-Jayo, Aitor Almeida

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to draw a map of a bustling city. In this city, genes are the buildings, and regulatory networks are the roads connecting them. Some roads are one-way streets (a gene turns another on), and some are two-way (they influence each other).

For years, scientists have tried to use two different types of GPS to draw these maps from "single-cell" data (snapshots of individual cells).

The "Correlation" GPS: This looks at traffic patterns. If Building A and Building B always have cars coming and going at the same time, the GPS assumes they are connected. It's simple and fast.
The "Causal" GPS: This tries to understand the rules of the road. It asks, "Did Building A actually cause the traffic at Building B, or were they just both reacting to a third factor?" Theoretically, this should be much better at figuring out the true direction of the roads.

The Puzzle:
Despite the "Causal" GPS being theoretically smarter, real-world tests kept showing that the simple "Correlation" GPS was often just as good, or even better. This confused everyone. Why would the smart GPS fail?

The New Experiment:
The authors of this paper decided to stop guessing and start diagnosing. Instead of testing on messy, real-world cities where everything goes wrong at once, they built a perfectly controlled simulation lab.

Think of it like a video game where they can turn specific "glitches" on and off one by one. They created a digital city and introduced seven specific problems (which they call "pathologies") to see which GPS breaks first:

Dropout: The camera glitches, and 80% of the buildings disappear from the photo (common in real cell data).
Hidden Confounders: An invisible mayor is secretly controlling traffic in two different districts, making them look connected when they aren't.
Cell-Type Mixing: You accidentally mix a map of a hospital with a map of a school.
Feedback Loops: Roads that loop back on themselves (A leads to B, which leads back to A).
Network Density: The city is either a sparse village or a dense metropolis.
Sample Size: You only have photos of 200 buildings instead of 3,000.
Pseudotime Drift: The city is changing shape while you are trying to map it.

The Findings:

In a Perfect World: When the data is clean (no glitches), the Causal GPS (methods like NOTEARS and GES) is the undisputed champion. It draws the map almost perfectly, knowing exactly which roads go which way.
The "Dropout" Disaster: When the camera starts glitching and hiding buildings (the "Dropout" problem), the Causal GPS gets confused. However, the Correlation GPS is surprisingly tough. It doesn't get the direction right, but it still finds some connections. In fact, when the glitch is severe, the simple Correlation GPS actually wins because the complex Causal GPS tries to overthink the missing data and fails.
The "Hidden Mayor" Problem: When there are invisible factors controlling things (Latent Confounders), both GPS systems fail equally. No amount of math can solve a mystery if the evidence is hidden.
The "Direction" Issue: The Correlation GPS is great at finding that two buildings are connected, but it's terrible at knowing which way the road goes. The Causal GPS is excellent at direction, but only if the data is clean.

The "Error" Breakdown:
The authors also looked at how the maps were wrong.

The Causal GPS tends to be "confidently wrong." It draws a road that doesn't exist but is sure it's the right direction.
The Correlation GPS tends to be "missing the point." It misses roads entirely or draws them in the wrong direction because it can't tell cause from effect.

The Big Takeaway:
The reason simple methods often beat complex ones in real life isn't because the complex methods are bad. It's because real data is "dirty" (full of glitches like missing data).

If your data is clean: Use the smart, complex Causal methods. They are the best.
If your data is full of missing values (Dropout): The smart methods break. You should use the simple Correlation method or clean the data first.
If you have hidden factors: Neither method works well; you need different types of experiments (like interventions) to solve it.

In Summary:
The paper doesn't say "Causal inference is useless." It says, "Causal inference is a high-performance sports car. It wins every race on a clean track. But if you drive it through a mud pit (missing data) or a fog bank (hidden factors), it breaks down, and a sturdy old truck (correlation) might actually get you there."

The goal now is to know exactly which "road condition" you are driving on so you can pick the right vehicle.

Technical Summary: When Does Gene Regulatory Network Inference Break?

Problem Statement

Inferring Gene Regulatory Networks (GRNs) from single-cell RNA-seq data is a central challenge in computational biology. While causal inference methods (e.g., PC, GES, NOTEARS) theoretically offer advantages over correlation-based baselines by distinguishing direct regulatory interactions from indirect associations, empirical benchmarks on real and semi-real data consistently show that simple correlation-based methods (e.g., Pearson correlation, GENIE3) often match or outperform them. This discrepancy has led to skepticism regarding the utility of causal methods for GRN inference.

The authors argue that existing benchmarks are insufficient for diagnosing this failure because they evaluate on real data where multiple "pathologies" (biological or technical challenges) co-occur. These include dropout artifacts, latent confounders, cell-type heterogeneity, and non-stationarity. Because these factors are confounded with each other and the unknown ground truth, it is impossible to attribute performance gaps to specific causes. Consequently, current benchmarks cannot identify the specific conditions under which different inference methods succeed or fail.

Methodology

To address this, the authors introduce a controlled diagnostic framework inspired by ablation studies in machine learning. They construct a synthetic simulator based on a known Structural Causal Model (SCM) that generates single-cell expression data with seven independently controllable "pathology dials."

Data Generating Process

The simulator uses a linear additive-noise SCM (repeated under a nonlinear tanh SCM in the appendix to verify robustness).

Ground Truth: Directed Acyclic Graphs (DAGs) are sampled with controlled density and parent selection bias.
Pathology Dials: The framework isolates seven specific challenges:
1. Dropout ( $\delta$ ): Zero-inflation of lowly expressed genes.
2. Latent Confounders ( $k$ ): Unobserved Gaussian factors driving gene groups.
3. Cell-Type Mixing ( $\alpha$ ): Mixture of cells from two distinct SCMs.
4. Feedback Loops ( $\phi$ ): Addition of back-edges to create cycles.
5. Network Density ( $\rho$ ): Variation in the sparsity of the ground-truth graph.
6. Sample Size ( $n$ ): Variation in the number of observed cells.
7. Pseudotime Drift ( $\tau$ ): Non-stationarity along a trajectory.

Evaluated Methods

Six representative methods spanning three inference paradigms are tested:

Correlational: Pearson correlation, Mutual Information (MI).
Tree-Ensemble: GENIE3 (Random Forest).
Causal: PC (Constraint-based), GES (Score-based), NOTEARS (Optimization-based).

Evaluation Metrics

AUPRC (Area Under the Precision-Recall Curve): Computed for both undirected (edge recovery) and directed (orientation recovery) performance.
Error-Type Decomposition: A novel metric classifying predicted edges into five categories: True, Reversed, Confounded, Spurious, and Missed. This reveals qualitative differences in failure modes that aggregate AUPRC masks.

Key Contributions and Results

The study performs 6,120 controlled experiments (7 pathologies $\times$ 5 levels $\times$ 10 replicates $\times$ 6 methods) to map the degradation of performance as pathologies intensify.

1. Pathology-Specific Failure Modes

Dropout: This is the most discriminating pathology. It selectively neutralizes the advantages of causal methods.
- MI and GENIE3 collapse catastrophically ( $\Delta$ AUPRC $\approx -0.7$ ) because discretization and tree-splitting become unreliable with heavy zero-inflation.
- Pearson correlation degrades gracefully ( $\Delta$ AUPRC $\approx -0.28$ ) and becomes the nominal best performer under heavy dropout, outperforming all causal methods.
- Causal methods (NOTEARS, GES) remain robust but lose their structural advantage, converging to similar performance levels as Pearson.
Latent Confounders & Cell-Type Mixing: These act as "great equalizers." They degrade all method classes at comparable rates, compressing performance into narrow bands. No method class is inherently more robust to these violations of causal sufficiency.
Structural Pathologies (Density, Feedback, Pseudotime): Causal methods, particularly NOTEARS, dominate in these regimes. NOTEARS maintains high AUPRC (>0.9) even under high density and feedback loops, whereas correlational methods suffer significant drops. This is attributed to NOTEARS's $\ell_1$ sparsity penalty and acyclicity constraints.

2. Error-Type Analysis

Aggregate accuracy metrics hide distinct failure modes:

Causal Methods: When they fail, they often produce confounded or spurious edges with high confidence (hallucinations). However, they maintain low reversal rates (<16%), preserving directional reliability.
Correlational/Tree Methods: They are more prone to reversals (predicting $j \to i$ instead of $i \to j$ ) and missed edges. Under dropout, tree-based methods produce near-empty effective prediction sets.

3. Interaction Effects

An interaction sweep over dropout, latent confounders, and network density reveals:

Sub-additivity: The joint degradation of multiple pathologies is less severe than the sum of their individual effects.
Density-Conditional Cross-Overs: In high-density, high-dropout regimes, GES displaces Pearson as the best performer, a crossover invisible in single-dial analysis.

Significance and Claims

The paper claims to provide a nuanced understanding of when and why GRN inference methods succeed or fail, moving beyond simple benchmark rankings.

Resolution of the "Correlation vs. Causality" Puzzle: The observation that correlation beats causality on real data is not a fundamental failure of causal theory but a consequence of specific pathologies (primarily dropout and latent confounders) that selectively neutralize structural advantages.
Actionable Guidance: The authors offer practical recommendations:
- Use Pearson correlation (or imputation) under heavy dropout.
- Use interventional data to address latent confounding, as observational methods cannot distinguish direct effects from confounded associations.
- Prefer NOTEARS for sparse networks with adequate sample size and low dropout, as it offers the best accuracy-per-compute tradeoff.
- Always examine error decomposition, not just aggregate AUPRC, as different methods fail in qualitatively different ways with distinct biological implications.

The study concludes that benchmark leaderboards should be interpreted through the lens of pathology-specific failure modes rather than overall method labels. It provides a roadmap for future method development, highlighting that latent confounding remains an essentially unsolved problem for observational data alone.