Density Ratio-based Causal Discovery from Bivariate Continuous-Discrete Data

This paper proposes a Density Ratio-based Causal Discovery (DRCD) method that infers the causal direction between continuous and discrete variables by leveraging theoretical results on the monotonicity of density ratios and the non-genericity of location-shift families to distinguish between $X \to Y$ and $Y \to X$ models.

The Gist

Imagine you are a detective trying to solve a mystery: Who is causing whom?

You have two suspects:

1. Mr. Continuous (X): A variable that can be any number (like your height, temperature, or blood pressure).
2. Ms. Discrete (Y): A variable that only has specific categories (like "Sick" vs. "Healthy," or "Male" vs. "Female").

Usually, you can't tell who is the cause and who is the effect just by looking at a snapshot of data. Did the high blood pressure cause the heart attack, or did the heart attack cause the blood pressure to spike?

This paper introduces a new detective tool called DRCD (Density Ratio-based Causal Discovery) to solve this specific type of mystery. Here is how it works, explained simply.

The Core Idea: The "Shape-Shifting" Test

The researchers realized that the relationship between the cause and the effect leaves a unique "fingerprint" in the data. They focus on a specific mathematical tool called a Density Ratio.

Think of the Density Ratio as a magnifying glass that compares how likely Mr. Continuous is to have a certain value under two different conditions of Ms. Discrete.

• Condition A: Ms. Discrete is "Type 1" (e.g., Male).
• Condition B: Ms. Discrete is "Type 2" (e.g., Female).

The magnifying glass asks: "If I look at the data for Males, how does the shape of the curve change compared to Females?"

The Two Scenarios

The paper proves that the answer to this question depends entirely on who is the boss (the cause).

Scenario 1: Mr. Continuous is the Boss ($X \rightarrow Y$)

Imagine Mr. Continuous is a River, and Ms. Discrete is a Dam built across it.

• The river flows (Continuous).
• The dam decides whether to flood or not (Discrete) based on how high the water gets.
• The Fingerprint: If you look at the water levels before the dam decides to flood versus after it decides not to, the ratio of the water levels forms a perfectly smooth, one-way slope. It goes up (or down) steadily and never wiggles back.
• The Metaphor: It's like a ramp. Once you start walking up, you keep going up. You don't go up, then down, then up again.

Scenario 2: Ms. Discrete is the Boss ($Y \rightarrow X$)

Imagine Ms. Discrete is a Chef, and Mr. Continuous is the Soup.

• The Chef decides what kind of soup to make (Type 1: Spicy, Type 2: Sweet).
• The Chef then pours ingredients into the pot.
• The Fingerprint: If the Chef makes two different soups, the ingredients (the data) might look totally different. One soup might be thick and spicy, the other thin and sweet.
• The Metaphor: If you compare the "Spicy" soup to the "Sweet" soup, the ratio of ingredients doesn't follow a smooth ramp. It's wobbly and chaotic. It might go up, then down, then up again. It's a "non-monotonic" mess.

The "Special Case" (The Location Shift)

There is one tricky situation. Sometimes, if the Chef (Ms. Discrete) is very rigid, she might just add a little more salt to the soup without changing the recipe at all. The soup is just "shifted" to the right.

• The paper proves that if the Chef is this rigid (a "location-shift"), the data looks like a smooth ramp again.
• However, the authors argue that in the real world, nature is rarely this rigid. Usually, different causes create different shapes of effects, not just shifted versions. So, if you see a "wobbly" ratio, you can be almost 100% sure the Discrete variable is the cause.

How the New Detective Tool (DRCD) Works

The DRCD method follows a simple 4-step checklist:

1. Is there a relationship at all?
   • Check if the data for "Type 1" and "Type 2" are actually different. If they are identical, there is no cause-and-effect. Case closed.
2. Is it a "Shift" or a "Shape Change"?
   • Check if the two groups are just shifted versions of each other (like the rigid Chef). If yes, the Discrete variable is likely the cause.
3. Draw the "Ratio Ramp".
   • If it's not a simple shift, the tool calculates the "Density Ratio" (the magnifying glass comparison) and draws a line.
4. Is the line straight or wobbly?
   • Straight/Ramp-like? The Continuous variable is the cause ($X \rightarrow Y$).
   • Wobbly/Chaotic? The Discrete variable is the cause ($Y \rightarrow X$).

Why is this a Big Deal?

Previous detective tools had two major problems:

1. They were too rigid: They assumed that if a Discrete variable caused a Continuous one, the shapes had to be identical (just shifted). Real life is messier than that.
2. They were unfair: They tried to compare "apples to oranges" (Continuous vs. Discrete) using complex scoring systems that often failed.

DRCD is better because:

• It handles messy, real-world data where the shapes change.
• It doesn't try to compare scores; it just looks for the smoothness of the ratio. It's a much simpler, more reliable rule.

The Verdict

The authors tested this tool on fake data and real-world medical data (like heart disease and cholesterol).

• Result: DRCD solved the mystery correctly more often than any other existing method.
• Takeaway: If you have a continuous number and a category, and you want to know which causes which, check the "smoothness" of their relationship. A smooth ramp means the number is the boss; a wobbly mess means the category is the boss.

Technical Summary

1. Problem Statement

The paper addresses the fundamental challenge of inferring the causal direction between a continuous variable ($X$) and a discrete variable ($Y$) using only observational data.

• Context: Existing methods struggle with mixed data types. Constraint-based methods (e.g., PC, FCI) fail in bivariate settings due to a lack of conditioning variables. Functional causal model methods often rely on restrictive assumptions (e.g., that conditional distributions differ only by a location shift). Flexible score-based methods face difficulties in fairly comparing causal directions because continuous and discrete variables differ fundamentally in information content and scale, requiring ad-hoc normalization.
• Goal: To develop a theoretically grounded, identifiable method that determines whether $X \to Y$, $Y \to X$, or no causal relationship exists, without relying on restrictive distributional assumptions or cross-type score comparisons.

2. Methodology: DRCD (Density Ratio-based Causal Discovery)

The authors propose DRCD, a method that leverages the properties of the conditional density ratio $G_{c_s, c_t}(x) = \frac{p(x|Y=c_t)}{p(x|Y=c_s)}$. The method operates on three distinct causal models:

A. Theoretical Models

1. $X \to Y$ (Threshold Model): $Y$ is generated by a threshold mechanism on $X$ (e.g., $Y = \mathbb{I}[f(X) + N \ge 0]$).
   • Key Property: The density ratio $G_{c_s, c_t}(x)$ is monotonic.
   • Genericity: Under the principle of independent mechanisms, the conditional distributions $P(X|Y)$ generally do not form a location-shift family.
2. $Y \to X$ (Two Cases):
   • Case 1 (Location-Shift): $P(X|Y=c)$ differs only by a shift in mean (standard assumption in prior work).
   • Case 2 (Non-Location-Shift): $P(X|Y=c)$ are mixtures of generalized normal distributions with independently parameterized components (different shapes, scales, or variances).
   • Key Property: In Case 2, the density ratio is non-monotonic almost everywhere (except on a set of Lebesgue measure zero).
3. No Causal Relationship: $X$ and $Y$ are independent.
   • Key Property: The conditional distributions are identical, so the density ratio is constant (equal to 1).

B. The DRCD Algorithm

The algorithm proceeds in four steps:

1. Test for Causal Existence: Use a two-sample Kolmogorov-Smirnov (KS) test on conditional samples of $X$ given different values of $Y$. If distributions are identical, conclude "No Causation."
2. Test for Location-Shift: Center the conditional samples (subtract means) and apply a KS test. If the centered distributions are statistically indistinguishable, conclude $Y \to X$ (Location-Shift case).
3. Estimate Density Ratio: If Step 2 fails (distributions are not location-shifts), estimate the density ratio $G_{c_s, c_t}(x)$ using unconstrained Least-Squares Importance Fitting (uLSIF) within the overlapping support of the distributions.
4. Evaluate Monotonicity: Compute the Spearman rank correlation ($r_s$) between the input values and the estimated density ratio.
   • If $|r_s| > \rho$ (strong monotonicity): Conclude $X \to Y$.
   • Otherwise: Conclude $Y \to X$ (Non-location-shift case).

3. Key Contributions

1. Theoretical Identifiability:
   • Lemma 3: Proves that under $X \to Y$, the density ratio is strictly monotonic.
   • Lemma 2 & Corollary 1: Proves that under $Y \to X$ with non-location-shift conditionals, the density ratio is non-monotonic on a set of Lebesgue measure zero (i.e., monotonicity is a non-generic property for this direction).
   • Lemma 1 & Assumption 2: Shows that for $X \to Y$ to produce location-shift conditionals, the causal mechanism and input distribution must be precisely coordinated, which violates the Principle of Independent Mechanisms. Thus, location-shift conditionals are a signature of $Y \to X$.
2. Novel Methodology: DRCD avoids the "apples-to-oranges" comparison problem of score-based methods by testing a property (monotonicity) of the density ratio rather than comparing scores across different variable types.
3. Robustness: The method handles heterogeneous conditional distributions (different variances/shapes) under $Y \to X$, a limitation of previous functional causal models like LiM and MIC.

4. Experimental Results

The authors evaluated DRCD on synthetic and real-world datasets against state-of-the-art baselines (LiM, MIC, MANMs, CRACK, GSF).

• Synthetic Data:
  • DRCD achieved >80% accuracy across all four scenarios: No Causation, $X \to Y$, $Y \to X$ (Location-Shift), and $Y \to X$ (Non-Location-Shift).
  • Competitors failed significantly in the Non-Location-Shift scenario (e.g., LiM and MIC dropped to ~5-68% accuracy) because they assume location-shift conditionals for $Y \to X$.
• Real-World Data:
  • UCI Heart Disease Dataset: DRCD correctly identified causal directions in 3 out of 4 pairs, outperforming most baselines. Notably, when DRCD erred, it inferred "No Causation" rather than an incorrect direction.
  • Tübingen Cause-Effect Pairs: DRCD correctly identified 3 out of 4 mixed-type pairs, matching the best-performing baseline (CRACK) but with zero reversed inferences (CRACK had one).
• Sensitivity: The method is robust to hyperparameters ($\rho, \gamma$) and requires at least $n \approx 500$ observations for reliable performance.

5. Significance

• Theoretical Advancement: This work provides the first rigorous identifiability proof for bivariate causal discovery between continuous and discrete variables that does not rely on the restrictive assumption that conditionals must be location-shifts.
• Practical Utility: By handling heterogeneous variances and shapes in conditional distributions, DRCD is applicable to a wider range of real-world phenomena (e.g., biological markers, economic indicators) where the "effect" distribution changes shape based on the "cause."
• Methodological Shift: It introduces a principled alternative to score-based causal discovery for mixed data types, eliminating the need for arbitrary normalization schemes by focusing on the structural property of the density ratio.

In conclusion, DRCD establishes that monotonicity of the density ratio is a unique signature of $X \to Y$, while non-monotonicity or location-shift conditionals characterize $Y \to X$, enabling reliable causal discovery in mixed continuous-discrete settings.