An association measure for mixed-type variables

Technical Summary: An Association Measure for Mixed-Type Variables

Problem Statement
Quantifying the association between a real-valued variable ($X$) and a categorical variable ($Y$) is a fundamental challenge in data analysis, particularly as mixed-type data become prevalent in fields like genomics and social sciences. Existing methods for this "numerical–categorical" setting suffer from significant limitations:

• Parametric Assumptions: Methods like polyserial correlation rely on latent-normal models or linear/ordinal assumptions that may not hold.
• Arbitrary Encoding: Common workarounds involve encoding nominal categories as integers to apply numerical methods (e.g., Chatterjee's $\xi_n$) or discretizing numerical variables. These approaches introduce artificial orderings and spacings, leading to unstable results where the statistic fluctuates significantly based on the chosen coding scheme.
• Lack of Scale: Standard tests like the ANOVA $F$-test assess mean differences but do not provide a dependence measure on a fixed, interpretable scale (e.g., 0 to 1).

Methodology
The authors propose a new label-invariant population measure of association, denoted $\xi'$, and its corresponding sample estimator, $\xi'_n$, specifically designed for the mixed real-valued–categorical setting.

1. Population Measure ($\xi'$):
   The measure is defined based on the reduction in Gini impurity (or information gain) achieved by observing $X$. Let $g_j(x) = P(Y=j | X=x)$ be the conditional probability functions. The population measure is:
   $$\xi'(X, Y) = \frac{E\left[\sum_{j=1}^k g_j(X)^2\right] - \sum_{j=1}^k P(Y=j)^2}{1 - \sum_{j=1}^k P(Y=j)^2}$$
   This formulation aggregates the squared conditional probabilities (concentration of the conditional distribution) and normalizes them against the marginal concentration.

2. Sample Estimator ($\xi'_n$):
   The estimator is constructed nonparametrically without smoothing or tuning parameters:

   • Sort the sample $(X_i, Y_i)$ by $X$ to obtain $(X_{(1)}, Y_{(1)}), \dots, (X_{(n)}, Y_{(n)})$.
   • Calculate the adjacent-match proportion $A_n = \frac{1}{n-1}\sum_{i=1}^{n-1} \mathbb{I}(Y_{(i+1)} = Y_{(i)})$.
   • Calculate the marginal coincidence probability $B_n = \sum_{j=1}^k \hat{p}_j^2$, where $\hat{p}_j$ is the sample proportion of class $j$.
   • The estimator is $\xi'_n = \frac{A_n - B_n}{1 - B_n}$.
   • This estimator is computable in $O(n \log n)$ time due to the sorting step.

3. Theoretical Properties:

   • Normalization: $\xi' \in [0, 1]$.
   • Independence: $\xi' = 0$ if and only if $X$ and $Y$ are independent.
   • Functional Dependence: $\xi' = 1$ if and only if $Y$ is a measurable function of $X$ almost surely.
   • Invariance: The measure is invariant to strictly monotone transformations of $X$ and, crucially, to any permutation of the categorical labels of $Y$.
   • Connection to Classical Measures: $\xi'$ generalizes the coefficient of determination ($R^2$) using Gini impurity. It is structurally similar to Chatterjee's $\xi$ but aggregates class-specific variances without prevalence weighting, making it suitable for nominal (unordered) categories.

Asymptotic Theory and Inference
The paper establishes the strong consistency of $\xi'_n$ and derives its asymptotic normality under two regimes:

• Under Independence ($H_0$): $\sqrt{n}\xi'_n$ converges to a normal distribution with mean 0. The authors derive a consistent variance estimator, enabling a computationally efficient, permutation-free Wald test for independence. A finite-sample centering correction is proposed to address negative bias in small samples.
• Under General Dependence: The authors prove asymptotic normality for $\sqrt{n}(\xi'_n - \xi')$ under mild conditions (bounded variation of conditional probabilities). A plug-in variance estimator using $k$-nearest neighbors ($k$-NN) is provided, allowing for the construction of asymptotically valid confidence intervals for the population measure.

Key Results

• Simulation Studies: Extensive simulations using a block design model demonstrate that $\xi'_n$ achieves the nominal coverage probability for confidence intervals and maintains correct type I error rates for the Wald test.
• Coding Stability: Unlike Chatterjee's $\xi_n$, which fluctuates wildly under different integer encodings of the same nominal data, $\xi'_n$ remains constant regardless of label permutations.
• Power: The proposed method demonstrates competitive power against established alternatives like Distance Covariance (dCov), HSIC, and ANOVA, particularly in scenarios involving weak signals or minority class detection.
• Computational Efficiency: The $O(n \log n)$ complexity makes the method scalable, avoiding the $O(n^2)$ cost of kernel-based methods like HSIC.
• Real Data Application: An application to The Cancer Genome Atlas (TCGA) breast cancer data shows the method's ability to detect general dependencies, including variance heterogeneity, that might be missed by mean-based tests.

Significance and Claims
The paper claims to provide a robust, model-free solution for measuring association in mixed-type data. Its primary significance lies in:

1. Label Invariance: Solving the instability problem inherent in applying rank-based statistics to nominal categorical variables by aggregating class indicators directly rather than imposing an artificial order.
2. Interpretability: Offering a normalized measure (0 to 1) that quantifies the proportion of Gini impurity reduction (information gain), bridging the gap between statistical testing and predictive modeling concepts.
3. Efficient Inference: Enabling fast, permutation-free hypothesis testing and confidence interval construction via asymptotic theory, which is computationally advantageous for large-scale datasets compared to resampling-heavy alternatives.

The authors position $\xi'$ as a necessary extension of Chatterjee's work, tailored specifically for the challenges of nominal categorical responses where standard rank correlations fail due to their sensitivity to arbitrary coding.