Maximum-of-Differences Test for Comparing Multivariate K-Sample Distributions

1. Problem Statement

The paper addresses the fundamental statistical problem of comparing the distributions of $K$ samples in a multivariate setting. This problem arises in diverse fields such as bioinformatics, economics, finance, and machine learning (e.g., transfer learning, database matching, and gene set analysis).

Key Challenges:

• High Dimensionality: Existing methods often struggle when the dimension $p$ is large relative to the sample size $n$ (the "curse of dimensionality").
• Non-Parametric Nature: Many classical tests (like Hotelling's $T^2$) rely on parametric assumptions (e.g., multivariate normality) or focus only on mean/covariance equality, failing to detect general distributional differences.
• Regression Context: Most existing non-parametric K-sample tests do not account for covariates (explanatory variables), making them unsuitable for regression settings where one wishes to test the equality of error distributions across groups.
• Graph-Based Limitations: While graph-based methods (e.g., Minimum Spanning Trees) exist, they often face computational complexity or power issues in high dimensions or with $K > 2$ samples.

2. Methodology

The authors propose a novel Maximum-of-Differences (MOD) test and a Covariance-Adjusted (CA-MOD) variant. The methodology proceeds in three main steps:

A. Data Connection and Probability Estimation

1. Pooling: All observations from the $K$ samples are pooled into a single set.
2. Distance & Connection: Pairwise Euclidean distances ($\|X_i - X_j\|$) are calculated. Two observations are defined as "connected" if their distance is less than a pre-specified threshold $\tau$.
3. Within/Between Probabilities: For each observation $i$, the authors calculate two probabilities:
   • $\hat{p}_{in}^i$: The proportion of connected observations within the same sample as $i$.
   • $\hat{p}_{bet}^i$: The proportion of connected observations in other samples.
4. Difference Statistic: Under the null hypothesis ($H_0$: all $K$ distributions are equal), the probability of connection should be uniform regardless of the sample. Thus, the difference $\hat{p}_{bet}^i - \hat{p}_{in}^i$ should be close to zero.

B. The MOD Test Statistic

The test statistic $T_\tau$ is defined as the maximum of the standardized squared differences across all observations:
$$T_\tau = \max_{1 \le i \le n} \frac{(\hat{p}_{bet}^i - \hat{p}_{in}^i)^2}{\text{Var}(\hat{p}_{bet}^i - \hat{p}_{in}^i)}$$

• Asymptotic Distribution (Null): Under $H_0$, $T_\tau$ converges to the maximum of $K$ correlated weighted sums of standard normal and Type I extreme value distributions. Because this distribution lacks a closed form, critical values are estimated via a numerical replication method (generating multivariate normal vectors with the estimated covariance matrix).

C. The CA-MOD Test (Covariance-Adjusted)

To avoid the computational burden of the numerical replication method and the complex correlation structure of the raw statistics:

1. Covariance Profiling: The authors estimate the covariance matrix $\hat{\Sigma}$ of the standardized statistics.
2. Whitening: They transform the statistics using $\hat{\Sigma}^{-1/2}$ to decorrelate them.
3. CA-MOD Statistic: $T^{adj}_\tau = \max_{1 \le i \le n} (M^{adj}_{i,\tau})^2$, where $M^{adj}$ are the covariance-adjusted statistics.
4. Asymptotic Distribution (Null): Under mild conditions (positive definiteness of $\Sigma$), $T^{adj}_\tau$ converges to a Type I Extreme Value Distribution. This allows for the direct calculation of critical values without simulation.

D. Extension to Multivariate Regression

The framework is extended to test the equality of error distributions in multivariate regression models:
$$X_i^{(k)} = \beta^{(k)}W_i^{(k)} + \epsilon_i^{(k)}$$
The tests are applied to the residuals ($\hat{\epsilon}$) obtained from Ordinary Least Squares (OLS) estimation, allowing for the comparison of distributions after adjusting for covariates.

E. Tuning Parameter Selection

The threshold $\tau$ is crucial. The authors theoretically derive that $\tau$ should be chosen to maximize the test's power under specific alternatives. They recommend setting $\tau$ to the median of the pairwise distances $\|X_i - X_j\|$, which corresponds to the 50% quantile.

3. Key Contributions

1. Novel Test Statistic: Introduction of the MOD and CA-MOD tests, which utilize a "maximum-of-differences" approach based on pairwise connection probabilities rather than graph edges or mean vectors.
2. Theoretical Rigor:
   • Proved asymptotic consistency under both null and alternative hypotheses.
   • Established that CA-MOD converges to a Type I Extreme Value distribution, facilitating practical implementation.
   • Demonstrated applicability in both fixed and high-dimensional settings ($p$ can be larger than $n$).
3. Regression Extension: Successfully adapted the non-parametric K-sample test for multivariate regression models, a gap in existing literature.
4. Robustness: The tests rely on binary connection indicators ($I(\|X_i - X_j\| \le \tau)$), making them robust to heavy-tailed distributions and outliers, and not dependent on specific parametric assumptions.

4. Results

The authors conducted extensive simulation studies and a real-data analysis to validate the methods.

Simulation Studies

• Size Control: Both MOD and CA-MOD controlled the Type I error rate well (close to the nominal 0.05 level) across various sample sizes ($n$) and dimensions ($p$), including cases where $p > n$.
• Power Performance:
  • Mean Shift: MOD and CA-MOD performed comparably to or slightly worse than specialized graph-based tests (like GEC/WEC) for mean shifts but were competitive.
  • Covariance & Distribution Shifts: CA-MOD significantly outperformed all competing methods (including EC, GEC, WEC, GLP, and MMCM) in detecting covariance shifts and general distribution differences.
  • K-Sample ($K>2$): For $K=6$, CA-MOD and MOD were superior to the GLP and MMCM tests in detecting covariance and distribution shifts.
  • Regression Setting: In regression contexts, CA-MOD maintained superior power for detecting distributional differences in residuals compared to other methods.

Real Data Analysis

• Application: Tested for the "Day-of-the-Week Effect" in the Chinese stock market (comparing returns from Monday to Friday).
• Findings:
  • Without covariates: p-values were 0.199 (MOD) and 0.247 (CA-MOD).
  • With market index adjustment (Regression): p-values were 0.140 (MOD) and 0.151 (CA-MOD).
  • Conclusion: Neither test rejected the null hypothesis, suggesting no evidence of a day-of-the-week effect, supporting the efficiency of the Chinese stock market post-reform.

5. Significance

• Versatility: The proposed tests are unique in their ability to handle multivariate K-sample comparisons in both fixed and high dimensions and to extend seamlessly to multivariate regression models.
• Computational Efficiency: The CA-MOD test provides a closed-form asymptotic distribution (Type I Extreme Value), eliminating the need for computationally intensive resampling methods required by the raw MOD test.
• Practical Utility: The method is robust to outliers and heavy tails, making it suitable for real-world financial and biological data where normality assumptions often fail.
• Theoretical Advancement: It bridges the gap between graph-based non-parametric testing and high-dimensional statistical inference, offering a powerful tool for detecting complex distributional shifts beyond simple mean or variance changes.