Technical Summary: Robust Classification of High-Dimensional Data using Data-Adaptive Energy Distance

Problem Statement

The classification of High-Dimensional Low Sample Size (HDLSS) data presents significant challenges in fields such as gene expression studies, medical imaging, and chemometrics. Traditional classifiers, including Support Vector Machines (SVM), logistic regression, and k-Nearest Neighbors (k-NN), often fail in HDLSS regimes due to the "curse of dimensionality." Specifically, distance concentration phenomena render Euclidean distance-based metrics ill-defined, causing k-NN to exhibit erratic behavior. Furthermore, many existing methods lack robustness to outliers and rely on moment conditions (e.g., existence of fourth moments) that may not hold for heavy-tailed distributions. While dimension reduction techniques are frequently employed as pre-processing steps, they risk losing critical information if features are weakly dependent or if the reduction is not aligned with class separation goals.

Methodology

The authors propose a suite of classifiers based on a data-adaptive energy distance, denoted as $W^*_{FG}$, which measures the distance between two multivariate distributions $F$ and $G$.

1. Initial Classifier ($\delta_0$)

The authors first develop a classifier $\delta_0$ based on the original data-adaptive energy distance $W^*_{FG}$. This metric utilizes angular distances between projections of data points onto a unit sphere, weighted by a mixture of the two class distributions.

• Mechanism: The classifier assigns a test point $z$ to the class that minimizes a specific discrepancy function derived from $W^*_{FG}$.
• Limitations: Theoretical analysis (Theorem 2.2) shows that $\delta_0$ achieves asymptotically perfect classification (misclassification rate $\to 0$ as dimension $d \to \infty$) only if the distributions differ in their location (mean) or scale (variance). Crucially, this approach requires the existence of fourth moments of the underlying distributions (Assumption A1), limiting its robustness.

2. Refined Classifiers ($\delta_1, \delta_2, \delta_3$)

To overcome the moment-dependency and improve performance when location/scale differences are subtle or non-existent, the authors refine the distance measure by averaging distances across individual marginal dimensions. This leads to three robust classifiers:

• $\delta_1$ (Based on Average Energy Distance): Utilizes the average energy distance across marginals, $W^*_{FG} = \frac{1}{d}\sum W^*_{F_k G_k}$. It relies on the assumption that the average separation between classes does not vanish as $d \to \infty$ (Assumption A5).
• $\delta_2$ (Based on Squared Disparity): Introduces a new disparity measure $\bar{\tau}_{FG} = (T_{FG} - T_{FF})^2 + (T_{FG} - T_{GG})^2$, where $T$ terms represent expected angular distances. This squares the components to prevent cancellation effects when $T_{FG}$ lies between $T_{FF}$ and $T_{GG}$.
• $\delta_3$ (Based on Absolute Disparity): Proposes a measure $\bar{\psi}_{FG}$ using absolute values instead of squares: $\bar{\psi}_{FG} = \frac{1}{2}W^*_{FG} + \frac{1}{2}|T_{FF} - T_{GG}|$. This is designed to be the most "stable" among the proposed methods.

Key Theoretical Properties:

• Moment-Free: Unlike $\delta_0$, the refined classifiers $\delta_1, \delta_2,$ and $\delta_3$ do not require the existence of any moments of the underlying distributions.
• Asymptotic Consistency: Under fairly general assumptions regarding weak dependence among variables (Assumption A4), these classifiers yield perfect classification (misclassification probability $\to 0$) in the HDLSS asymptotic regime.
• Parameter-Free: The methods are free of tuning parameters.

Key Contributions

1. Development of Robust Classifiers: The paper introduces three new classifiers ($\delta_1, \delta_2, \delta_3$) specifically designed for HDLSS data that are robust to outliers and do not rely on moment conditions.
2. Theoretical Guarantees: The authors provide rigorous proofs demonstrating that these classifiers achieve asymptotically zero misclassification rates under general conditions, provided the distributions are not identical in all marginals.
3. Comparative Analysis: The paper establishes theoretical ordering conditions for the misclassification rates of the proposed classifiers based on the relative magnitudes of the underlying distance metrics ($T_{FF}, T_{GG}, T_{FG}$).
4. Empirical Validation: Extensive simulations and real-world data analyses confirm that the proposed methods outperform widely recognized classifiers (GLMNET, SVM, k-NN, Neural Networks) in HDLSS settings, particularly in the presence of heavy-tailed distributions and outliers.

Results

Simulation Studies

The authors tested the classifiers on five simulated examples involving location shifts, scale shifts, Cauchy distributions (heavy-tailed), and contaminated mixtures.

• Performance: As the dimension $d$ increased (from 5 to 1000), the misclassification rates of $\delta_1, \delta_2,$ and $\delta_3$ consistently approached zero.
• Robustness: In scenarios with identical first and second moments (where $\delta_0$ failed) or heavy-tailed distributions (Cauchy), the proposed classifiers significantly outperformed traditional methods like SVM and k-NN, which often hovered near random guessing (45-50% error).
• Outlier Resistance: In a contaminated mixture example (Example 5), the proposed classifiers maintained high accuracy as $d$ increased, whereas popular classifiers struggled.

Real Data Analysis

The methods were applied to six real-world HDLSS datasets, including microarray gene expression data (GSE1577, GSE89, Golub-1999-v2, Gordon-2002) and time-series data (Computers, DodgerLoopDay).

• Superiority: In the majority of datasets, the proposed classifiers achieved lower misclassification rates than GLMNET, SVM (linear and RBF), Neural Networks, and 1-NN.
• Stability: The ordering of performance among $\delta_1, \delta_2,$ and $\delta_3$ in real data aligned with the theoretical predictions derived in Theorem 3.4, validating the relationship between the empirical distance metrics and classifier efficacy.

Significance and Claims

The paper claims that its primary contribution lies in providing tuning-parameter-free, robust classification tools for HDLSS data that are theoretically guaranteed to work under general conditions without requiring moment assumptions. The authors emphasize that their approach avoids the information loss associated with dimension reduction.

The significance of the work is framed around its ability to handle "difficult" HDLSS scenarios where traditional distance-based methods fail due to distance concentration and where moment-based methods fail due to heavy tails. The authors modestly note that their theoretical guarantees hold when the competing distributions differ in a non-negligible number of marginals; if the distributions differ in only $o(d)$ marginals, the assumptions may not hold, representing a boundary for the method's applicability.

The work is presented as a step toward more reliable statistical learning in high-dimensional regimes, supported by both theoretical proofs and empirical evidence across diverse datasets.