Rank-based Maxsum test for high dimensional regression coefficient

1. Problem Statement

The paper addresses the challenge of global hypothesis testing for regression coefficients in high-dimensional linear models where the number of covariates ($p$) is comparable to or larger than the sample size ($n$). The specific goal is to test:
$$H_0: \beta = 0 \quad \text{vs} \quad H_1: \beta \neq 0$$

Key Challenges Identified:

• Sparsity-Density Trade-off: Existing methods generally fall into two categories:
  • Sum-type tests (e.g., quadratic forms) are powerful when signals are dense (many non-zero coefficients) but lose power when signals are sparse.
  • Max-type tests are powerful for sparse signals but perform poorly when signals are dense.
  • In practice, the sparsity level is often unknown, making it difficult to choose the optimal test.
• Heavy-Tailed Errors: Most existing high-dimensional tests rely on light-tailed error assumptions (Gaussian or sub-Gaussian). They often fail or lose power when errors are heavy-tailed or contaminated.
• Lack of Adaptive Robustness: There is a need for a procedure that is simultaneously robust to heavy-tailed errors and adaptive to unknown sparsity levels without requiring prior knowledge of the signal structure.

2. Methodology

The authors propose Rank-based Maxsum Tests that combine rank-based statistics with Cauchy combination methods.

A. Model and Statistics

The model is $Y_i = X_i^\top \beta + \varepsilon_i$. The authors utilize three core statistics derived from Wilcoxon scores ($e_i$), which are rank-based and distribution-free under $H_0$:

1. Rank-based Sum Statistic ($T_{RS}$): Based on Feng et al. (2013). It is a U-statistic tailored for dense alternatives and robust to heavy tails.
2. Marginal Max Statistic ($T_{RM1}$): Based on Xu and Zhou (2021). It is the maximum of studentized rank-score correlations, tailored for sparse alternatives.
3. Precision-Adjusted Max Statistic ($T_{RM2}$): Also from Xu and Zhou (2021). It incorporates an estimator of the precision matrix ($\Omega = \Sigma^{-1}$) to account for dependence among covariates, improving power in sparse settings with correlated predictors.

B. Theoretical Foundation: Asymptotic Independence

The core theoretical contribution is proving that under $H_0$, the rank-based sum statistic ($T_{RS}$) and the rank-based max statistics ($T_{RM1}$ and $T_{RM2}$) are asymptotically independent.

• Theorems 1 & 2: Under mild regularity conditions (independent component model for regressors, bounded covariance/precision matrices, and smooth error density with finite Fisher information), the joint distribution of the sum statistic and the max statistic converges to the product of their marginal limits:
  $$P(T_{RS} \le x, T_{RM} - 2\log p + \log \log p \le y) \to \Phi(x)F(y)$$
  where $\Phi$ is the standard normal CDF and $F$ is the Gumbel CDF.
• This independence holds even when $p$ grows exponentially with $n$ (specifically $\log p = o(n^{1/5})$).

C. Cauchy Combination

Leveraging the asymptotic independence, the authors propose two Maxsum tests using the Cauchy Combination Method (Liu and Xie, 2020).

• They transform the p-values of the sum and max statistics into Cauchy random variables:
  $$T_{C} = \frac{1}{2}\tan\left[\left(\frac{1}{2} - p_{RS}\right)\pi\right] + \frac{1}{2}\tan\left[\left(\frac{1}{2} - p_{RM}\right)\pi\right]$$
• Under $H_0$, the combined statistic $T_C$ converges to a standard Cauchy distribution $Cauchy(0,1)$.
• This allows for a simple rejection rule: reject $H_0$ if $T_C > c_\alpha$ (the upper $\alpha$-quantile of the Cauchy distribution).

3. Key Contributions

1. Theoretical Innovation: Established the asymptotic independence between rank-based sum-type and max-type statistics in high dimensions. This is a non-trivial result because rank statistics involve complex dependencies, and proving independence between a sum and a max of such statistics requires rigorous handling of the underlying U-statistics and extreme value theory.
2. Robustness: By utilizing Wilcoxon scores, the proposed tests are distribution-free under $H_0$ and do not require finite variance of the error term. They remain valid and powerful under heavy-tailed errors (e.g., t-distributions, log-normal, mixtures).
3. Adaptivity: The Cauchy combination creates a "Maxsum" test that automatically adapts to the unknown sparsity level. It inherits the power of the sum test for dense signals and the max test for sparse signals.
4. Practical Implementation: The method avoids the need for resampling (like permutation tests) or explicit estimation of the dependence structure between the component statistics, as the Cauchy combination relies only on marginal p-values and the theoretical independence result.

4. Simulation Results

The authors conducted extensive Monte Carlo simulations comparing their proposed tests ($RC1$ and $RC2$) against eight existing methods (including standard F-tests, sum-type, max-type, and adaptive combinations).

• Size Control: The proposed tests maintained accurate size control (Type I error) across various sample sizes ($n$), dimensions ($p$), and error distributions (Normal, $t_3$, Log-normal, Gaussian Mixture). They performed well even under heavy-tailed errors where least-squares-based methods (like EB or COM) showed inflated or conservative behavior.
• Power Performance:
  • Dense Signals: The rank-sum component ($RS$) and the proposed combinations ($RC1, RC2$) outperformed max-type tests.
  • Sparse Signals: The max-type components ($RM1, RM2$) and combinations outperformed sum-type tests.
  • Heavy Tails: The rank-based combinations ($RC1, RC2$) consistently maintained high power across all sparsity regimes and error distributions. In contrast, least-squares-based max tests suffered significant power loss under heavy-tailed errors.
  • Dependence: In scenarios with correlated covariates, the precision-adjusted combination ($RC2$) showed superior performance, particularly when the signal was sparse.

5. Significance

This paper provides a unified, robust, and adaptive framework for high-dimensional regression testing.

• Bridging the Gap: It solves the "sparsity dilemma" by combining dense and sparse detectors without needing to know the sparsity level beforehand.
• Handling Real-World Data: By removing the reliance on finite variance and Gaussian assumptions, the method is highly applicable to real-world data in genomics, finance, and bioinformatics, where heavy tails and outliers are common.
• Theoretical Rigor: The proof of asymptotic independence between rank-based sum and max statistics opens the door for similar combination strategies in other non-parametric high-dimensional inference problems.

In conclusion, the proposed Rank-based Maxsum tests represent a state-of-the-art solution for global inference in high-dimensional linear models, offering superior reliability and power compared to existing methods when faced with unknown sparsity and non-Gaussian noise.