Rank-Based Sparse Regression in Principal Components Space under Measurement Error

1. Problem Statement

The paper addresses the challenge of high-dimensional sparse regression under two simultaneous complications:

1. Measurement Error in Predictors: The observed predictors ($Z$) are contaminated by additive noise ($W$), such that $Z = X + W$, where $X$ is the latent true design matrix.
2. Heavy-Tailed Response Errors: The response variable ($y$) may contain outliers or follow a heavy-tailed distribution, rendering standard least-squares methods unstable.

Context:

• Existing methods for measurement error (e.g., CoCoLasso, Matrix Uncertainty Selector) typically operate in the original coordinate system and rely on squared loss, making them sensitive to heavy-tailed response errors.
• Existing methods for robust regression (e.g., Rank Lasso) often ignore the specific geometric structure induced by measurement error in high dimensions.
• The Gap: Song and Zou (2026) demonstrated a "blessing-of-dimensionality" phenomenon in Principal Components (PC) space for measurement error using $\ell_1$-penalized least squares. However, their method fails under heavy-tailed response errors. This paper seeks to combine the robustness of rank-based loss with the contamination geometry of PC space.

2. Methodology

The authors propose a two-stage rank-based procedure (RPCR) operating in the empirical principal-components space.

A. Model Setup

• Latent Model: $y = U\gamma^* + \varepsilon$, where $U$ are the latent principal components of the true design $X$. The parameter of interest is $\theta^* = n^{-1/2}\gamma^*$, assumed to be weakly sparse.
• Observed Data: $Z = X + W$. The empirical PC basis $\hat{U}$ is derived from the SVD of $Z$.
• Goal: Estimate the mean response $\hat{y}$ robustly despite $W$ and heavy-tailed $\varepsilon$.

B. Stage 1: Rank-Lasso Pilot Estimator

Instead of minimizing the squared loss, the first stage minimizes a Wilcoxon-type rank loss with an $\ell_1$ penalty:
$$\hat{\theta}^{(0)}(\lambda_0) = \arg\min_{\theta} \left\{ Q_n(\theta; \hat{U}) + \lambda_0 \|\theta\|_1 \right\}$$
Where the rank loss is defined as:
$$Q_n(\theta; \hat{U}) = \frac{1}{n(n-1)} \sum_{i \neq j} |(y_i - y_j) - (\hat{u}_i - \hat{u}_j)^\top \theta|$$

• Tuning: The penalty $\lambda_0$ is calibrated via a simulation-based procedure (similar to Wang et al., 2020) to ensure it scales as $\sqrt{\log m / n}$.
• Purpose: To obtain a robust initial estimate that is resistant to heavy-tailed response errors while utilizing the PC geometry.

C. Stage 2: Adaptive Reweighting and Bias Reduction

To correct the shrinkage bias inherent in the $\ell_1$-penalized pilot, a second stage applies a local-linear approximation to a folded-concave penalty (e.g., SCAD or MCP).

1. Adaptive Weights: $w_j(\lambda) = p'_\lambda(|\hat{\theta}^{(0)}_j|)$.
2. Weighted Optimization:
   $$\hat{\theta}^{(1)}(\lambda) = \arg\min_{\theta} \left\{ Q_n(\theta; \hat{U}) + \sum_{j=1}^m w_j(\lambda) |\theta_j| \right\}$$
3. Model Selection: The tuning parameter $\lambda$ is selected using a High-dimensional BIC (HBIC) criterion applied to the refitted unpenalized model on the selected support.

3. Key Theoretical Contributions

The paper establishes non-asymptotic prediction bounds for the second-stage estimator.

A. Prediction Bound (Theorem 3.3)

The prediction error of the second-stage fitted mean $\hat{y}^{(1)}$ satisfies:
$$\frac{1}{n} \|\hat{y}^{(1)} - y^*\|_2^2 = O_P \left( \underbrace{\frac{|A|}{n} + R_q \tau_*^{2-q}}_{\text{Estimation Term}} + \underbrace{\frac{\|y^*\|_2^2}{n\kappa^2} \left( \frac{n}{p} + \frac{n^2}{p^2} + \frac{n\|X\|_2^2}{p^2} \right)}_{\text{Contamination Term}} \right)$$

• Estimation Term: Achieves the oracle rate ($|A|/n$) associated with the active principal components, significantly improving upon the first-stage rate.
• Contamination Term: Remains identical to the least-squares analysis of Song and Zou (2026). It depends on the eigengap ($\kappa$) and the measurement error level.
• Implication: The method successfully decouples the robustness to response errors from the geometric handling of predictor errors. It retains the "blessing-of-dimensionality" (error decreases as $p$ increases) while being robust to heavy tails.

B. Assumptions

• Heavy-tailed responses: No finite variance is required; only the existence of a density with specific local behavior (Assumption 3.2).
• Sub-Gaussian measurement errors: The noise $W$ is sub-Gaussian.
• Eigengap: Active eigenvalues must be separated from the rest by a gap $\kappa$.

4. Numerical Results

The authors compare their method (RPCR) against the Song & Zou (2026) least-squares PC method (L1PCR) and standard Lasso.

• Simulation Settings:
  • Two signal models (sparse in leading PCs vs. sparse in trailing PCs).
  • Three error regimes: No error, Independent Gaussian noise, Correlated Gaussian noise.
  • Three response distributions: Gaussian, Standardized $t_3$ (heavy-tailed), and Mixture Normal (extremely heavy-tailed).
• Findings:
  • Gaussian Errors: RPCR is competitive with L1PCR; L1PCR sometimes has a slight efficiency edge in clean or mildly contaminated settings.
  • Heavy-Tailed Errors: RPCR substantially outperforms L1PCR. The advantage widens as the dimension $p$ increases and the eigengap $\kappa$ grows.
  • Blessing of Dimensionality: Both methods exhibit decreasing prediction error as $p$ grows under contamination, confirming the phenomenon holds for rank-based loss.
  • Real Data (Scheetz et al., 2006): Applied to an eQTL study (predicting TRIM32 expression).
    • Residuals were non-Gaussian (Shapiro-Wilk $p < 0.01$).
    • Under moderate to high measurement error ($c \ge 0.2$), RPCR achieved the lowest prediction error, outperforming both L1PCR and standard Lasso.

5. Significance and Conclusion

• Robustness: The paper provides the first theoretical and empirical evidence that rank-based regression can be effectively integrated into principal components regression under measurement error.
• Theoretical Breakthrough: It proves that one can achieve oracle estimation rates for the signal while maintaining the contamination bounds derived from the PC geometry, even without assuming finite variance for the response.
• Practical Impact: The method offers a stable alternative for high-dimensional biological or financial data where predictors are noisy and responses are prone to outliers, situations where standard least-squares PC regression fails.

In summary, this work bridges the gap between robust statistics and high-dimensional errors-in-variables models, demonstrating that replacing squared loss with rank loss preserves the geometric benefits of principal components while drastically improving stability under heavy-tailed noise.