Limiting laws and consistent estimation criteria for fixed and diverging number of spiked eigenvalues

1. Problem Statement

The paper addresses two fundamental challenges in high-dimensional Principal Component Analysis (PCA) and spiked covariance models:

1. Limiting Laws: Deriving the asymptotic behavior (limiting distributions) of the sample spiked eigenvalues ($d_1, \dots, d_k$) when both the dimension $p$ and the sample size $n$ tend to infinity, specifically allowing the number of spikes $k$ to diverge.
2. Consistent Estimation: Developing information criteria to consistently estimate the number of spikes $k$ in two regimes:
   • Fixed $p$: Where $n \to \infty$ but $p$ is fixed.
   • Diverging $p$: Where $n, p \to \infty$ with $p/n \to c > 0$.

Key Challenges in Existing Literature:

• Most existing results assume $k$ is fixed.
• Many require spiked eigenvalues to be uniformly bounded or tend to infinity.
• Existing criteria often rely on strict conditions (e.g., specific penalty terms like BIC) or assume a "standard" spiked model where bulk eigenvalues are identical.
• The paper aims to relax these assumptions to handle general spiked covariance models (where bulk eigenvalues differ) and allow $k$ to grow with $n$.

2. Methodology and Framework

Model Setup

The authors consider a sample $x_1, \dots, x_n$ from a $p$-dimensional distribution with mean 0 and covariance $\Sigma$.

• General Spiked Model: $\Sigma = \Gamma \Lambda \Gamma^\top$, where $\Lambda = \text{Diag}(\lambda_1, \dots, \lambda_k, \lambda_{k+1}, \dots, \lambda_p)$.
• Spikes: $\lambda_1 \ge \dots \ge \lambda_k > \lambda_{k+1} \ge \dots \ge \lambda_p$.
• Bulk: The bulk eigenvalues $\lambda_{k+1}, \dots, \lambda_p$ are allowed to differ (unlike the standard model where they are equal).
• Assumptions:
  • $n, p \to \infty$ with $c_n = (p-k)/n \to c > 0$.
  • The empirical spectral distribution of the bulk converges weakly to a limiting distribution $H$.
  • The "distant spike" condition: $\psi'(\lambda_k) > 0$, where $\psi(x) = x + c \int \frac{xt}{x-t} dH(t)$.

Theoretical Tools

The authors utilize Random Matrix Theory (RMT), specifically:

• Stieltjes Transforms: To analyze the empirical spectral distribution of the sample covariance matrix.
• Determinant Equations: Deriving the limit of spiked eigenvalues by solving $|dI - K_n(d)| = 0$, where $K_n(d)$ is a random matrix involving the bulk eigenvalues.
• Concentration Inequalities: Using matrix concentration bounds (e.g., Bickel & Levina, Weyl's inequality) to control the deviation of sample eigenvalues from population limits.

3. Key Contributions

A. Limiting Laws for Diverging $k$

The paper establishes the limiting laws for spiked sample eigenvalues $d_i$ ($1 \le i \le k$) under a general spiked model where $k$ can diverge.

• Consistency: They prove that $d_i$ is a consistent estimator for a transformed population parameter $\psi_n(\lambda_i)$.
• Convergence Rates:
  • For Gaussian data: $k = o(n^{1/3})$.
  • For General data (finite 4th moment): $k = o(n^{1/4})$.
  • Improvement: This relaxes previous bounds (e.g., Cai et al. (2020) required $k = o(n^{1/6})$).
• Asymptotic Normality: They derive the asymptotic normality of the spiked eigenvalues.
  • $\sqrt{n}(d_i - \psi_n(\lambda_i)) \xrightarrow{d} N(0, \sigma^2)$.
  • The variance depends on the kurtosis of the data and the eigenvector structure.
  • This result holds even when $k$ diverges, provided the growth rate conditions are met.

B. Consistent Estimation Criteria

The authors propose new information criteria to estimate $k$ that are consistent under weaker conditions than existing methods.

1. Fixed $p$ Regime:

   • They propose a General Information Criterion (GIC):
     $$\text{GIC}(k') = -\log \tilde{L}_{k'} + k'(p - k'/2 + 1/2)C_n$$
   • Theoretical Breakthrough: They prove consistency if $C_n \to \infty$ and $C_n/n \to 0$.
   • Significance: This relaxes the condition in Zhao et al. (1986) which required $C_n / \log \log n \to \infty$. This allows for weaker penalties (e.g., $C_n = \log \log n$), which are less prone to underestimation in finite samples compared to BIC ($C_n = \log n / 2$).

2. Diverging $p$ Regime (Standard Model):

   • They propose a General Information Criterion (GIC) adapted for $p \to \infty$:
     $$\text{GIC}(k') = \frac{n}{2} \left( \sum_{i=1}^{k'} \log d_i + (p-k') \log \bar{d}_{k'+1} \right) + \gamma k'(p - k'/2 + 1/2)$$
   • Consistency Condition: The penalty parameter $\gamma$ must satisfy $\phi(1+\sqrt{c}) < \gamma < \phi(\lambda_k)$, where $\phi$ is a function derived from the Stieltjes transform.
   • This allows for consistent estimation of $k$ (fixed or $o(n^{1/3})$) without requiring delocalization or sparsity conditions on eigenvectors.

3. Diverging $p$ Regime (General Model):

   • For the general model where bulk eigenvalues differ, standard likelihoods fail.
   • They propose an Adjusted GIC (AGIC) based on the sample correlation matrix and an adjusted thresholding procedure:
     $$\text{AGIC}(k') = -\frac{n}{2} \sum_{i=1}^{k'} \log \left( \frac{d_i}{1+\delta_i} \right) + \gamma k'(p - k'/2 - 1/2)$$
     where $\delta_i$ is a correction term derived from the bulk eigenvalues.
   • This estimator is consistent under general distributions with finite moments of order $>4$.

4. Key Results

• Theorem 1 & 2: Establish the limits and convergence rates ($O_p(\sqrt{k/n})$ or $O_p(k/\sqrt{n})$) for spiked eigenvalues under diverging $k$.
• Theorem 3: Proves asymptotic normality for spiked eigenvalues with diverging $k$.
• Theorem 4: Proves the consistency of GIC for fixed $p$ under the relaxed condition $C_n \to \infty, C_n/n \to 0$.
• Theorem 5 & 6: Prove the consistency of GIC and AGIC for diverging $p$ under general spiked models and general distributions.
• Simulation Studies:
  • Small $p$: The proposed ILP (Iterated Logarithm Penalty) and ILP$^{1/2}$ methods outperform BIC, AIC, and other existing methods, especially when the Signal-to-Noise Ratio (SNR) is low.
  • Large $p$: GIC and AGIC demonstrate superior performance compared to BCF, PC3, ON2, KN, DDPA, ACT, and BEMA, particularly in low SNR scenarios and when $p$ or $n$ is small.
  • Real Data: Applied to Fama-French portfolios, 1000 Genomes Project, and PBMC single-cell RNA-seq data. The proposed methods correctly identified the known number of factors/clusters (e.g., 3 factors in finance, 4 clusters in genetics, 6 cell types in biology) where other methods over- or under-estimated.

5. Significance

1. Relaxation of Assumptions: The work significantly broadens the applicability of spiked covariance analysis by allowing the number of spikes $k$ to diverge and removing the requirement for uniformly bounded or diverging spiked eigenvalues.
2. Theoretical Advancement: It provides the first asymptotic normality results for spiked eigenvalues when $k$ diverges, a crucial step for constructing confidence intervals and hypothesis tests in high dimensions.
3. Practical Utility: The proposed GIC and AGIC criteria offer a unified, theoretically grounded approach to selecting the number of components that is robust to low SNR and general data distributions.
4. Flexibility: The ability to handle general bulk eigenvalues makes the methodology applicable to a wider range of real-world datasets where the "noise" is not homogeneous.

In summary, this paper bridges the gap between theoretical random matrix theory and practical high-dimensional statistics, providing robust tools for dimension reduction and factor selection in modern, complex data environments.