📊 statistics

Adaptable Regularized CCA Tests for Independence of High-Dimensional Random Vectors

This paper proposes an adaptable testing procedure for assessing the independence of high-dimensional random vectors by integrating ridge regularization and principal component-based dimension reduction into the canonical correlation analysis framework, establishing asymptotic properties, and providing a data-driven method for parameter selection.

Original authors: Haoran Li

Published 2026-07-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoran Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Do two huge groups of clues, let's call them Group X and Group Y, actually talk to each other? Or are they just two strangers passing in the night, completely independent?

In the old days, when these groups were small (like a few dozen clues), detectives had a standard magnifying glass called Canonical Correlation Analysis (CCA). It worked great. But in the modern world, these groups have exploded in size. Now, Group X and Group Y might have hundreds or even thousands of clues each, and sometimes the number of clues is bigger than the number of cases you have to investigate (the sample size, $n$ ).

When you try to use the old magnifying glass on these giant groups, it breaks. The math gets "singular," which is a fancy way of saying the tool jams because there are too many variables and not enough data to hold them all together. It's like trying to solve a puzzle where you have more pieces than the box has pictures of; the pieces just don't fit, and the math crashes.

The Big Idea: A New, Flexible Tool
The authors of this paper, led by Haoran Li, built a new, super-adaptable tool to fix this jam. They combined two clever tricks:

Ridge Regularization: Think of this as adding a little bit of "glue" or "shock absorber" to the math. It stops the tool from shaking apart when the data gets messy or the groups get too big.
Principal Component Reduction: Instead of trying to look at every single clue in Group Y, they decided to focus only on the "top players." Imagine Group Y is a choir of 1,000 singers. Most of them are just humming quietly in the background. The authors say, "Let's only listen to the top 10 or 20 singers who are actually carrying the tune." These are the Principal Components (PCs).

By focusing on these top singers and adding the "glue," they created a stable way to test if Group X and Group Y are connected, even when the groups are massive.

Two Different Ways to Listen
The cool part is that this new tool has two different modes, depending on how many "top singers" (the reduced dimension, $k$ ) you decide to listen to:

Mode 1: The "All-Hands" Approach (Trace-Based Test)
If you only listen to a small number of top singers (say, $k$ is small, like less than 20), the tool sums up the energy from all of them. It's like taking a vote from the whole choir. The authors found that when $k$ is small, this method behaves very predictably, following a standard "bell curve" (Normal distribution). It's great for catching connections that are spread out across many clues.
Mode 2: The "Star Power" Approach (Largest-Root Test)
If you decide to listen to a larger chunk of the choir (where $k$ grows big as the sample size grows), the tool changes tactics. Instead of listening to everyone, it focuses entirely on the loudest single voice (the largest eigenvalue). This is powerful if the connection between the groups is driven by just one or two dominant factors. In this mode, the math follows a very specific, rare pattern called the Tracy-Widom law (named after two mathematicians, not a candy bar).

What They Proved and What They Simulated
The authors didn't just guess this would work; they did the heavy math lifting to prove it.

The Theory: They proved mathematically that if the groups are truly independent, their new tools will behave exactly as predicted (following the bell curve or Tracy-Widom law) as the data gets huge.
The Simulations: Since real-world data is messy, they ran thousands of computer simulations to see how the tools performed with smaller, realistic sample sizes (like $n=200$ $n = 200$ or $n=400$ $n = 400$ with dimensions $p_1, p_2$ $p_{1}, p_{2}$ up to 200).
- They tested different "flavors" of data: normal bell curves, heavy-tailed distributions (like a $t$ -distribution with 6 degrees of freedom), and even Poisson distributions.
- They found that the Trace-Based Test (Mode 1) is the superstar when the connection is spread out. It caught the signal better than older methods in almost every scenario they simulated.
- The Largest-Root Test (Mode 2) was slightly less powerful when the signal was spread out, but it was the only reliable choice when they needed to look at a large number of principal components ( $k$ ).

What They Argued Against
The paper explicitly argues against using the old, unregularized methods when the dimensions are high.

They showed that if you try to use the classic "Roy's largest root" test without the new "glue" (regularization) when the dimensions are close to the sample size, the test becomes unstable or breaks entirely.
They also compared their method to a previous "regularized" method by Yang and Pan (2015). They found that while Yang and Pan's method works when Group Y is smaller than the sample size, it fails when Group Y is huge (larger than $n$ ). The authors' new method, by focusing on the top principal components first, stays strong even when Group Y is massive.

The "Magic" Number: Choosing $k$ and $\lambda$
One of the hardest parts of using these tools is picking the right settings:

$k$ (How many singers?): The authors suggest a data-driven way to pick this. You start small and keep adding singers until the "noise" in the background stops changing much. They recommend checking until the change in the total energy is less than 5% of the total.
$\lambda$ (How much glue?): They developed a smart, data-driven way to pick the amount of "glue" (the regularization parameter) that maximizes the chance of catching a connection. They use a "minimax" strategy, which basically means picking the glue amount that works best even in the worst-case scenario.

The Verdict
In their simulations, the new method kept the "false alarm" rate (Type-I error) very close to the target 5% level, which is exactly what a good detective tool should do.

When the connection was spread out (like many small whispers), the Trace-Based Test was the most powerful.
When the connection was concentrated (like one loud shout), both tests worked, but the Trace-Based Test still held its own.
Most importantly, the new method worked where the old ones failed: when the number of variables ( $p_1, p_2$ ) was comparable to or even larger than the number of samples ( $n$ ).

The authors suggest that this approach—mixing "glue" with "focusing on the top players"—is a game-changer for high-dimensional statistics. They believe this same idea could help solve other tough puzzles in the future, like analyzing complex networks or financial markets, but for now, they have firmly established that it works for testing independence between two giant groups of variables.

Technical Summary: Adaptable Regularized CCA Tests for Independence of High-Dimensional Random Vectors

Problem Statement
The paper addresses the fundamental statistical problem of testing the independence between two high-dimensional random vectors, $X \in \mathbb{R}^{p_1}$ and $Y \in \mathbb{R}^{p_2}$ , based on an i.i.d. sample of size $n$ . The setting is characterized by the "high-dimensional regime" where the dimensions $p_1$ and $p_2$ are comparable to, or may even exceed, the sample size $n$ .

In classical multivariate analysis, independence testing relies on Canonical Correlation Analysis (CCA), specifically examining the eigenvalues of the normalized cross-covariance matrix. However, traditional test statistics (such as the Likelihood Ratio Test and Roy's Largest Root test) become ill-defined or unstable in high-dimensional settings. Specifically, when $n < p_1 + p_2$ , the necessary sample covariance matrices become singular or near-singular, rendering the inversion operations required for classical tests impossible or numerically unstable. Existing high-dimensional extensions often fail when both dimensions exceed the sample size or rely on specific structural assumptions that limit their applicability.

Methodology
The authors propose a flexible testing framework that integrates ridge regularization with principal component (PC)-based dimension reduction into the CCA framework. The core methodology proceeds as follows:

Model Assumptions: The authors assume that any dependence between $X$ and $Y$ is primarily captured through the leading $m$ principal components of $Y$ . The residual components of $Y$ are assumed to be approximately independent of $X$ . This leads to an asymmetric model where $X$ depends linearly on the projection of $Y$ onto its first $m$ PCs.
Dimension Reduction: Instead of projecting $X$ onto the full row space of $Y$ (which causes singularity issues when $p_2 > n$ ), the method projects $X$ onto the subspace spanned by the first $k$ sample principal components of $Y$ , where $k \le p_2$ is a reduced dimension parameter.
Regularized F-Matrix: The authors construct a modified $F$ $F$ -matrix, $F_{k\lambda} = W_{k1}(W_{k2} + \lambda I_{p_1})^{-1}$ $F_{k λ} = W_{k 1} (W_{k 2} + λ I_{p_{1}})^{- 1}$ , where:
- $W_{k1}$ and $W_{k2}$ are cross-product matrices derived from the projection of $X$ onto the $k$ -dimensional PC subspace and its orthogonal complement, respectively.
- $\lambda > 0$ is a ridge regularization parameter added to $W_{k2}$ to ensure invertibility and stability when $p_1 \gtrsim n - 1 - k$ .
Dual Testing Procedures: The choice of test statistic depends on the magnitude of the reduced dimension $k$ $k$ :
- Small $k$ Regime: A regularized trace-based statistic $T(k, \lambda) = \sum_{j=1}^k \ell_j(F_{k\lambda})$ is used. This aggregates information across all $k$ eigenvalues.
- Large $k$ Regime: A regularized largest-root statistic $\ell_{\max}(k, \lambda) = \ell_{\max}(F_{k\lambda})$ is used, relying on the largest eigenvalue.
Parameter Selection: The paper develops a data-driven procedure for selecting the regularization parameter $\lambda$ based on a Bayesian-minimax principle. This approach maximizes the asymptotic signal-to-noise ratio (SNR) over a class of prior distributions for the signal direction, avoiding the need to estimate the unknown signal vector directly.

Key Contributions

Methodological Extension: The paper extends classical CCA tests to regimes where $p_1, p_2 \ge n$ by combining ridge regularization with PC-based dimension reduction. This ensures the test statistics are well-defined and stable regardless of the relationship between dimensions and sample size.
Asymptotic Theory: The authors establish the asymptotic null distributions for both proposed statistics under two distinct regimes:
- For fixed $k$ , the normalized trace-based statistic converges to a standard normal distribution.
- For $k$ growing proportionally with $n$ , the normalized largest-root statistic converges to the Tracy-Widom distribution (Type 1).
Power Analysis: The paper provides an asymptotic power analysis under representative alternatives, characterizing how the covariance structures, cross-covariance, and tuning parameters ( $k, \lambda$ ) influence test performance.
Data-Driven Tuning: A principled, data-driven method for selecting the regularization parameter $\lambda$ is developed, which is shown to be consistent and does not invalidate the asymptotic null distributions.

Results
Extensive simulation studies evaluate the finite-sample performance of the proposed methods:

Size Control: The proposed procedures maintain satisfactory control of the Type-I error rate across a broad range of settings (varying dimensions, sample sizes, and covariance structures). The trace-based test performs well for small $k$ (e.g., $k \lesssim 20$ ), while the largest-root test is effective for larger $k$ .
Power Performance:
- The trace-based procedure generally exhibits the highest empirical power, particularly under "Exp-Decay" correlation models where signal is distributed across many eigendirections.
- The largest-root procedure performs comparably under "Low-rank" correlation models (concentrated signal) but is advantageous when a larger reduced dimension $k$ is required to capture the dependence structure, as it maintains accurate size control at higher $k$ values where the trace-based test may inflate.
Comparison: The proposed methods outperform existing regularized LRTs (e.g., Yang and Pan, 2015) and unregularized largest-root tests (e.g., Han et al., 2018) in high-dimensional regimes where dimensions exceed the sample size, primarily due to the stabilization provided by ridge regularization.

Significance and Claims
The paper claims that the proposed framework offers a statistically stable and adaptable solution for independence testing in modern high-dimensional applications. By leveraging the premise that dependence is often driven by a few dominant factors (leading PCs), the method effectively reduces dimensionality while using ridge regularization to handle the singularity of covariance matrices.

The authors emphasize that the procedures are rotation invariant, making them robust to coordinate system representations, and applicable to a wide range of high-dimensional problems where structural information (like sparsity) is limited. The work unifies previous results by providing a common framework that accommodates CCA, PC-based dimension reduction, and ridge regularization, extending the applicability of independence tests to scenarios where both $p_1$ and $p_2$ exceed $n$ . The paper concludes that the methodology is ready for practical application, supported by data-driven strategies for selecting tuning parameters.

Technical Summary: Adaptable Regularized CCA Tests for Independence of High-Dimensional Random Vectors

More like this