Fixed-level calibration of the Cauchy combination test

1. Problem Statement

The Cauchy Combination Test (CCT), introduced by Liu and Xie (2020), is a popular method for combining $K$ p-values into a single global test statistic. It is valued for providing a closed-form combined p-value and numerical stability at very small significance levels.

• Theoretical Gap: Existing asymptotic theory for CCT justifies its validity under the regime where the nominal significance level $\alpha \to 0$ (vanishing level) while $K$ is fixed or growing. Under broad dependence structures, the CCT statistic converges to a standard Cauchy distribution in this regime.
• Practical Reality: In practice, researchers apply CCT at fixed, ordinary levels (e.g., $\alpha = 0.05$ or $0.01$) with a large number of tests ($K \to \infty$).
• The Core Question: Does the standard Cauchy cutoff remain asymptotically exact when $\alpha$ is fixed and $K \to \infty$ under dependence?
• Observed Issue: Recent numerical studies suggest the answer is often no, leading to inflated Type I error rates (size distortion). The paper aims to theoretically explain this phenomenon and propose a correction.

2. Methodology and Model Setup

The author analyzes the problem under a one-factor equicorrelated Gaussian copula model, a standard framework in genomics and high-dimensional statistics.

• Model: Let $Z = (Z_1, \dots, Z_K)^\top \sim N(0, \Sigma_\rho)$, where $\Sigma_\rho$ has unit variances and constant correlation $\rho$.
• Latent Factor Representation: The dependence is generated by a single latent factor $V \sim N(0,1)$:
  $$Z_i = \sqrt{\rho} V + \sqrt{1-\rho} \varepsilon_i, \quad \varepsilon_i \overset{i.i.d.}{\sim} N(0,1)$$
• Statistic: The CCT statistic is the average of Cauchy-transformed p-values:
  $$T_K = \frac{1}{K} \sum_{i=1}^K \tan\left(\pi(\Phi(Z_i) - 1/2)\right)$$
• Regimes: The paper investigates two regimes for the correlation $\rho$ as $K \to \infty$:
  1. Fixed Positive Correlation: $\rho > 0$ is constant.
  2. Triangular Regime: $\rho = \rho_K \downarrow 0$ as $K \to \infty$.

3. Key Theoretical Findings

A. Failure of Raw CCT at Fixed Levels

The paper demonstrates that the standard Cauchy reference law fails at fixed $\alpha$ due to a latent-factor drift.

1. Fixed $\rho > 0$:

   • As $K \to \infty$, the statistic $T_K$ converges almost surely to a random variable $\mu_\rho(V)$, which depends on the realization of the latent factor $V$.
   • Consequently, there is no universal fixed-level reference law; the rejection probability depends on the distribution of $\mu_\rho(V)$, not the standard Cauchy distribution.
   • The limits do not commute: $\lim_{\rho \to 0} \lim_{K \to \infty} P(T_K > t_\alpha) = 1/2$, whereas $\lim_{K \to \infty} \lim_{\rho \to 0} P(T_K > t_\alpha) = \alpha$.

2. Weakening Dependence ($\rho_K \to 0$):

   • The behavior is governed by two scales:
     • Broad scale: $c_K = \rho_K \log K$
     • Boundary-layer scale: $s_K = \sqrt{\rho_K} (\log K)^{3/2}$
   • Decomposition: Conditionally on $V=v$, the statistic decomposes into a 1-stable fluctuation (which converges to Cauchy if $c_K \to 0$) and a deterministic centring term $b_K(v)$.
   • The Distortion: The size distortion arises entirely from this centring term $b_K(v)$, not from the heavy-tailed fluctuation.
   • Exactness Threshold for Raw CCT: The raw CCT is asymptotically exact at fixed $\alpha$ if and only if:
     $$\rho_K (\log K)^3 \to 0$$
     If this condition fails (i.e., $s_K$ is bounded away from zero), the test suffers from size distortion.

B. The Boundary-Layer Calibrated CCT (BL-CCT)

Since the statistic $T_K$ itself is not flawed (the distortion comes from the reference law), the author proposes correcting the reference law rather than modifying the statistic.

• Proposed Correction: Replace the standard Cauchy cutoff with a Gaussian-smoothed Cauchy family.
  • Define a new random variable $T_s \stackrel{d}{=} C + \kappa s V$, where $C \sim \text{Cauchy}(0,1)$, $V \sim N(0,1)$, and $\kappa = \frac{4\sqrt{2}}{3\pi}$.
  • The new p-value is $p_{BL}(t; s) = P(T_s > t)$.
• Calibration: The test rejects if $T_K > q_\alpha(s_K)$, where $q_\alpha(s_K)$ is the $\alpha$-quantile of the distribution of $T_{s_K}$.
• Improved Exactness Threshold: Under this calibration, the test is asymptotically exact under the weaker condition:
  $$\rho_K \log K \to 0$$
  This is a significant improvement over the raw CCT's requirement of $(\log K)^3$.
• Behavior on Broader Scales: If $\rho_K \log K \to c > 0$, the calibrated test becomes strictly conservative (size $< \alpha$), but the distortion is explicit and bounded, unlike the uncontrolled distortion of the raw test.

4. Implementation and Numerical Validation

• Estimation: The paper proposes a plug-in estimator for $\rho_K$ based on the sample variance of the transformed z-scores. Theoretical results show this estimator is consistent ($O_p(K^{-1/2})$), ensuring the plug-in BL-CCT performs as well as the oracle procedure.
• Simulations: Monte Carlo experiments confirm the theory:
  • The raw CCT exhibits severe size inflation as the dependence scale $s_K$ increases.
  • The BL-CCT (both oracle and plug-in versions) maintains size close to the nominal level (e.g., 0.05) across a wide range of dependence structures, significantly outperforming the raw test.

5. Significance and Contributions

1. Resolution of Theory-Practice Gap: The paper provides the first rigorous explanation for why the CCT fails at fixed levels under dependence, identifying the "latent-factor centring" mechanism that vanishing-level theory misses.
2. Sharp Thresholds: It establishes precise mathematical thresholds ($\rho_K (\log K)^3 \to 0$ vs. $\rho_K \log K \to 0$) for the validity of the raw and calibrated tests, respectively.
3. Novel Calibration Strategy: Unlike previous approaches that modify the test statistic (e.g., truncation or weight adjustment), BL-CCT preserves the computational simplicity of the original statistic by only adjusting the reference distribution. This is a "reference-law correction."
4. Practical Utility: The proposed method is easy to implement (requires only a numerical integration for the p-value) and is immediately applicable to high-dimensional data analysis in genomics, microbiome studies, and finance where dependence is common.

Conclusion

Hirofumi Ota demonstrates that the standard Cauchy combination test is not asymptotically exact at fixed significance levels under dependence due to a latent-factor induced shift. By introducing the Boundary-Layer Calibrated CCT (BL-CCT), which replaces the standard Cauchy reference with a Gaussian-smoothed Cauchy family, the paper provides a method that is asymptotically exact under much weaker dependence conditions ($\rho_K \log K \to 0$) and offers a robust, conservative alternative when dependence is stronger.