📊 statistics

Improving the Power of Bonferroni Adjustments under Joint Normality and Exchangeability

This paper proposes a practical modification to the Bonferroni correction for jointly normal and exchangeable test statistics that maintains family-wise error rate control while achieving higher power in sparse alternatives compared to the standard method.

Original authors: Caleb Hiltunen, Yeonwoo Rho

Published 2026-02-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Caleb Hiltunen, Yeonwoo Rho

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a crime, but you have 1,000 suspects instead of just a few. You run a test on every single one of them to see if they are guilty.

The Problem: The "Over-Protective" Judge

In statistics, this is called Multiple Comparisons. If you test 1,000 innocent people, you will almost certainly find a few who look guilty just by pure luck (random noise). To stop this, you need a rule to make your tests stricter.

The most famous rule is the Bonferroni Correction. Think of it as a very strict, over-protective judge.

How it works: The judge says, "If I'm testing 1,000 people, I need to be 1,000 times more sure before I convict anyone."
The Flaw: This judge is so strict that they often let the actual guilty people go free. In statistics, this is called low power. The judge is so afraid of making a mistake (convicting an innocent person) that they miss the real criminals.

This gets even worse if the suspects are connected. Imagine the suspects are a group of friends who all hang out together. If one friend acts suspicious, the others might act suspicious too, not because they are guilty, but because they are influenced by each other. The standard "strict judge" doesn't understand this connection and becomes even more conservative, missing even more real crimes.

The New Solution: The "Smart Detective"

The authors of this paper, Caleb and Yeonwoo, propose a new method. They are like a Smart Detective who understands that the suspects are friends (statistically, they call this "exchangeable" and "jointly normal").

Here is how their new method works, using simple analogies:

1. The "Cluster" Idea

When these "connected" suspects are actually innocent, they tend to huddle together in a tight group.

The Old Way: The judge ignores the group dynamic and treats everyone as a stranger.
The New Way: The Smart Detective looks at the group. "Hmm, these guys are all standing very close together. They are probably just nervous friends, not a gang of criminals."

2. Measuring the "Huddle" (Estimating Correlation)

The key to the new method is figuring out how tightly the suspects are huddled. In math, this is called the correlation ( $\rho$ ).

The old method (Gupta et al., 1973) knew this was important but said, "You need to know the exact tightness of the huddle to use our rule."
The Problem: In real life, you rarely know the exact tightness. It's a secret number.
The Innovation: The authors say, "Don't worry! We can guess the tightness by looking at how spread out the group is."
- If the group is very spread out (high variance), they aren't huddling much (low correlation).
- If the group is a tight knot (low variance), they are huddling hard (high correlation).
- They use a simple math trick (sample variance) to estimate this secret number on the fly.

3. The Result: Catching More Criminals

Because the Smart Detective understands the group dynamic and estimates the "huddle" correctly, they can relax the rules just enough to catch the real criminals without letting too many innocent people go.

In "Sparse" Situations: Imagine only one suspect is actually guilty, and the other 999 are innocent. The old strict judge misses the one guilty guy because the rules are too hard. The new method spots him because it knows the 999 innocent ones are just a tight, innocent group, making the one outlier stand out clearly.
The Trade-off: The new method is still safe. It guarantees that you won't convict an innocent person more often than you are supposed to (controlling the "Family-Wise Error Rate").

Why This Matters

Think of this like tuning a radio:

Bonferroni is like turning the volume down so low you can't hear the music (the signal), only the static (the noise).
The Old Methods for Dependent Data are like trying to tune the radio without knowing the station frequency.
This New Method is like a radio that automatically detects the frequency of the static and tunes itself perfectly to hear the music clearly, even when the signal is weak and the static is loud.

Summary

This paper takes a very old, very strict rule (Bonferroni) and gives it a "smart upgrade." By realizing that data points often move together (like friends in a group) and by using a simple trick to estimate how closely they are connected, the new method finds the "guilty" signals much better than before, especially when there are only a few guilty signals hidden among thousands of innocent ones. It's safer, smarter, and much more effective.

1. Problem Statement

The paper addresses the issue of multiplicity correction in multiple hypothesis testing, specifically focusing on the Family-Wise Error Rate (FWER).

The Limitation of Bonferroni: While the Bonferroni correction is popular due to its simplicity, it is notoriously conservative (low power) when test statistics are dependent.
The Gap in Existing Methods: Other methods (e.g., Holm, Benjamini-Hochberg, or heavy-tailed combination tests) exist to handle dependence, but they often lack the intuitive "step-down" nature of Bonferroni that allows practitioners to easily identify which specific individual tests are significant after rejecting the global null.
The Specific Challenge: Previous attempts to improve Bonferroni under dependence (specifically Gupta et al., 1973) assumed joint normality and exchangeability but required the common pairwise correlation ( $\rho$ ) to be known. In practice, $\rho$ is almost always unknown, and estimating it from a single set of test statistics is difficult because standard consistency arguments (like the Weak Law of Large Numbers) often fail without replication.

2. Methodology

The authors propose a modified procedure, termed GNP-MOM*, which extends the Gupta-Nagel-Panchapakesan (GNP) framework to the two-sided case and incorporates a method-of-moments estimator for the unknown correlation.

A. Statistical Framework

Assumptions: The test statistics $X = (X_1, ..., X_n)$ are jointly normal, standardized (variance 1), and exchangeable.
Correlation Structure: The variables share a common correlation $\rho \in [0, 1)$ . They can be represented as:
$X_i = \sqrt{\rho}Z_0 + \sqrt{1-\rho}Z_i$
where $Z_0, Z_1, ..., Z_n$ are i.i.d. standard normal variables.
Global Test Statistic: The method uses the maximum absolute value of the test statistics:
$M_n(|X|) = \max_{1 \le i \le n} |X_i|$

B. The Proposed Procedure (GNP*-MOM)

Estimation of $\rho$ : Since $\rho$ is unknown, the authors propose estimating it using the sample variance ( $s_n^2$ ) of the test statistics, based on the estimator by Follmann and Proschan (2013):
$\hat{\rho}_{MOM} = (1 - s_n^2)I(s_n^2 < 1)$
Rationale: Under the null hypothesis, positively correlated statistics cluster around a single random number. The tightness of this cluster (variance) is inversely related to the correlation magnitude.
Critical Value Calculation:
- The method calculates a critical value $c_\alpha$ such that $P(M_n(|X|) \le c_\alpha) = 1 - \alpha$ .
- This probability is derived via numerical integration over the standard normal density $\phi(z)$ , conditioning on the common factor $Z_0 = z$ :
  $P(M_n(|X|) \le c_\alpha) = \int_{-\infty}^{\infty} \Psi^n_{\mu=\sqrt{\rho}z, \sigma=\sqrt{1-\rho}}(c_\alpha) \phi(z) dz = 1 - \alpha$
  where $\Psi$ is the CDF of the folded-normal distribution.
- In practice, the unknown $\rho$ is replaced by $\hat{\rho}_{MOM}$ .
Decision Rule:
- Reject the global null $H_0: \mu = 0$ if $M_n(|X|) \ge c_\alpha$ (or if the calculated p-value $p_0 \le \alpha$ ).
- Post-hoc Identification: Any individual test $X_i$ is declared significant if $|X_i| \ge c_\alpha$ .

C. Theoretical Validity

The paper provides rigorous proofs (Theorems 1–3) demonstrating:

Consistency: The estimator $\hat{\rho}_{MOM}$ satisfies the necessary asymptotic conditions (Assumption 1) to ensure the estimated p-value converges to the true p-value as $n \to \infty$ .
FWER Control: The procedure controls the FWER at level $\alpha$ for both two-sided and one-sided alternatives, even with the estimated $\rho$ .

3. Key Contributions

Practical Implementation of GNP: The paper bridges the gap between the theoretical GNP method (1973) and practical application by introducing a robust estimator for $\rho$ that does not require prior knowledge of the correlation structure.
Two-Sided Extension: It extends the original one-sided GNP framework to two-sided tests, which are more common in general scientific applications.
Hybrid Advantage: The method retains the intuitive interpretability of Bonferroni (identifying specific significant tests) while achieving the power of dependence-aware methods.
Sparse Alternative Optimization: The method is specifically highlighted for its superior performance in sparse alternatives (where only a few hypotheses are true), a regime where standard Bonferroni fails and heavy-tailed combination tests (like HMP) may struggle with power.

4. Simulation Results

The authors conducted extensive simulations comparing GNP-MOM* against Bonferroni, Harmonic Mean P-value (HMP), Adjusted HMP (HMPADJ), and Follmann-Proschan (F-P).

Sparse Alternatives (Single Non-Zero Mean):
- GNP*-MOM consistently achieved equal or higher power than all competitors.
- As correlation ( $\rho$ ) and significance level ( $\alpha$ ) increased, GNP*-MOM's power advantage grew, while other methods faltered.
Dense Alternatives (Increasing Proportion of Non-Zero Means):
- When the proportion of non-zero means was < 10%, GNP*-MOM significantly outperformed other methods, particularly at higher $\rho$ and $\alpha$ .
Size Control (Type I Error):
- Simulation tables (Appendix B) show that GNP*-MOM provides exact control of the FWER across various sample sizes ( $n$ ), correlations ( $\rho$ ), and significance levels ( $\alpha$ ).
- In contrast, methods like HMP and F-P often exhibited inflated Type I error rates (exceeding $\alpha$ ) under certain dependence structures.
Limitation Identified: While the global null rejection is powerful, the method is conservative for identifying secondary significant tests (those that are not the maximum). The critical value $c_\alpha$ is driven by the maximum, making it difficult to detect other true positives that are slightly below the maximum.

5. Significance and Conclusion

Practical Utility: This method offers a "best of both worlds" solution for practitioners dealing with dependent, exchangeable normal test statistics. It avoids the complexity of heavy-tailed combination tests while correcting the severe power loss of standard Bonferroni.
Robustness: The method is robust to unknown correlation parameters, a common real-world constraint.
Future Directions: The authors suggest that while the global test is optimal, the selection of individual significant tests could be improved via sequential testing or order statistics approaches to reduce the conservatism for non-maximal tests.

In summary, Hiltunen and Rho provide a theoretically sound, simulation-validated modification to Bonferroni correction that significantly enhances statistical power in dependent, exchangeable settings without sacrificing error rate control.