📊 statistics

Detection coherence of tests

This paper introduces a framework for assessing "detection coherence" to reveal that many widely used statistical tests possess "blind spots" where they fail to detect true effects, arguing that such detection incoherent tests should be abandoned in favor of universally coherent alternatives.

Original authors: Marian Grendar

Published 2026-08-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Marian Grendar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective's Dilemma: When Your Clues Lie

Imagine you are a detective trying to solve a mystery. In the world of statistics, this mystery is often about figuring out if two groups of things are truly different or just happen to look different by chance. Scientists use tools called "statistical tests" to act as their magnifying glasses. These tests look at data—like the heights of plants in two different gardens or the survival times of patients in two different treatments—and ask: "Is there a real signal here, or is it just noise?"

For a test to be a good detective, it needs to be calibrated correctly. This means the rules for what counts as a "clue" must match exactly what the test is looking for. If a test is designed to spot a specific type of difference, but it accidentally ignores other types of differences that are just as real, it has a "blind spot." It's like a metal detector that beeps for coins but stays silent for gold bars. If you don't know about this flaw, you might walk right past a treasure chest, thinking the ground is empty, or you might think you found a coin when you actually found something else entirely. This paper dives into the hidden blind spots of some of the most famous detective tools used in science today.

The Paper's Big Discovery: The "Blind Spot" Problem

This paper, written by M. Grendár, introduces a new way to check if these statistical detectives are actually doing their job correctly. The author calls this concept "detection coherence." In simple terms, a test is "coherent" if its blind spot is empty—meaning it can see every single type of difference it claims to be able to see. If a test has a blind spot, it is "incoherent," and the paper argues that this is a serious problem that shouldn't be ignored.

The author proves that four of the most popular non-parametric tests (tools used when data doesn't follow a neat, bell-curve pattern) are actually detection incoherent when used without strict restrictions. These four tests are:

The Wilcoxon–Mann–Whitney (WMW) test: Used to compare two groups.
The Kruskal–Wallis (KW) test: Used to compare three or more groups.
The Friedman test: Used for comparing treatments in a block design (like testing different fertilizers on the same plot of land over time).
The Logrank test: Used to compare survival rates, often in medical studies.

The paper demonstrates that these tests have a "blind spot." This blind spot is a specific set of situations where the groups are actually different, but the test's internal logic says they are the same. Because of this, the test fails to raise the alarm.

The "Aha!" Moment: The Variance Trap
To understand the WMW test's blind spot, imagine you are comparing the heights of two groups of people.

Group A has everyone exactly 170 cm tall.
Group B has a mix of very short and very tall people, but their average height is also 170 cm.

The WMW test is designed to see if one group is generally taller than the other. In this scenario, because the averages are the same, the test thinks, "Hey, these groups are identical!" and says "No difference found." But wait! Group B is clearly more varied than Group A. They are different distributions. The test is blind to the difference in spread or variance because its internal "detector" only cares about the order of the numbers, not how far apart they are.

The paper shows that for the WMW test, if you have two groups with the same average but different spreads (like a normal distribution with a standard deviation of 1 versus one with a standard deviation of 10), the test's "power" (its ability to find the difference) gets stuck. It doesn't get better even if you collect more and more data. It stays at a low level, effectively ignoring the difference forever. The author calls this a "missed discovery."

The "Spurious" Trap
The problem goes both ways. The paper also explains that if a test does reject the idea that the groups are the same, it might be a "spurious discovery." Imagine the test sounds the alarm because it found a difference in the spread, but the researcher thought they were only looking for a difference in the average. The test is technically right that the groups are different, but it's giving the wrong reason. The paper argues that because of these blind spots, these tests can lead scientists to miss real effects or claim to find effects that aren't what they think they are, regardless of how much data they collect.

The "Fix" That Isn't a Fix
You might think, "Okay, so these tests have blind spots. Can't we just tell them to only look at specific types of data where the blind spot doesn't exist?" The paper says, "Yes, but be careful."

The author shows that if you restrict the WMW test to only look at data where the shapes of the distributions are the same (just shifted left or right), the blind spot disappears. However, the paper argues that relying on these restrictions is dangerous in the real world. Why? Because in real life, you rarely know for sure if your data fits those strict rules. If you assume your data fits the rule, but it actually doesn't, you are back to square one with a blind spot. The paper concludes that trying to "salvage" these tests with restrictions is unreliable.

The Good Guys
Not all tests are broken. The paper highlights three tests that are "universally detection coherent," meaning they have no blind spots at all. These are:

The Kolmogorov–Smirnov (KS) test: This looks at the entire shape of the distribution, not just the order, so it sees everything.
The Zaremba test: A clever reformulation of the WMW test that changes the rules to look for a different kind of "null" (AUC = 1/2) instead of just "identical distributions."
The Maximum Mean Discrepancy (MMD) test: A modern tool that uses a special "kernel" to compare distributions, which is proven to see every possible difference.

The Final Verdict
The paper's main conclusion is bold: because of these permanent blind spots, the four major tests (WMW, KW, Friedman, and Logrank) should be abandoned when used as "unrestricted" tests (tests used without knowing the specific shape of the data beforehand). The author suggests that using them is like using a metal detector that ignores gold; you might as well stop using it. Instead, scientists should switch to the coherent alternatives like the KS test or the Zaremba test, which don't have these hidden flaws.

The paper doesn't just suggest this; it provides a mathematical framework to prove it, showing exactly where the blind spots are and calculating how the tests behave in those spots. It's a wake-up call for the scientific community to check their tools and make sure they aren't missing the most important discoveries right under their noses.

Technical Summary: Detection Coherence of Tests

Problem Statement
The paper addresses a fundamental flaw in the calibration of several widely used nonparametric statistical tests. Specifically, it identifies that certain tests are calibrated under a null hypothesis ( $\mathcal{H}_0$ ) that is strictly narrower than the set of distributions implied by their test statistic's asymptotic behavior. This discrepancy creates a "blind spot" ( $\mathcal{SP}$ ): a set of alternative distributions where the test statistic behaves as if the null were true (i.e., the detection functional equals its null value), even though the distributions are not in $\mathcal{H}_0$ .

When a test possesses a non-empty blind spot, it is termed detection incoherent. The consequences of this incoherence are severe and persistent regardless of sample size:

Missed Discoveries: When the test fails to reject, the result may stem from a distribution in the blind spot (a false negative relative to the alternative) rather than the true null.
Spurious Discoveries: When the test rejects, the result may stem from a distribution in the blind spot (a true positive relative to the blind spot, but a spurious discovery relative to the intended null hypothesis).

The author argues that standard nonparametric theory often overlooks this gap, treating the null hypothesis and the detection-null set as equivalent when they are not.

Methodology and Framework
The paper introduces a formal framework for assessing detection coherence, built upon several new definitions:

Detector ( $\hat{D}_n$ ): The component of a calibration statistic ( $T_n$ ) that converges in probability to a detection functional ( $\theta(P)$ ). The extraction of the detector involves removing scaling sequences, variance normalizations, and nuisance estimators from the asymptotic expansion of $T_n$ .
Detection Functional ( $\theta(P)$ ): The probability limit of the detector.
Detection-Null Set ( $\mathcal{D}_0$ ): The set of all distributions $P$ where the detection functional equals its null value ( $\theta^*$ ).
Blind Spot ( $\mathcal{SP}$ ): The set difference $\mathcal{D}_0 \setminus \mathcal{H}_0$ .
Detection Coherence: A test is detection coherent with respect to a domain $\mathcal{R}$ if its blind spot within that domain is empty ( $\mathcal{SP} \cap \mathcal{R} = \emptyset$ ). This is equivalent to the condition that the alternative hypothesis ( $\mathcal{H}_0^c$ ) coincides with the detection set ( $\mathcal{D}$ ) over the domain.

The paper proposes a systematic procedure (Procedure 1) to assess coherence: identify the detector and functional, determine the detection-null value, construct the detection-null set, and verify if it coincides with the null hypothesis over the domain of interest.

Key Results
The paper applies this framework to four major nonparametric tests and three coherent alternatives:

Detection Incoherence of Unrestricted Tests:
The following tests are demonstrated to be detection incoherent when applied as unrestricted tests (over the full distribution space $\mathcal{P}$ ):
- Wilcoxon–Mann–Whitney (WMW): Calibrated under $F_1 = F_2$ , but its detector (eAUC) implies a detection-null set where $AUC = 1/2$. The blind spot includes distributions with equal means but different variances (e.g., $N(0,1)$ vs. $N(0, \sigma^2)$ for $\sigma \neq 1$ ).
- Kruskal–Wallis (KW): Calibrated under $F_1 = \dots = F_K$ , but the detection-null set includes distributions with different variances or shapes that yield identical mean ranks.
- Friedman: Calibrated under equal within-block distributions, but the blind spot includes scenarios where treatment effects cancel out across blocks (e.g., alternating signs of treatment effects).
- Logrank: Calibrated under equal survival functions ( $S_1 = S_2$ ), but the blind spot includes crossing hazard functions where the weighted integral of the hazard difference is zero.
For these tests, power against blind spot alternatives does not grow with sample size; it converges to a fixed value (often the significance level $\alpha$ or slightly above), behaving indistinguishably from Type I error inflation if the blind spot is unrecognized.
Coherence Under Domain Restrictions:
The paper identifies specific domain restrictions under which these tests can achieve coherence, though it emphasizes these are sufficient conditions, not reliable practical remedies:
- WMW: Coherent under Monotone Likelihood Ratio (MLR) families (including location-shift families).
- Logrank: Coherent under Ordered Hazards families (a broader class than Proportional Hazards).
- KW and Friedman: Coherent under location-shift and block-consistent stochastic ordering, respectively.
Universally Detection Coherent Tests:
The paper demonstrates that the following tests are universally detection coherent (blind spot is empty over the entire distribution space):
- Kolmogorov–Smirnov (KS): The detection-null set ( $\sup |F-G|=0$ ) coincides exactly with the null hypothesis ( $F=G$ ).
- Zaremba Test: This test recalibrates the WMW statistic under the hypothesis $AUC = 1/2$ rather than $F_1 = F_2$ , thereby aligning the null with the detection-null set.
- Maximum Mean Discrepancy (MMD): With a characteristic kernel, $MMD^2(F, G) = 0$ if and only if $F=G$ , ensuring coherence.

Significance and Claims
The paper claims that detection incoherence is a structural flaw that leads to permanent inferential errors (missed and spurious discoveries) that cannot be resolved by increasing sample size. Consequently, the author asserts that the four major incoherent tests (WMW, KW, Friedman, Logrank) "should be abandoned" as unrestricted tests.

The framework provides a theoretical basis for understanding why calibration under the detection-null set is a necessary condition for coherent inference. It distinguishes detection coherence from robustness: while robustness concerns Type I error control under violated assumptions, detection coherence concerns whether the test can actually detect all violations of the null hypothesis. The paper suggests that future work should involve assessing all current nonparametric tests for detection coherence and advocates for the use of coherent alternatives (like Zaremba's reformulation of WMW or MMD) over traditional rank-based tests when unrestricted inference is required.

The Detective's Dilemma: When Your Clues Lie

The Paper's Big Discovery: The "Blind Spot" Problem

More like this