Technical Summary: Finite Resources False Discovery Rate Control in Structured Hypothesis Spaces

1. Problem Statement

Scientific discovery increasingly relies on large-scale hypothesis testing, yet three fundamental challenges hinder the identification of true discoveries while controlling the False Discovery Rate (FDR):

1. Finite-Data Uncertainty: Obtaining sufficient reference data (null distribution samples) for each hypothesis is resource-intensive. Standard approaches often rely on point estimates of p-values derived from finite null draws, ignoring the inherent uncertainty in these estimates, which can lead to FDR violations.
2. Structured Hypothesis Spaces: Hypotheses often exhibit inherent dependencies (e.g., spatial proximity). Ignoring this structure wastes statistical power, while existing methods for structured FDR often require pre-defined dependency graphs or fail to handle finite-sample uncertainty.
3. Resource Allocation: There is a lack of frameworks to efficiently allocate a finite budget of null samples across hypotheses, particularly when leveraging structural relationships to "borrow strength" from neighbors.

The paper addresses these challenges simultaneously by presenting a unified framework that controls FDR in structured hypothesis spaces where each hypothesis is evidenced only by a finite count of null draws, without requiring a resolved point estimate of the p-value.

2. Methodology

2.1 Generative Model and Count-Based Likelihood

The authors model the problem as a three-level hierarchy:

• Structural Layer: The prior probability of a hypothesis being null, $\alpha(\text{loc})$, is a smooth function over an arbitrary domain $\mathcal{X}$, modeled within a Reproducing Kernel Hilbert Space (RKHS) defined by a kernel $K$.
• Latent Layer: Each hypothesis has a latent p-value $p^*_i$. Under the null ($H_0$), $p^*_i \sim U[0,1]$; under the alternative ($H_1$), it follows a Beta distribution concentrated near zero.
• Observation Layer: Instead of observing $p^*_i$, the observer sees a count $k_i$ of null draws (out of $m_i$ total) that are at least as extreme as the observed statistic. Thus, $k_i | p^*_i, m_i \sim \text{Binomial}(m_i, p^*_i)$.

Crucially, the framework abandons the "plug-in" approach (estimating $p^*_i$ as $(k_i+1)/(m_i+1)$). Instead, it explicitly marginalizes out the latent $p^*_i$ to derive exact count-based likelihoods under the Beta-Binomial conjugacy. This yields closed-form expressions for the marginal likelihoods $P_0(k, m)$ and $P_1(k, m)$, avoiding the discretization error and sampling noise associated with point estimates.

2.2 Spatial Estimation

To leverage structure, the global null proportion $\bar{\alpha}$ is lifted to a spatial function $\alpha(\text{loc})$. The authors estimate $\hat{\alpha}$ by minimizing a regularized negative log-likelihood in the RKHS:
$$\hat{\alpha} = \arg \min_{\alpha \in \mathcal{H}_K} -\sum_{i=1}^N \log[\alpha(\text{loc}_i)P_{0,i} + (1-\alpha(\text{loc}_i))P_{1,i}] + \lambda \|\alpha - \hat{\bar{\alpha}}\|^2_{\mathcal{H}_K} + \gamma \Lambda_{[0,1]}(\alpha)$$
This optimization is solved efficiently using Natural Gradient Descent, which avoids inverting the kernel Gram matrix, reducing complexity to $O(N^2)$ per iteration.

2.3 Decision Rules

The paper proposes two decision rules with distinct trade-offs between exact FDR control and statistical power:

• Rule 1 (Model-Free): Uses the spatial estimator $\hat{\alpha}$ solely as a pre-selection gate to filter hypotheses. Within this gate, it applies a running-average threshold on the non-spatial marginal local FDR (lfdr).

  • Guarantee: Provides exact FDR control ($\le \tau$) regardless of the accuracy of the spatial model $\hat{\alpha}$, as the spatial information does not influence the rejection ranking.
  • Trade-off: Conservative; may sacrifice power if the spatial model is informative but ignored in the ranking.

• Rule 2 (Mirror Statistics in Count Space): Uses the spatial lfdr directly as the rejection score. It adapts the Barber-Candès mirror statistic framework to the count space, utilizing the reflection $k \leftrightarrow m-k$ as the mirror operation.

  • Guarantee: Under a "mirror assumption" (invariance of the scoring surface to flipping a single null's count), it achieves exact control. Since the surface is data-dependent, this assumption is only approximately true. The authors provide an analytical framework showing that the FDR violation is bounded by a slack term $\delta = O(\max(1/N, 1/\lambda))$.
  • Trade-off: Maximizes statistical power by utilizing all available information (including spatial structure) for ranking, at the cost of a small, quantifiable slack in FDR control.
  • Theoretical Result: Under correct specification, Rule 2 is strictly more powerful than Rule 1.

2.4 Adaptive Resource Allocation

The framework includes a policy for the efficient allocation of a finite null-sampling budget. It classifies hypotheses into three states based on their local FDR confidence intervals:

1. Decided: CI is entirely on one side of the threshold.
2. Ambiguous: CI straddles the threshold (targets for further sampling).
3. Hopeless: CI straddles the threshold even as uncertainty approaches its theoretical "variance floor" (due to isolation or uninformative p-values).

The allocation policy prioritizes ambiguous hypotheses based on a score combining:

• Local Fisher Information: How much the variance can shrink with more local samples.
• Spatial Borrowing: How much sampling a neighbor reduces the uncertainty of the target hypothesis via the kernel.
  The policy dynamically adjusts the balance between local and spatial sampling and can sample "decided" or "hopeless" hypotheses if doing so provides significant spatial benefit to neighbors (helper sampling).

3. Key Contributions

1. Unified Framework: The first framework to simultaneously address finite-sample p-value uncertainty, structured hypothesis spaces, and resource allocation. It extends the Bayesian FDR framework to finite-sample regimes using RKHS.
2. Count-Based Likelihoods: A derivation of exact marginal likelihoods in count space that avoids the $O(1/\sqrt{m})$ sampling noise of plug-in p-value estimates, achieving $O(1/m)$ convergence to the continuous lfdr.
3. Two Decision Rules:
   • A robust, exact-control rule (Rule 1) that is immune to spatial model misspecification.
   • A high-power rule (Rule 2) that adapts mirror statistics to count space, with a rigorous analytical bound on the FDR slack caused by data-dependent scoring surfaces.
4. Adaptive Allocation Policy: A tractable method for allocating null samples that accounts for both local information gain and spatial information propagation, including mechanisms to identify and exclude "hopeless" hypotheses.
5. Analytical Slack Characterization: A detailed analysis of the "mirror assumption" violation in structured spaces, quantifying the FDR slack in terms of the influence of a single count flip on the fitted surface.

4. Empirical Results

The framework was evaluated on:

• ADbench (Semi-synthetic): 10 high-dimensional anomaly detection datasets.
  • FDR Control: Both rules maintained FDR control across various structural settings (signal density, smoothness), whereas standard Benjamini-Hochberg (BH) and RegFDR with plug-in p-values violated FDR control significantly in the finite-sample regime.
  • Power: Rule 2 achieved consistently higher power than Rule 1 and baselines, operating closer to the target FDR budget.
• AlpacaEval 2.0 (LLM-as-judge): A benchmark with no spatial structure.
  • The framework significantly outperformed a heuristic baseline (uncalibrated BH) in actionable discoveries, validating the finite-m machinery in a non-spatial setting.
• Adaptive Allocation:
  • The "Active" allocation policy (proposed in Section 5.2) outperformed Uniform and Random sampling strategies, achieving higher power and a greater fraction of decided hypotheses per unit budget.
  • Active allocation converged faster to the oracle spatial function in RKHS distance compared to baselines.

5. Significance and Claims

The paper claims to present the first unified framework to address the triad of finite-data uncertainty, structured hypothesis spaces, and resource allocation in FDR control.

• Theoretical Significance: It establishes that working in count space with marginal likelihoods avoids the asymptotic sampling noise floor of continuous p-value estimation ($O(1/m)$ vs $O(1/\sqrt{m})$). It provides a rigorous characterization of the trade-off between exact FDR control and power in structured settings, introducing a quantifiable slack for data-dependent mirror statistics.
• Practical Significance: The proposed allocation policy allows researchers to maximize discoveries under strict resource constraints by intelligently directing null samples where they reduce uncertainty most effectively.
• Modesty: The authors acknowledge limitations, including the assumption of uniform null p-values (addressable via Beta mixtures), the existence of variance floors for isolated hypotheses, and the reliance on a "burn-in" phase for the allocation policy to satisfy i.i.d. assumptions. They note that the exact FDR control of Rule 2 relies on stability assumptions (threshold and ranking stability) that, while typical under the Beta-Binomial geometry, are not guaranteed in all configurations.

The work positions itself as a bridge between kernel-based regularization methods and finite-sample FDR control, offering a tractable approach to complex, structured scientific discovery problems where data is scarce.