Simultaneous false discovery rate control in location families

Technical Summary: Simultaneous False Discovery Rate Control in Location Families

Problem Statement
In multiple hypothesis testing, the False Discovery Rate (FDR) is traditionally defined and controlled only at the null value (e.g., $\theta = 0$). However, in practical applications involving location families, researchers often require low FDR not just for the null, but also for parameter values deemed "practically insignificant" (e.g., small effect sizes). The standard Benjamini-Hochberg (BH) procedure controls the FDR at the null but does not explicitly guarantee control over the entire "FDR curve"—the FDR as a function of the location parameter $\theta$. This paper addresses the need for simultaneous FDR control across a continuum of parameter values within location families.

Methodology
The authors propose a generalization of the standard BH procedure to control the FDR curve below a user-specified level. The setup involves independent data $X_i \sim P_{\theta_i}$ from a location family where $P_\theta(X \le x) = F(x - \theta)$. The goal is to test one-sided hypotheses $H_{i,\theta}: \theta_i \ge \theta$.

1. FDR Curve Definition: The FDR curve is defined as the expectation of the False Discovery Proportion (FDP) curve, $FDP_S(\theta)$, which measures the proportion of false discoveries among rejected hypotheses for a given threshold $\theta$.
2. Generalized BH Procedure: Instead of using a scalar nominal level $q$, the method utilizes a user-specified non-increasing FDR curve $q(\theta)$.
   • Normalized P-values: For each hypothesis $i$, a normalized p-value is computed as:
     $$\bar{P}_i = \sup_{\theta \in \mathbb{R}} \frac{P_{i,\theta}}{q(\theta)}$$
     where $P_{i,\theta} = F(X_i - \theta)$ is the p-value for testing $H_{i,\theta}$.
   • Rejection Rule: The procedure applies the standard BH algorithm to the set of normalized p-values $\{\bar{P}_1, \dots, \bar{P}_m\}$ with a target level of 1. The rejection set $S$ is the largest subset such that $i \in S$ if and only if $\bar{P}_i \le |S|/m$.

Key Theoretical Results
The paper establishes that the generalized BH procedure simultaneously controls the FDR curve. Specifically, let $q^*(\theta)$ be a transformed version of the user-specified curve $q(\theta)$, defined as:
$$q^*(\theta) = \inf_{\theta' \in \mathbb{R}} \sup_{a \le x - \theta' \le b} q(\theta') \cdot \frac{F(x - \theta)}{F(x - \theta')}$$
where $a = F^{-1}(q(\theta')/m)$ and $b = F^{-1}(q(\theta'))$.

• Theorem 3.1: The procedure guarantees that $\sup_{\theta \in \mathbb{R}} \frac{FDR(\theta)}{q^*(\theta)} \le 1$. Consequently, $FDR(\theta) \le q^*(\theta)$ holds almost surely for any $\theta$.
• The "Free Lunch" Phenomenon: A significant corollary is that the standard BH procedure (which corresponds to using a step function $q_{BH}(\theta) = q \cdot \mathbb{1}_{\theta \ge 0} + \mathbb{1}_{\theta < 0}$) inherently provides simultaneous control over the entire FDR curve. The actual control level $q^*_{BH}(\theta)$ is strictly below the nominal step function for $\theta \neq 0$, while touching it at $\theta = 0$.
• Touching Property: Under the "monotone ratio property" (satisfied by common families like the Gaussian), Proposition 3.1 proves that for any decreasing, right-continuous $q$, there exists at least one point $\theta^*$ where the actual control curve $q^*(\theta^*)$ touches the target curve $q(\theta^*)$. This implies the control cannot be uniformly better than the nominal specification everywhere.
• Lower Bounds: Proposition 3.2 provides a lower bound for the worst-case FDR curve, showing that the analysis is tight, particularly when $q^*(\theta)$ is small.

Extensions and Numerical Examples
The framework is extended to settings where hypotheses come from different location families (different $F_i$). In this general case, the definition of $q^*$ is modified to take a supremum over the specific distributions.

• Proposition 4.1: For piecewise constant target curves, there exists a subset of constraints such that the induced $q^*$ curve controls all prescribed constraints and touches the target at least once.
• Breast Cancer Dataset: The authors apply the method to the Storey and Tibshirani (2003) breast cancer dataset. They compare testing based on effect sizes (where distributions vary by gene) versus signal-to-noise ratios (where a common distribution applies).
  • In the effect-size scenario, using all constraints simultaneously resulted in fewer rejections than using a single constraint, suggesting that selecting a minimal set of constraints that satisfies the $q^* \le q$ condition is optimal.
  • In the signal-to-noise scenario (common $F$), the $q^*$ curve touched the target at all specified locations, and the rejection sets were stable across different constraint choices.

Significance and Claims
The paper claims a "free lunch" theorem: the standard BH procedure, designed solely for null control, implicitly offers stronger simultaneous control over the FDR curve for non-null values without requiring additional computational complexity or data. The generalized procedure allows researchers to explicitly target FDR control at practically insignificant parameter values, offering a flexible tool for discovery in multiple testing. The authors note that this approach is a specific instance of a broader framework for selective compound decision problems (citing Gao, Hu, and Zhao 2024) and suggest potential future directions involving empirical Bayes approaches to select the target curve $q$ based on estimated priors, though they caution about the "double-dipping" issue in such implementations.