📊 statistics

SCORE: A Unified Framework for Overshoot Refund in Online FDR Control

This paper introduces SCORE, a unified framework that enhances the statistical power of online False Discovery Rate control methods based on $e$ -values by reclaiming wasted evidence through overshoot refunds and enabling retroactive alpha-wealth updates, while strictly preserving valid finite-sample FDR control.

Original authors: Qi Kuang, Bowen Gang, Yin Xia

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Qi Kuang, Bowen Gang, Yin Xia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective running a massive, never-ending investigation. Every day, a new suspect (a hypothesis) walks into your office, and you have a limited amount of "detective time" (a budget) to decide if they are guilty. Your goal is to catch as many real criminals as possible (find true discoveries) without falsely accusing too many innocent people (controlling the False Discovery Rate).

In the past, detectives used a strict rule: if the evidence against a suspect was strong enough to cross a specific line, they were arrested. But here was the problem: if the evidence was massive—like a smoking gun found in their pocket—the detective would still just say, "Guilty," and throw away the extra evidence. That "extra" evidence was wasted. It was like finding a million dollars in a suspect's pocket but only counting it as one dollar toward your budget.

This paper, titled SCORE, introduces a new, smarter way to run these investigations.

The Core Idea: "Overshoot Refund"

The authors realized that when evidence is overwhelmingly strong, it shouldn't just trigger a "Guilty" verdict; it should also pay you back.

Think of your "detective time" as a bank account.

The Old Way: To arrest someone, you pay a flat fee of $10. If the evidence is worth $100, you still only get credit for the $10 fee. The remaining $90 of evidence is thrown in the trash.
The SCORE Way: You still pay the $10 fee, but because the evidence was so strong (the "overshoot"), the bank refunds you the extra $90. Now, you have more money in your account to investigate future suspects.

This "refund" allows you to be more aggressive later on. You can afford to take more risks on weaker suspects because your "strong" wins earlier have paid for them.

How It Works (The Magic Math)

The paper uses a mathematical tool called an e-value (think of it as a "guilt score").

If the guilt score is low, the suspect is innocent.
If the guilt score is high, they are likely guilty.

The old methods treated the "guilt score" like a simple light switch: On or Off. If it was above the line, it was "On."
The SCORE framework looks at the brightness of the light. If the light is blindingly bright, that extra brightness is converted back into cash (budget) for your investigation.

They proved mathematically that you can do this without ever falsely accusing an innocent person more often than you promised. It's like having a magic rule that says, "You can spend your refund, but the math guarantees you won't go broke on false accusations."

The Three New Detectives

The authors took three of the best existing detective methods (called LOND, LORD, and SAFFRON) and gave them this new "refund" superpower:

SCORE-LOND: The basic version that refunds the extra evidence.
SCORE-LORD: A version that manages the budget more dynamically, refunding evidence to help future cases.
SCORE-SAFFRON: A smart version that also filters out weak suspects early, saving even more budget to refund later.

They also created a "Super Version" called SCORE+. This version does something even more clever: it looks at the current total number of arrests made so far to retroactively lower the cost of past arrests. It's like realizing, "Wow, we caught so many criminals today, that means the cost of every arrest we made last week was actually cheaper than we thought!" This frees up even more budget for the future.

Does It Work?

The authors tested this in two ways:

Simulations: They created fake data streams with thousands of suspects. In every test, the SCORE detectives found significantly more criminals than the old detectives, while keeping the rate of false accusations exactly the same.
Real Data: They applied this to a real dataset involving yeast genetics (testing how chemicals affect yeast growth). The SCORE methods found nearly double the number of significant interactions compared to the standard methods.

The Bottom Line

The paper claims that by simply refusing to throw away "extra" strong evidence, we can make our online testing systems much more powerful. We can find more true discoveries without breaking the rules of statistical safety. It turns a "waste" into a "reward," making the entire scientific process more efficient.

Technical Summary: SCORE: A Unified Framework for Overshoot Refund in Online FDR Control

1. Problem Formulation

The paper addresses the challenge of online multiple hypothesis testing, where hypotheses arrive sequentially, and decisions must be made immediately without knowledge of future tests. The primary objective is to maximize the number of true discoveries (statistical power) while rigorously controlling the False Discovery Rate (FDR) at every step.

While e-value-based methods (e.g., e-LOND, e-LORD, e-SAFFRON) have emerged as robust alternatives to p-value-based approaches because they require minimal dependence assumptions (specifically, conditional e-validity), they often suffer from reduced statistical power. The authors identify a specific source of this inefficiency: the "overshoot" phenomenon.

In standard e-value procedures, a hypothesis is rejected if the e-value $e_t$ exceeds the threshold $1/\alpha_t$ . However, existing methods treat the rejection as a binary event. Once the threshold is crossed, any evidence exceeding the threshold (the "overshoot") is discarded. This leads to conservative error estimates and a loss of potential "alpha-wealth" that could be used for future tests.

2. Methodology: The SCORE Framework

The authors propose SCORE (Sequential Control with Overshoot Refund for E-values), a unified framework designed to reclaim this "wasted" evidence.

Core Mathematical Insight

The framework relies on a refined inequality that tightens the standard Markov-based bound used in FDR proofs.

Standard Bound: $\delta_j = \mathbb{I}(e_j \ge \alpha_j^{-1}) \le \alpha_j e_j$ .
SCORE Bound: Using the elementary inequality $\mathbb{I}(y \ge 1) \le y - (y-1)_+$ for $y \ge 0$ , the authors derive:
$\delta_j \le \alpha_j e_j - (\alpha_j e_j - 1)_+$
Here, the term $O_j = (\alpha_j e_j - 1)_+$ represents the overshoot.

Mechanism

The SCORE framework modifies the "cost" assigned to a rejection. Instead of charging the full budget $\alpha_j$ , the algorithm charges a reduced cost:
$C_j^{SCORE} = (C_j - O_j)_+$
where $C_j$ is the original cost (e.g., $\alpha_j$ in e-LOND). The overshoot $O_j$ is effectively "refunded" to the algorithm's error budget. This allows the algorithm to maintain valid FDR control while increasing the available wealth for subsequent tests, thereby improving power.

Extensions: Retroactive Updates (SCORE+)

The paper further introduces SCORE+, which incorporates retroactive wealth updates.

Standard Approach: Uses a local denominator $R_{j-1} + 1$ in the False Discovery Proportion (FDP) estimator, which is conservative and non-retroactive.
SCORE+ Approach: Uses a global denominator $R_t \vee 1$ . This allows the effective cost of past tests to be reduced as new discoveries are made.
Assumption: This aggressive strategy requires a mild dependence assumption called Conditional Positive Quadrant Dependence (CPQD), which posits that a larger current e-value does not decrease the final number of discoveries. The authors argue this is a natural consequence of alpha-investing intuition and holds under standard conditions (independence + monotone updates).

3. Key Contributions

Unified Framework: The introduction of SCORE as a "plug-in" module that can be applied to any existing e-value-based online testing algorithm (e-LOND, e-LORD, e-SAFFRON) to improve power without altering the fundamental validity structure.
Theoretical Dominance: The authors prove that SCORE-enhanced algorithms strictly dominate their original counterparts. They offer uniformly higher power while preserving valid finite-sample FDR control under the same minimal assumptions (conditional e-validity).
Retroactive Updates: The development of SCORE+ variants that leverage retroactive wealth updates to further narrow the efficiency gap with p-value-based methods, provided mild dependence assumptions (CPQD) are met.
Application to State-of-the-Art: The derivation of specific update rules for SCORE-LOND, SCORE-LORD, and SCORE-SAFFRON, demonstrating how the overshoot refund mechanism is integrated into each.

4. Experimental Results

The effectiveness of SCORE and SCORE+ is validated through extensive simulations and real-data experiments:

Independent Setting (Gaussian Mixture): In scenarios with independent data streams, SCORE+ variants consistently achieve the highest power, followed by standard SCORE, and then the original baselines. The power ratio over baselines increases over time as the accumulated refunds release more wealth.
Dependent Setting (Autoregressive Exponential Model): Under temporal dependence, the methods maintain valid FDR control. SCORE still provides significant power gains over baselines, with SCORE+ yielding the largest improvements, particularly for the SAFFRON family where the overshoot refund is applied to both candidate penalties and rejection overshoots.
Real-World Data (Yeast Chemical-Genomics Screen): Applied to a dataset of 417,026 interactions.
- SCORE-LORD detected 28,808 interactions vs. 25,954 for e-LORD.
- SCORE-SAFFRON detected 37,748 vs. 29,867 for e-SAFFRON.
- SCORE+ variants showed the most dramatic gains, with SCORE+-LORD detecting 44,210 interactions (nearly double the baseline) and SCORE+-SAFFRON detecting 43,624.
Additional Simulations: Comparisons with p-value-based methods in AR(1) settings show that e-value methods with SCORE controls achieve power comparable to or exceeding the best p-value methods, while maintaining robustness under dependence where marginal p-value methods fail.

5. Significance and Claims

The paper claims that SCORE addresses a fundamental inefficiency in online hypothesis testing: the discrepancy between the actual error committed and the theoretical cost charged by the algorithm.

Robustness: Unlike previous methods that attempt to recover wealth (e.g., ADDIS, SUR) by relying on specific distributional assumptions or super-uniformity of p-values, SCORE operates within the e-value paradigm. It requires only the standard conditional e-validity assumption, making it applicable to complex, real-world data streams where dependence structures are unknown or violated.
Universality: The framework is presented as a universal refinement. Because the inequality $\mathbb{I}(y \ge 1) \le y$ is widely used in validity proofs for e-values, the authors suggest that a broad range of e-value-based procedures (both online and potentially offline) could be enhanced by explicitly accounting for overshoot.
Practical Impact: The results demonstrate that recovering "wasted" evidence via overshoot refund and retroactive updates is highly effective in large-scale, high-throughput screening scenarios, significantly increasing the yield of true discoveries without compromising error control.

The authors remain modest regarding the scope of the retroactive updates, noting that while CPQD is a mild assumption, it is strictly required for the SCORE+ variants, whereas the basic SCORE algorithms remain valid under the minimal conditional e-validity assumption alone.