📊 statistics

Dynamic $e$ -closure for online hypotheses with any-time-valid evidence: closure principles and projective mergers

This paper develops a dynamic $e$ -closure framework for online hypothesis testing that ensures simultaneous stopped-FDR and SupFDR control under evolving evidence and arbitrary stopping times, while characterizing the admissibility and limitations of pointwise $e$ -mergers across infinite hypothesis universes.

Original authors: Rianne de Heide

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Rianne de Heide

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective's Dilemma: Solving Mysteries That Keep Changing

Imagine you are a detective trying to solve a massive case with hundreds of suspects. In the old days, you would lock the door, gather all the clues, pick a specific list of suspects, and then start your investigation. You'd have a clear "stop" button: once you finished your report, the case was closed. But modern science is more like a bustling, 24-hour newsroom. New suspects (hypotheses) walk through the door every minute, while the evidence for the suspects already inside keeps changing. A new fingerprint might appear on a suspect who was cleared yesterday, or a witness might change their story.

The big problem for scientists is knowing when to stop and declare a "winner" without making a mistake. If you stop too early, you might catch the wrong person (a false alarm). If you stop too late, you might miss the real culprit. To handle this, scientists use a special tool called an e-value. Think of an e-value as a "suspicion score." If the score gets high enough, it means the evidence is strong enough to say, "This suspect is likely guilty." The tricky part is that in the real world, you can't just look at the score at one fixed moment; you have to be able to look at it anytime you want, even if you decide to stop the investigation based on what you see right then. This paper tackles the math of how to manage these ever-changing lists of suspects and shifting scores without losing your mind or making false accusations.

The Paper's Big Idea: The "Future-Proof" Detective

This paper, written by Rianne de Heide, introduces a new way to manage these chaotic, ever-changing investigations. The author calls it Dynamic e-closure. It's a set of rules that lets scientists keep adding new suspects and updating old evidence while still guaranteeing that their final conclusions are trustworthy, no matter when they decide to stop the investigation.

Here is how the magic works, using the analogy of a detective's notebook:

1. The "Future-Proof" Notebook (Future-Extension Coherence)
Imagine you have a notebook where you write down how "guilty" a group of suspects looks. Usually, if you add a new suspect to the group, the math gets complicated. But this paper introduces a rule called Future-Extension Coherence. It's like having a magical notebook where adding a new, unproven suspect to your list never makes the evidence for the current suspects look weaker. It ensures that if you decide to stop investigating today, your conclusion is just as valid as if you had planned to stop today from the very beginning. This solves the problem of "random stopping," where a detective might accidentally pick a moment to stop that makes the evidence look better than it really is.

2. The "Never-Forget" Rule (Setwise Persistence)
Sometimes, a detective might find a group of suspects that looks guilty, but then later, new evidence comes in that makes them look innocent again. That's a disaster if you already announced they were guilty! This paper introduces a way to make sure that once a group is certified as "guilty" (statistically significant), it stays certified even as new evidence rolls in. This is called Setwise Persistence. It's like a detective's badge that, once earned, can never be taken away, even if the case gets messier.

3. The "One-Size-Fits-All" Weight (The Rigidity of Merging)
To build these notebooks, the author had to figure out how to combine the suspicion scores of different suspects into a group score. The paper proves a surprising and strict rule: you cannot just make up new weights for every new group of suspects. If you want your method to work for any possible group size and any stopping time, you are forced to use a single, pre-determined "weight" for every suspect that never changes.

The Catch: If you try to be "fair" by giving every suspect the exact same weight (symmetry) in an infinite universe of suspects, the math forces you to give everyone a weight of zero. In other words, you can't have a perfectly fair, symmetric system that works for an endless stream of new suspects without breaking the rules. You have to accept that some suspects get more "budget" for their evidence than others.

4. The "Shared Control" Trick
The paper also shows how to handle situations where suspects share a common piece of evidence (like a shared control group in a medical trial). Even if the suspects are connected and their evidence is tangled together, the author proves you can still build a valid suspicion score for the whole group, provided you look at the evidence through a "global lens" that sees everything happening at once.

What the Paper Rules Out

The author is very clear about what doesn't work.

You can't rely on standard rules without a plan: If you don't use these specific "coherent" rules, a clever (or unlucky) detective could stop the investigation at a random moment that makes innocent suspects look guilty. The paper proves that without coherence, there is no guarantee that error rates will stay low for every possible stopping rule; some random stops will inevitably lead to false accusations.
You can't have a perfectly fair, symmetric system for infinite suspects: As mentioned, if you try to treat every new suspect exactly the same as the old ones in an endless stream, the math forces the system to collapse into doing nothing. You have to accept a "global budget" where the total weight is fixed, and new suspects eat into the budget of the old ones.
Old rules don't survive the stop: The paper shows that a famous method called "BH" (Benjamini-Hochberg), which works great when you stop at a fixed time, fails miserably if you stop at a random time based on the data. You can't just use the old tools; you need these new dynamic ones.

How Sure Are We?

The author doesn't just guess or simulate these results; they prove them mathematically. The paper provides rigorous theorems that show:

If you follow these new rules, you are guaranteed to keep your error rates low (specifically, controlling the "False Discovery Rate" even when you stop at random times).
If you don't follow these rules (specifically, if you don't have "coherence"), you cannot guarantee that error rates will stay low for all possible stopping strategies; some strategies will inevitably fail.
The "rigidity" of the weights is a hard mathematical fact, not just a suggestion.

The paper also offers a "recipe" for scientists to build these systems. It says: "Here is how you construct the evidence, here is how you combine it, and here is how you know it works." It even provides examples, like a Gaussian (bell-curve) model for medical trials, to show that this isn't just abstract math but can be built in real-world scenarios.

The Takeaway

In a world where science is moving faster than ever, with new questions popping up and old answers changing, this paper gives us a sturdy, unbreakable framework. It tells us that while we can't predict the future or stop at a perfect moment, we can build a system that is robust enough to handle the chaos. It's like giving the detective a notebook that updates itself, never forgets a guilty verdict, and ensures that no matter when they close the case, the verdict stands up to scrutiny. The cost? You have to be strict about how you weigh your suspects, and you can't be perfectly fair to an infinite number of them, but the trade-off is a system that actually works when the clock is ticking and the suspects are multiplying.

Technical Summary: Dynamic e-closure for Online Hypotheses with Any-Time-Valid Evidence

Problem Setting
The paper addresses a dual-sequentiality problem in modern statistical testing: hypotheses may arrive over time (online setting), while evidence for existing hypotheses continues to evolve and can be inspected at arbitrary stopping times (any-time setting). This scenario arises in contexts like continuously operating platform trials, where new treatment arms are added while recruitment for existing arms continues, and shared controls induce dependence across streams.

The core challenge is to maintain valid error control (specifically False Discovery Rate, FDR) under global stopping times where both the time of stopping and the set of active true-null hypotheses are random. Classical closed testing and fixed-time e-closure fail here because the "active true-null intersection" at a stopping time is a random object, whereas standard e-process guarantees are defined for fixed intersections or fixed stopping times. Furthermore, standard online closure methods often assume evidence is frozen upon hypothesis arrival, which does not accommodate evolving evidence.

Methodology and Framework
The author develops a theory of dynamic e-closure built upon three existing pillars:

Fixed-time e-closure: Using intersection e-values to certify families of rejection sets simultaneously.
Online closure: Using predictability conditions to prevent future hypotheses from invalidating current decisions.
Global filtration validity: Ensuring e-processes remain valid under stopping times adapted to the global information flow (including shared controls).

The framework introduces a dynamic intersection e-process collection $(E^S_t)_{t \ge 0, S \subseteq I}$ , where $S$ represents an intersection hypothesis and $t$ represents the global evidence time. Two critical conditions are imposed on this collection:

Future-extension coherence: Adding hypotheses that have not yet arrived (dormant hypotheses) cannot decrease the current certificate value ( $E^{S \cap I_t}_t \le E^S_t$ ). This ensures that decisions are invariant to the choice of a larger finite future horizon.
Time monotonicity: Certificates are non-decreasing over time ( $E^S_t \le E^S_u$ for $t \le u$ ). This is required for controlling the supremum of the error rate over time.

Key Contributions and Results

Dynamic e-closure Principles:
- Horizon Invariance: The paper proves that under future-extension coherence, the closure computed from the currently active family is identical to the restriction of the closure computed at any larger finite horizon.
- Random-Intersection Localization: A key lemma shows that for a future-extension coherent collection, the certificate for the random active true-null intersection at a stopping time $\tau$ can be bounded by a certificate for a fixed terminal intersection. This allows the application of standard optional stopping theorems to random intersections.
- Forward Theorems:
  - If the collection is future-extension coherent, the resulting dynamic closure controls simultaneous stopped FDR.
  - If the collection is also time-monotone, the closure controls simultaneous SupFDR (the expected supremum of the False Discovery Proportion over time) and ensures setwise persistence (once a set is certified, it remains certified).
Necessary and Sufficient Characterization:
- The paper establishes that every procedure satisfying the stopped FDR or SupFDR criteria is contained within a dynamic closure generated by canonical normalized-loss e-processes.
- These canonical certificates are constructed by normalizing the worst-case loss profile of the procedure itself. This provides a representation theorem: dynamic e-closure is necessary and sufficient (up to containment) for these error criteria.
- The theory extends beyond FDP to bounded, monotone, local loss functions.
Admissible Mergers Across Horizons:
- The paper investigates how to construct intersection processes from coordinate e-processes using pointwise merger rules.
- Cross-Horizon Rigidity: It is shown that for a family of admissible pointwise mergers to be coherent across growing hypothesis horizons (specifically under neutral padding, where dormant hypotheses contribute a neutral value of 1), the merger weights must be restrictions of a single globally summable weight sequence.
- Symmetry Impossibility: On a countably infinite hypothesis universe, no nontrivial symmetric, admissible, arbitrary-dependence pointwise merger family exists that is coherent over an unbounded horizon. Symmetry forces all weights to zero.
- Construction: The paper provides a "globally weighted projective closure" and constructions for persistent procedures using adjusted running maxima.
Global Filtration and Shared Controls:
- The paper verifies that validity can be maintained in the presence of shared controls (which induce dependence) by constructing test supermartingales directly in the global filtration.
- It demonstrates that fixed-time PRDS (Positive Regression Dependency on a Subset) is insufficient for controlling FDR at global stopping times, even if p-values are independent at every deterministic time. A direct sequential BH procedure can fail under global stopping rules.

Significance and Claims
The paper claims to provide the first comprehensive theory for dynamic e-closure that unifies online hypothesis arrival with any-time evidence updates. Its primary significance lies in:

Resolving the Random Intersection Problem: By introducing future-extension coherence and localization, it bridges the gap between fixed-intersection e-process validity and the random nature of active hypotheses at stopping times.
Unifying Error Criteria: It shows that for setwise persistent families, the distinctions between fixed-time FDR, stopped FDR, and SupFDR collapse, simplifying the design of persistent testing procedures.
Characterizing Mergers: It rigorously characterizes the constraints on merging rules across growing horizons, proving that admissible, coherent, arbitrary-dependence mergers require a global weight budget, thereby ruling out simple symmetric or renormalized approaches in open-ended settings.
Practical Benchmarks: It establishes "closed eBH" and "closed BY" as valid batch any-time benchmarks and provides a recipe for constructing persistent procedures.

The author notes limitations, including the computational complexity of general closure (exponential in the number of active hypotheses) and the assumption of deterministic active sets. Future work is suggested for random hypothesis arrivals, history-dependent mergers, and scalable algorithms. The paper does not claim to solve all platform trial complexities (e.g., delayed outcomes, regulatory rules) but provides a foundational theoretical framework for handling the dual-sequentiality of hypothesis arrival and evidence evolution.