📊 statistics

E-values as statistical evidence: A comparison to Bayes factors, likelihoods, and p-values

This paper advocates for e-values and e-processes as robust measures of statistical evidence, arguing that they synthesize the desirable properties of p-values, likelihood ratios, and Bayes factors while offering unique advantages in handling composite hypotheses, optional stopping, and intuitive wealth-based interpretations.

Original authors: Ben Chugg, Aaditya Ramdas, Peter Grünwald

Published 2026-03-26

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ben Chugg, Aaditya Ramdas, Peter Grünwald

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. You have a suspect (the Null Hypothesis, or "the innocent person"), and you are gathering clues (data) to see if they are actually guilty.

For decades, statisticians have argued over the best way to measure "how guilty" the suspect looks based on the clues. The paper you provided introduces a new, powerful tool to this debate called the E-value (and its time-traveling cousin, the E-process).

Here is a simple breakdown of the paper's main ideas, using everyday analogies.

1. The Old Tools: Why They Are Frustrating

Before the E-value, detectives had two main tools, and both had annoying flaws:

The P-value (The "One-Shot" Snapshot):
- How it works: You take all your clues at once and ask, "How unlikely is this if the suspect is innocent?"
- The Flaw: It's very rigid. If you decide to stop looking for clues early because you found something promising, or if you decide to keep looking because you didn't find anything yet, the P-value breaks. It's like a camera that only works if you promise before the photo is taken exactly how many photos you will take. If you change your mind mid-shoot, the photo is ruined.
The Bayes Factor (The "Opinionated" Judge):
- How it works: You start with a "gut feeling" (a prior belief) about the suspect's guilt and update it as you see clues.
- The Flaw: It depends entirely on your starting gut feeling. Two detectives with different gut feelings can look at the same clues and come to completely different conclusions. It's not "objective" enough for some.
The Likelihood Ratio (The "Comparison" Tool):
- How it works: It compares how likely the clues are under "Guilt" vs. "Innocence."
- The Flaw: It struggles when you don't have a clear "Guilty" scenario to compare against. What if you just want to know if the suspect is innocent, without having a specific "Guilty" story ready?

2. The New Tool: The E-Value (The "Betting Score")

The authors propose the E-value. To understand it, imagine a casino game.

The Setup: You are betting against the "House" (the Null Hypothesis/Innocence).
The Rule: The House claims the game is fair. If the House is telling the truth, you should never expect to make money in the long run. Your average winnings should be zero or less.
The E-value: This is your current bankroll.
- You start with $1.
- Every time you get a new clue, you place a bet.
- If the House is truly innocent, your bankroll will stay around $1 or go down.
- But, if the House is actually lying (the suspect is guilty), your bankroll will start to explode. It might go from $1 to $10, then $100, then $1,000.

Why is this cool?

It's Flexible (Optional Stopping): You can stop the game whenever you want! If your bankroll hits $100, you can cash out and say, "I'm done, the House is lying!" It doesn't matter if you planned to play for 10 rounds or 100. The math holds up.
It's Additive (Combining Evidence): Imagine you play a game with a friend. You both bet against the same House. If you win $10 and your friend wins $10, you can just multiply your scores ($10 $\times$ $10 =$ 100) to get the total evidence. You don't need to worry about whether your bets were independent or dependent; the math just works.
It's Objective: You don't need a "gut feeling" about the start. You just need a strategy that guarantees you won't get rich if the suspect is innocent.

3. The "E-Process" (The Time-Traveling Detective)

Sometimes, you don't just get one batch of clues; you get them one by one over time.

An E-value is like a snapshot of your bankroll at a specific moment.
An E-process is the entire history of your bankroll as you play the game round after round.
The magic of the E-process is that it allows you to change your mind about when to stop. You can say, "I'll keep betting until my bankroll hits $50," or "I'll stop if I see three red clues in a row." The E-process guarantees that if the suspect is innocent, you will almost never hit that $50 mark.

4. The Big Comparison

The paper compares these tools against a checklist of what a "good" measure of evidence should be. Here is the verdict:

5. The Catch (Limitations)

Is the E-value perfect? Not quite.

Too Many Choices: Just like there are many ways to bet in a casino, there are many ways to calculate an E-value. Different statisticians might choose different "betting strategies" and get different numbers. However, the paper argues this is no worse than the P-value (where you can choose different tests) or the Bayes Factor (where you choose different priors).
It's a "Generalized" Likelihood: The authors show that E-values are basically a super-charged version of the Likelihood Ratio. They keep the good parts of the Likelihood Ratio but fix the parts that break when you have messy, complex data.

The Bottom Line

The paper argues that E-values are the "Swiss Army Knife" of statistical evidence.

They have the flexibility of a betting game (you can stop anytime).
They have the objectivity of frequentist math (no gut feelings needed).
They have the power to combine evidence from many different sources easily.

While they don't solve every philosophical problem in statistics, they offer a very intuitive, robust, and practical way to say, "The evidence against this hypothesis is strong," without getting tripped up by the rigid rules of the past.

1. Problem Statement

The philosophy of statistics has long been divided regarding the definition of a valid "measure of statistical evidence." The three dominant candidates are:

P-values: Widely used but criticized for lack of consistency, sensitivity to sample size, and inability to handle optional stopping or composite hypotheses without ad-hoc corrections.
Likelihood Ratios (LR): Objective and intuitive but struggle with composite hypotheses (requiring generalizations like the Generalized Likelihood Ratio, GLR) and do not inherently provide long-run error rate guarantees.
Bayes Factors: Provide a coherent framework for comparing hypotheses but rely on subjective priors (in the subjective tradition) or complex objective priors that may violate the Likelihood Principle (in the objective tradition). They often lack strict frequentist error control.

The authors argue that E-values (and their sequential analogues, E-processes) represent a missing candidate that can synthesize the desirable properties of these three traditions while avoiding their specific pitfalls.

2. Methodology and Definitions

Core Definitions

E-variable ( $E$ ): A non-negative random variable such that its expectation under any distribution $P$ in the null hypothesis set $\mathcal{H}$ is at most 1:
$\sup_{P \in \mathcal{H}} \mathbb{E}_P[E] \leq 1$
Large values of $E$ constitute evidence against $\mathcal{H}$ .
E-process ( $E_t$ ): The sequential analogue of an e-variable. It is a sequence of non-negative random variables $(E_t)_{t \geq 1}$ adapted to a filtration such that for any stopping time $\tau$ (data-dependent stopping rule), $\mathbb{E}_P[E_\tau] \leq 1$ .
The Betting Game Interpretation: E-statistics are interpreted as the accumulated wealth of a bettor playing against nature. If the null hypothesis is true, the bettor cannot expect to make money (wealth $\leq 1$ ). If the null is false, a well-designed e-statistic allows the wealth to grow exponentially.

Evaluation Framework

The authors evaluate e-statistics against 21 evidential desiderata derived from three statistical traditions:

Error-Probability (Frequentist): Focus on controlling Type I/II errors and long-run error rates.
Bayesian Confirmation: Focus on updating beliefs via priors and posterior odds.
Likelihood: Focus on the relative support between hypotheses and the Likelihood Principle.

The criteria are categorized into:

Static Criteria: Evaluated on a fixed batch of data (e.g., coherence, consistency, scale invariance).
Dynamic Criteria: Evaluated in sequential or counterfactual settings (e.g., optional stopping, accumulation across studies, independence from unobserved data).

3. Key Contributions

A. Theoretical Synthesis

The paper demonstrates that e-statistics act as a "unifying bridge":

They generalize Likelihood Ratios: For simple hypotheses, the likelihood ratio is an e-variable. E-statistics preserve the "fairness" property ( $\mathbb{E} \leq 1$ ) even in composite settings where standard GLRs fail.
They offer Frequentist Guarantees: Unlike Bayes factors, e-statistics provide strict, non-asymptotic Type I error control via Markov's and Ville's inequalities, regardless of the stopping rule.
They accommodate Bayesian Priors: Through the "method of mixtures," e-statistics can incorporate priors (recovering Bayes factors in specific cases) while retaining frequentist validity.

B. Dynamic Validity (Optional Stopping and Continuation)

A primary contribution is the rigorous handling of optional stopping and optional continuation.

P-values fail here: Changing the sample size based on interim results invalidates the p-value (the "p-hacking" problem).
E-processes succeed: By definition, $\mathbb{E}[E_\tau] \leq 1$ for any stopping time $\tau$ . This allows researchers to stop an experiment early if evidence is strong or continue if it is weak, without inflating Type I error rates.
Accumulation: E-statistics allow for the multiplication of evidence across independent studies (even with flexible designs where the decision to run the next study depends on previous results), a property not shared by p-values or standard Bayes factors.

C. Taxonomy of Counterfactuals

The authors distinguish between Inter-experiment and Intra-experiment counterfactuals:

Inter-experiment: Does the evidence depend on whether a future study would have been run? (E-statistics are immune; p-values are not).
Intra-experiment: Does the evidence depend on the stopping rule or censoring mechanism that was not triggered? (E-processes and Likelihood Ratios are immune; p-values are not).

4. Results: Comparative Analysis

The paper provides a comprehensive comparison (summarized in Table 1 of the text) of P-values, Likelihood Ratios (LR), Generalized Likelihood Ratios (GLR), Bayes Factors (BF), and E-statistics.

Desideratum	P-values	LR / GLR	Bayes Factors	E-statistics
Scalar Measure	Yes	Yes	Yes	Yes
Continuous	Yes	Yes	Yes	Yes
Analyst Irrelevance	Partial (depends on test choice)	Yes	No (Subjective) / Yes (Objective)	Partial (depends on betting strategy)
Coherence	No	Yes (GLR)	Yes	Partial (depends on construction)
Consistency	No	Yes	Yes	Yes (many constructions)
Sample Size Invariance	No	Yes	Yes	Yes
Single Hypothesis Validity	Yes	No	No	Yes
Long-run Error Rates	Yes	No (GLR)	No	Yes (via Ville's inequality)
Composite Hypotheses	Yes	Partial (GLR issues)	Yes	Yes (natively)
Optional Stopping	No	Yes (LR) / No (GLR)	Yes (if priors fixed)	Yes (E-processes)
Counterfactual Independence	No	Yes (LR) / No (GLR)	Yes (Subjective) / No (Objective)	Yes (E-processes)
Composability (Dependent)	Partial	No	No	Yes (Convex combinations)

Key Findings:

Superiority in Composite Settings: E-statistics handle composite and nonparametric hypotheses more naturally than GLRs or Bayes factors, which often require arbitrary priors or suffer from asymptotic issues.
Robustness to Stopping Rules: E-processes are the only measure that simultaneously satisfies the Likelihood Principle (in a loose sense), provides strict error control, and allows for fully flexible data-dependent stopping.
Interpretability: The "wealth" interpretation provides an intuitive, game-theoretic justification for evidence that is absent in p-values (which are probabilities of tail events) and less intuitive than Bayes factors for non-Bayesians.

5. Significance and Limitations

Significance

Resolution of the "P-value Crisis": E-statistics offer a rigorous alternative to p-values that retains the ability to test a single hypothesis without an explicit alternative, while fixing the major flaw of sensitivity to stopping rules.
Bridging Schools of Thought: They unify Frequentist error control with Bayesian flexibility (via mixtures) and Likelihoodist intuition.
Practical Utility: They are already driving progress in multiple testing, changepoint detection, and mean estimation (e.g., bounded mean testing), allowing for "anytime-valid" inference where data collection can be continuous and adaptive.

Limitations and Counterarguments

Non-Uniqueness: Unlike the simple likelihood ratio, there is no single "optimal" e-variable for a given problem. The choice of e-statistic depends on the analyst's betting strategy (e.g., log-optimality vs. other criteria). While this mirrors the subjectivity of priors in Bayes factors, it introduces a new layer of choice for the analyst.
Birnbaum's Theorem: E-statistics occupy a middle ground. They are closely related to likelihood ratios (any e-variable is bounded by a likelihood ratio) but are not strictly defined by the Likelihood Principle. This may satisfy frequentists but leave strict likelihoodists unsatisfied.
Intra-experiment Counterfactuals: While e-processes handle optional stopping, the paper notes that fully satisfying the intra-experiment counterfactual desideratum (independence from unobserved censoring mechanisms) for general e-processes remains an open theoretical question, though it holds for likelihood-ratio-based e-processes.

Conclusion

The authors conclude that e-statistics are a compelling, interpretable, and intuitive measure of statistical evidence. They combine the best features of p-values (single-hypothesis validity, error control), likelihood ratios (objectivity, consistency), and Bayes factors (coherence, handling of composite hypotheses), while uniquely solving the problem of optional stopping and data-dependent experimental design. The paper advocates for their serious consideration as a standard tool in the statistical toolkit.