📊 statistics

Estimation of the sub-Gaussian parameter

This paper introduces and analyzes a consistent estimator for the sub-Gaussian parameter based on constrained maximization of an empirical weighted cumulant generating function, establishing convergence rates under specific assumptions on the underlying distribution's tail behavior and demonstrating its practical utility in constructing p-values for large-scale permutation tests.

Original authors: Jason Liu, Min Xu, Jinchuan Xing

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jason Liu, Min Xu, Jinchuan Xing

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the "worst-case scenario" for a group of random events. In statistics, we often deal with things that fluctuate randomly, like the daily temperature or the score of a game. Most of the time, these things stay close to the average. But sometimes, they spike wildly.

This paper is about measuring exactly how wild those spikes can get. Specifically, the authors are trying to estimate a number called the sub-Gaussian parameter (let's call it the "Wildness Score").

The Problem: The "Unstable Ruler"

To measure this Wildness Score, the authors use a mathematical tool that acts like a ruler. However, this ruler has a glitch: if you try to use it to measure extremely rare, extreme events (the far ends of the ruler), it starts to shake and give you nonsense numbers. It's like trying to measure the height of a skyscraper with a tape measure that stretches and snaps when you pull it too hard.

If you just try to measure the whole range, your estimate becomes unreliable.

The Solution: The "Safe Zone" Strategy

The authors' main idea is simple: Don't measure the whole ruler.

Instead, they propose a "Safe Zone" strategy. They say, "Let's only measure the part of the ruler that is stable and reliable." They set a boundary (a cutoff point) and ignore anything beyond it.

The Truncation: They cut off the extreme, shaky part of the data.
The Result: By ignoring the "noise" at the very edge, they can get a very accurate estimate of the Wildness Score for the part of the data that matters most.

They prove that if the data behaves "nicely" (meaning the wild spikes aren't too wild), this safe-zone method works perfectly. In fact, it gets more accurate as you collect more data, just like you'd expect a good measurement tool to do.

The "Fundamental Difficulty"

The paper also explores a deep question: Is it possible to measure this Wildness Score perfectly for ANY kind of data?

The answer is no, not without making some assumptions.

The "Impossible" Case: If the data can be infinitely wild (like a distribution with no limit on how big a spike can be), the authors prove that no method can ever give you a consistent answer. It's like trying to guess the maximum height of a mountain range that keeps growing taller every time you look.
The "Possible" Case: If you assume the data has a limit (even if that limit is very high), the problem becomes solvable. The paper shows a sliding scale: the more you know about how the data behaves at the edges, the faster and more accurately you can estimate the score.

What if the Data is "Broken"?

The authors also checked what happens if the data is not sub-Gaussian (i.e., if the "Wildness Score" doesn't exist because the spikes are too crazy).

They found that their estimator acts like a smoke alarm. If the data is too wild, the estimator doesn't just give a wrong number; it blows up to infinity. This is actually a good thing! It tells the user, "Hey, the assumptions we made are broken; this data is too crazy for this model."

Real-World Application: The "Gene Hunt"

The paper applies this method to a specific real-world problem: Gene Ontology (GO) enrichment studies.

Imagine you are a detective trying to find which biological processes are "active" in a disease. You have thousands of suspects (genes) and you run a massive simulation (permutation test) to see if they are significant.

The Old Way: You count how many times your simulation produced a result as extreme as the real one. If you only run 1,000 simulations, the smallest probability you can find is 1 in 1,000. This is too "coarse" to catch subtle but important signals.
The New Way: The authors use their "Wildness Score" estimator to smooth out the data. Instead of just counting, they use the math to predict the probability of even rarer events.
The Comparison: They compared their method to another popular technique called "Peaks-over-Threshold" (POT), which tries to fit a curve to the extreme spikes.
- The Result: Their method worked better in some cases, and the POT method worked better in others. They are like two different tools in a toolbox: sometimes you need a hammer, sometimes a screwdriver. The authors show that their method is a reliable alternative, especially when the other method is shaky or when you don't have enough data to fit a complex curve.

Summary

In short, this paper introduces a smarter way to measure how "wild" random data can get.

The Trick: Ignore the unstable, extreme edges of the data to get a stable measurement.
The Limit: You can't measure this perfectly if the data is infinitely wild, but if it's bounded, the method is optimal.
The Warning: If the data is too wild, the method screams "I'm out of my depth!" by shooting to infinity.
The Use: It helps scientists find important genetic signals in large-scale experiments where traditional counting methods are too blunt.

Problem Statement

The paper addresses the statistical estimation of the sub-Gaussian parameter (also known as the variance proxy), denoted as $\xi^2_*$ , for a mean-zero random variable $X$ . A random variable is sub-Gaussian if there exists $\xi^2 > 0$ such that $E[e^{\lambda X}] \leq e^{\lambda^2 \xi^2 / 2}$ for all $\lambda \in \mathbb{R}$ . The parameter $\xi^2_*$ is defined as the smallest such $\xi^2$ , or equivalently, as the supremum of the function $L(\lambda) = \frac{2}{\lambda^2} \log E[e^{\lambda X}]$ over $\lambda \in \mathbb{R}$ (with $L(0) = \text{Var}(X)$ ).

Despite the ubiquity of sub-Gaussian variables in machine learning and statistics, the estimation of $\xi^2_*$ has received little attention. The primary motivation for this work is large-scale permutation testing in Gene Ontology (GO) enrichment studies. In these settings, the number of hypotheses often exceeds the number of available permutations ( $M$ ), rendering empirical p-values (bounded below by $1/(M+1)$ ) powerless after multiple testing corrections. If the null distribution is known to be sub-Gaussian, one can estimate $\xi^2_*$ to utilize sub-Gaussian tail bounds, yielding much smaller p-values and recovering statistical power.

Methodology

The authors propose a natural estimator based on the constrained maximization of the empirical analogue of $L(\lambda)$ .

Empirical Function: Given i.i.d. samples $X_1, \dots, X_n$ , the empirical function is defined as:
$L_n(\lambda) = \frac{2}{\lambda^2} \log \left( \frac{1}{n} \sum_{i=1}^n e^{\lambda(X_i - \bar{X})} \right)$
for $\lambda \neq 0$ , and $L_n(0) = \frac{1}{n} \sum (X_i - \bar{X})^2$ . Centering by the sample mean $\bar{X}$ is critical to avoid instability as $\lambda \to 0$ .
Truncation: Directly maximizing $L_n(\lambda)$ over $\mathbb{R}$ leads to excessive estimation error due to the unstable behavior of $L_n(\lambda)$ for large $\lambda$ (e.g., of order $\sqrt{\log n}$ ). To mitigate this, the authors define the estimator:
$\hat{\xi}^2_n = \sup_{|\lambda| \leq C_n} L_n(\lambda)$
where $C_n$ is a slowly diverging sequence, specifically chosen as $C_n = (\log n)^\alpha$ for $\alpha \in (0, 1/2)$ .
Truncation Gap: The analysis relies on the truncation gap function $\delta(C) = \sup_{|\lambda| \geq C} L(\lambda) - \sup_{|\lambda| \leq C} L(\lambda)$ . This function measures the bias introduced by truncation and is shown to govern the fundamental difficulty of the estimation problem.

Key Contributions and Results

1. Consistency and Convergence Rates
The authors prove that $\hat{\xi}^2_n$ is consistent for any sub-Gaussian distribution. The rate of convergence depends on the decay of the truncation gap $\delta(C)$ :

General Case: If there exists $C_0$ such that $\delta(C_0) \leq 0$ (meaning the global supremum is attained within a bounded region), the estimator achieves a rate of $O_p(n^{-1/2 + \varepsilon})$ for any $\varepsilon > 0$ .
Strictly Bounded Case: If $\delta(C_0) < 0$ , the rate improves to the parametric rate $O_p(n^{-1/2})$ .
Asymptotic Normality: Under stronger assumptions (unique maximizer $\lambda^*$ , negative second derivative, and $\delta(C_0) < 0$ ), the authors establish that $\sqrt{n}(\hat{\xi}^2_n - \xi^2_*)$ converges in distribution to a normal random variable, allowing for the construction of asymptotic confidence intervals.

2. Minimax Risk and Fundamental Limits
The paper establishes that the difficulty of estimating $\xi^2_*$ is intrinsically linked to the tail behavior of $\delta(C)$ .

Impossibility of Uniform Consistency: Without assumptions on the decay of $\delta(C)$ (i.e., over the class of all sub-Gaussian distributions), the minimax risk is lower bounded by $\Omega(1)$ . This implies that no estimator can be uniformly consistent over all sub-Gaussian distributions.
Continuum of Rates: By imposing polynomial decay constraints on $\delta(C)$ , the authors derive a continuum of minimax lower bounds ranging from $\Omega(1/\log n)$ (for distributions supported on $[-1, 1]$ ) to $\Omega(1)$ .
Optimality: When $\delta(C)$ eventually becomes strictly negative (implying the supremum is attained in a bounded region), the minimax risk is $\Omega(n^{-1/2})$ . In this regime, the proposed estimator $\hat{\xi}^2_n$ is shown to be minimax optimal and adaptive (it does not require prior knowledge of the bounds).

3. Behavior Under Misspecification
If the underlying distribution is not sub-Gaussian, the authors prove that $\hat{\xi}^2_n$ diverges almost surely to infinity. The rate of divergence is controlled by the tail heaviness of the distribution:

For heavy-tailed distributions (e.g., Laplace or $t$ -distributions), the estimator diverges at rates such as $\Omega_p(\log n)$ or $\Omega_p(n^{2/\nu}/\log n)$ , providing a diagnostic signal for model misspecification.

4. Application to Gene Ontology (GO) Enrichment
The estimator is applied to construct p-values for large-scale permutation tests in GO enrichment analysis. The authors compare their sub-Gaussian approach against the Peaks-over-Threshold (POT) method, which fits a Generalized Pareto Distribution to exceedances.

Results: On the "Fly Transcriptome" dataset, the sub-Gaussian method yielded more empirically validated GO terms than POT under the PAPER algorithm. On the "TNFa" dataset, POT performed better.
Conclusion: The methods are complementary. The POT method can suffer from slow convergence rates and sensitivity to threshold selection, while the sub-Gaussian method is more reliable for light-tailed distributions but may fail if the data is heavy-tailed.

Significance and Claims

The paper claims that the truncation gap $\delta_P$ is the fundamental quantity determining the estimability of the sub-Gaussian parameter.

Theoretical Insight: It demonstrates that uniform consistency is impossible without restricting the class of distributions, and that the proposed estimator achieves the optimal minimax rate under conditions where the supremum of $L(\lambda)$ is attained in a bounded region.
Practical Utility: The estimator provides a viable alternative to empirical p-values in permutation testing when the number of permutations is limited, provided the null distribution is sub-Gaussian. It also offers a diagnostic tool: if the estimator diverges, it suggests the sub-Gaussian assumption is violated.

The authors modestly note limitations, including the difficulty of verifying the necessary hypotheses (e.g., the existence of a bounded maximizer) from data in practice, the presence of an $\varepsilon$ term in the convergence rate that may not be sharp, and the lack of a data-driven method for selecting the truncation parameter $C_n$ . They identify closing the gap between upper and lower bounds for certain subclasses as an open problem for future work.