Technical Summary: Statistical Process Control via p-values

Problem Statement

Statistical Process Control (SPC) traditionally relies on monitoring test statistics ($Z_t$) derived from specific parametric assumptions (e.g., normality) to detect deviations from an in-control (IC) state. While methods like Shewhart, CUSUM, and Exponentially Weighted Moving Average (EWMA) charts are well-established, their theoretical guarantees—specifically regarding the Average Run Length (ARL) and false alarm rates—often depend heavily on the validity of these parametric assumptions. When the underlying data-generating process (DGP) is unknown, nonparametric, or exhibits complex temporal dependence, deriving exact control limits and ARL bounds becomes difficult or requires time-consuming simulation-based calibration.

This paper addresses the challenge of constructing SPC charts that offer universal, distribution-free guarantees without assuming specific stochastic dependence structures between time points. The authors propose a framework where the monitoring statistic is a sequence of p-values ($P_t$) rather than raw test statistics, leveraging the fundamental property of p-values under the null hypothesis.

Methodology

1. Theoretical Framework: Super-uniformity

The core of the methodology rests on the super-uniformity property of p-values. A sequence of p-values $(P_t)_t$ is considered valid if, under the null hypothesis $H_0$ (the process is IC), the probability of observing a p-value less than or equal to $\alpha$ is bounded by $\alpha$ for any $\alpha \in [0, 1]$.

• One-phase and Two-phase Designs: The framework applies to both designs where $H_0$ is defined relative to a fixed distribution or a Phase I baseline sample.
• Independence Assumption: Crucially, the authors do not assume independence between $P_t$ and past observations. They only require that $P_t$ satisfies the super-uniformity condition (marginal or conditional).

2. Control Limits and Run Length Bounds

The authors analyze the standard Shewhart rule: raising an alarm when $P_t \leq \alpha$.

• Universal Lower Bounds: Without assuming temporal dependence, they derive a universal lower bound for the Average Run Length (ARL) of $ARL \geq \frac{1}{2\alpha} + \frac{1}{2}$.
• Conditional Super-uniformity: If the p-values satisfy a stronger condition where $P(P_t \leq \alpha | \mathcal{F}_{t-1}) \leq \alpha$ (conditional on past information), the bound sharpens to the familiar parametric rate of $ARL \geq \alpha^{-1}$.
• k-ARL Generalization: The paper extends these results to the k-ARL (expected time until $k$ false alarms). Under general super-uniformity, the bound is $\geq \frac{k}{2\alpha} + \frac{1}{2}$; under conditional super-uniformity, it improves to $\frac{k}{\alpha}$. This allows for calibration of charts to control the family-wise error rate over time without parametric modeling.

3. EWMA-like Schemes via Merging Functions

Standard EWMA charts ($\tilde{P}_t = \lambda P_t + (1-\lambda)\tilde{P}_{t-1}$) fail to preserve the super-uniform property when applied directly to p-values. To overcome this, the authors utilize merging functions for dependent p-values (based on Vovk & Wang, 2020).

• They construct EWMA-like statistics $Q^{(r)}_{\lambda,t}$ using weighted generalized means of powers of p-values ($P_t^r$).
• By applying specific scaling constants derived from the maximum weight, they ensure the resulting sequence remains super-uniform (or conditionally super-uniform for specific parameter choices like $r \geq 1$).
• This allows for "smoothing" of p-value charts without ad hoc control limits or simulation-based calibration.

4. Multivariate Localisation

The framework is extended to multivariate SPC for directional and coordinate localisation.

• Closed Testing: Using a closed-testing procedure (Marcus et al., 1976) combined with Holm's step-down method, the authors propose a method to detect which specific coordinates and directions (increase/decrease) have deviated from the baseline.
• Error Control: This approach controls the Family-Wise Error Rate (FWER) at the time of alarm, ensuring that the probability of making any incorrect directional or coordinate claim is bounded by $\alpha$.

Key Results

Theoretical Contributions

• Universal ARL/k-ARL Bounds: The paper provides rigorous, non-asymptotic lower bounds for run lengths that hold for any DGP satisfying the null hypothesis, regardless of dependence structure.
• Valid EWMA Constructions: The derivation of $Q^{(r)}_{\lambda,t}$ and $\bar{Q}^{(r)}_{\lambda,t}$ provides the first generic method to create smoothed p-value charts with guaranteed IC behavior, avoiding the failure of naive averaging.
• Distribution Theory: Explicit probability density functions (PDFs) and cumulative distribution functions (CDFs) are derived for EWMA processes based on independent uniform random variables, clarifying why naive uniform EWMA charts fail to be super-uniform in the right tail but succeed in the left tail under certain initializations.

Numerical and Simulation Findings

• Conservatism vs. Sharpness: Simulations on normal and autoregressive data show that while the universal bounds (without conditional assumptions) can be conservative (ratios of realized ARL to bound up to ~33), they are faithful. When conditional super-uniformity holds, the bounds are sharp.
• EWMA Performance:
  • In persistent Out-of-Control (OOC) scenarios, raw p-value charts often detect changes faster than EWMA-like charts.
  • In dynamic OOC scenarios (where the shift evolves over time), EWMA-like charts with specific parameters ($r < 0$) can detect the $k$-th alarm faster than raw p-value charts, demonstrating a trade-off where increased IC conservativeness yields better power for dynamic shifts under the $k$-alarm rule.
• Localisation: The proposed localisation procedure successfully identifies affected coordinates and directions in both normal and heavy-tailed (Cauchy) multivariate settings, maintaining FWER control at the theoretical level.

Significance and Claims

The paper claims to provide a modular, distribution-free framework for SPC. Its primary significance lies in:

1. Decoupling from Parametric Assumptions: It allows practitioners to construct SPC charts with guaranteed false alarm rates for any process where valid p-values can be generated (e.g., via Kolmogorov–Smirnov or Mann–Whitney tests), eliminating the need for simulation-based calibration tied to specific parametric models.
2. Theoretical Rigor for Smoothing: It resolves the theoretical gap regarding EWMA charts for p-values, proving that valid smoothing is possible through merging functions, thereby enabling the use of smoothing techniques without sacrificing IC guarantees.
3. Unified Multivariate Approach: It offers a principled, simultaneous error-controlled method for localizing multivariate shifts, applicable to both parametric and nonparametric settings.

The authors emphasize that while the bounds may be conservative in the absence of dependence assumptions, this is the "natural price" for achieving robustness across all possible DGPs. The framework is presented as a general tool that can be layered onto any existing p-value construction to add smoothing or localisation capabilities while retaining theoretical control.