Return-to-Baseline Testing via Empirically Calibrated e-processes

Technical Summary: Return-to-Baseline Testing via Empirically Calibrated e-processes

Problem Formulation
The paper addresses the problem of detecting a "Return to Baseline" (RtB) in high-frequency monitoring data. The setting involves a subject-specific baseline distribution $P$ from which $n$ observations are collected. An intervention occurs, temporarily altering the data-generating distribution to an unknown $P_i$ for a duration $r$. The goal is to sequentially identify the time point at which the distribution realigns with the pre-intervention baseline $P$.

This problem is distinct from classical sequential testing (e.g., Wald's likelihood ratio test) or change-point detection (e.g., CUSUM) because:

1. The baseline distribution $P$ is unknown and must be estimated from a finite sample, making the null hypothesis composite.
2. The inferential goal is to certify a return to a previously observed distribution rather than detecting a deviation from a fixed null model.
3. The procedure must remain valid under continuous monitoring and optional stopping (anytime-valid inference), without requiring a pre-specified horizon.

Methodology
The authors propose a sequential, distribution-free testing procedure based on e-processes, which provide anytime-valid error control via non-negative super-martingales. The methodology proceeds as follows:

1. Data Model: The baseline data is modeled as quasi-periodic. Observations are collected at times $t_{i,j}$ within periods, where the conditional quantile functions $f^{(\tau)}(t)$ exhibit periodicity. Under the intervention, the distribution may change arbitrarily, potentially losing this periodicity.
2. Discrepancy Measure: The method employs universal inference concepts. It estimates conditional quantile functions $\hat{f}^{(\tau)}$ from the baseline data using quantile regression with B-splines. For a new observation vector $x$, a discrepancy statistic $T^{(\tau, -k)}(x, S)$ is defined based on the count of points exceeding the estimated quantiles, compared against the expected count $(1-\tau)|S|$.
3. Empirical Calibration: Unlike traditional e-processes that rely on parametric null models, this method calibrates the e-process empirically using only the baseline data.
   • A random subset of indices $Z$ is selected to compute a maximal discrepancy $F(Z)$.
   • A norming constant $D_\gamma$ is derived as a sample quantile of the distribution of these discrepancies computed over the baseline data (using a "leave-one-out" approach where each baseline period is held out for validation).
   • This calibration ensures the process is subject-specific and does not require specifying a null sub-model.
4. The E-Process Construction: The e-process $M_t$ is constructed as a running minimum of scaled discrepancy terms.
   • $M_t = \frac{1}{n} \sum_{k=1}^n M^{(-k)}_t$, where each $M^{(-k)}_t$ is a running minimum.
   • The running minimum structure is chosen specifically to ensure that once data from the null (baseline) distribution is observed, the process drops immediately, allowing for detection of RtB without the lag associated with running averages or products.
   • The process is stopped when $M_t$ falls below a threshold $1/\alpha$, certifying an RtB.

Key Contributions

• Formulation: The RtB problem is explicitly formulated as a sequential testing task for high-frequency data with unknown baseline distributions and optional stopping.
• Empirically Calibrated E-Process: A novel distribution-free procedure is proposed where the e-process is calibrated solely on baseline data, avoiding the need for parametric null models.
• Finite-Sample Bounds: The paper establishes finite-sample bounds for the calibration error (Theorem 1). These bounds depend on the amount of baseline data ($n, p$), the smoothness of the quantile functions ($\beta$), the number of spline knots ($d$), the subsampling size ($s$), and a gap parameter ($\delta$). The bounds clarify the trade-offs between conservativeness and power.
• Computational Analysis: The computational complexity is analyzed, particularly regarding the choice of subsample size $s$ and the number of quantile levels.

Results

• Simulations: The procedure was tested on synthetic data with varying numbers of baseline periods ($n$), observations per period ($p$), and subsample sizes ($s$).
  • The method accurately detected the RtB with high probability (e.g., ~90% of simulations detected the RtB with 0 or 1 unit of delay).
  • It was shown to be robust to the choice of spline degrees of freedom ($df$).
  • Comparisons with alternative methods (calibrated p-values converted to e-values and sequential Bonferroni) showed that the proposed empirically calibrated e-process outperforms the calibrated e-value approach (which loses power due to the calibrator function) and matches the Bonferroni approach while providing anytime-valid guarantees without a pre-specified horizon.
• Clinical Case Study: The method was applied to maternal-fetal monitoring data (fetal heart rate) from the NIEM-O study following corticosteroid administration.
  • Using data from a single patient, the procedure detected a return to baseline approximately 35 hours after administration.
  • The analysis was conducted across multiple temporal resolutions (sub-segments of 15 to 50 minutes), demonstrating the method's multi-scale capability.
  • The results were consistent with pharmacokinetic evidence and coarser analyses in existing literature, but offered finer temporal resolution and subject-specific inference.

Significance and Claims
The paper claims that the proposed procedure successfully meets the stringent requirements of modern high-frequency monitoring:

1. Subject-Specific: It adapts to individual baselines without relying on population-level parametric models.
2. Anytime-Valid: It provides valid Type I error control under continuous monitoring and optional stopping via the e-process framework (Ville's inequality).
3. Robustness: It is distribution-free and robust to outliers and model misspecification, as the calibration is empirical.
4. No Lag: The use of a running minimum allows for immediate detection of the return to baseline, avoiding the delay inherent in other sequential methods.

The authors note that while the theoretical bounds rely on smoothness assumptions (Hölder class) that cannot be verified in practice, the calibration step remains valid even if the model is misspecified. The numerical experiments suggest the procedure is conservative, implying the theoretical bounds may be improved in future work. The framework is also presented as extensible to multiple signals and covariates.