Diagnostic Tools for Extreme Value Regression Models

Title: Diagnostic Tools for Extreme Value Regression Models

Problem Statement
Extreme value regression models are essential for extrapolating beyond observable ranges of response variables, particularly in fields like hydrology, climate science, and offshore engineering. A critical step in statistical modelling is assessing how well a model fits observed data; however, if a model fails to agree with the distribution of sub-asymptotic extreme events, confidence in extrapolation is compromised. While visual and quantitative diagnostics exist for stationary univariate extreme value models (e.g., PP and QQ plots), extending these to regression settings presents significant challenges.

In regression, the conditional distribution $Y|X$ varies across the covariate domain $X$. Standard diagnostic plots rely on ordering observations, but in non-stationary settings, ordering the response variable does not induce an ordering of the conditional probability integral transform (PIT) values. Consequently, existing methods often require partitioning the covariate domain into regions to assess fit. However, this approach suffers from several limitations:

1. Scalability: As the number of regions increases, manually inspecting separate plots for each becomes infeasible.
2. Inconsistency: Regions often have varying sample sizes, and the shape and scale of the conditional tails vary, making it difficult to compare diagnostics consistently.
3. Global vs. Local: Global diagnostics (aggregating all data) can mask poor performance in specific local or regional subsets of the covariate space.
4. Bias: Existing regional diagnostics, such as worm plots, rely on asymptotic approximations that are inaccurate for extreme order statistics and are sensitive to sample size and plotting position choices.

Methodology
The authors propose two novel visual diagnostics and a suite of summary statistics designed to assess goodness-of-fit for extreme value regression models (parametric, semi-parametric, or non-parametric) across the entire covariate domain. The core methodology relies on the conditional PIT, $P_x = F_{Y|X}(Y|x)$, which is uniformly distributed on $[0,1]$ under the true model.

1. Standardised Tail Plot:

   • Concept: This is a detrended Quantile-Quantile (QQ) plot based on transforming exceedance probabilities to standard exponential margins ($Z = -\log(1-P)$).
   • Standardisation: To address varying sample sizes across regions, the authors utilize an asymptotic property of the sampling distribution of exceedance probabilities. They show that the difference between the expected value of the $k$-th exponential order statistic and the observed value ($D_k = E[Z_{(k)}] - Z_{(k)}$) converges to a log-gamma distribution that is independent of the sample size $n$ (depending only on rank $k$).
   • Visualization: The plot displays exponential residuals ($z_k - \hat{z}_k$) against rank $k$ for multiple regions overlaid on a single plot. Asymptotic confidence intervals (derived from the log-gamma distribution) allow for a consistent visual assessment of significance regardless of regional sample size.

2. Normalised Residual Plot:

   • Concept: This plot visualises deviations across all probability levels, not just the tail.
   • Transformation: It applies a probability integral transform based on the exact Beta distribution of the ordered exceedance probabilities, mapping them to a standard normal scale ($\nu_k = \Phi^{-1}(F_{\beta_k}(\hat{q}_{(k)}))$).
   • Advantage: Unlike the standardised tail plot, this does not rely on asymptotic assumptions regarding sample size and provides a visualisation of model fit significance at all probability levels.

3. Summary Statistics:

   • The authors introduce the Exponential Mean Absolute Deviation (EMAD) statistic, defined as the mean absolute deviation of the exponential order statistics. They derive its asymptotic distribution and demonstrate via sensitivity analysis that it is more sensitive to deviations in the upper tail (exceedance probabilities $< 0.155$) than the standard Anderson-Darling (AD) or right-tail-weighted AD (ADR) statistics.
   • They also utilise the Mean Square of Normalised Residuals, which is shown to be asymptotically equivalent to the Anderson-Darling statistic.
   • Regional and Global Summaries: The paper proposes aggregating regional p-values (from EMAD or ADR tests) into density histograms to assess regional uniformity, and pooling all data for global diagnostics.

Key Results and Applications
The proposed diagnostics were validated using two examples involving deep learning-based inference (Artificial Neural Networks) for extreme value regression:

1. Multivariate Extremes (SPAR Model):

   • Setup: A Peaks-Over-Threshold (POT) model was fitted to 5-dimensional simulated data with a multivariate normal copula and Laplace margins. The radial component was modelled conditional on the pseudo-angle.
   • Process: 5,000 candidate models with varying hyperparameters (thresholds, network architectures) were fitted.
   • Findings: Global diagnostics (e.g., global ADR p-values) often indicated good fit, while regional diagnostics revealed significant local failures. Specifically, one model (Model 1) showed excellent global fit but poor regional fit (masked by the global average), while another (Model 3) passed both. The diagnostics successfully identified the optimal threshold level ($\tau \approx 0.825$) and highlighted that global metrics alone are insufficient for model selection in non-stationary settings.

2. Offshore Wind Turbine Surrogate Model:

   • Setup: A probabilistic surrogate model for the extreme tension in a mooring line was conditioned on three environmental variables (wind speed, wave height, wave period).
   • Comparison: Deep GP regression models (ANNs) were compared against a simpler Generalised Additive Model (GAM) formulation.
   • Findings: While several deep learning models achieved high global p-values, their standardised tail plots revealed outliers corresponding to the boundaries of the covariate domain, indicating overfitting. In contrast, the simpler GAM formulation showed better local fit across the domain while maintaining good global performance. This demonstrated the utility of the diagnostics in guiding model design and tuning.

Significance and Claims
The paper claims that the proposed diagnostics provide a practical, scalable solution for assessing extreme value regression models where covariate domains are non-stationary and sample sizes vary. Key contributions include:

• Visual Consistency: The ability to overlay diagnostics from regions with different sample sizes on a single plot with valid uncertainty bounds, overcoming the limitations of existing regional diagnostics like worm plots.
• Tail Sensitivity: The introduction of the EMAD statistic, which offers greater sensitivity to upper-tail deviations than traditional goodness-of-fit tests.
• Actionable Insights: The diagnostics facilitate the comparison of thousands of candidate models and help identify specific regions of poor fit, supporting actionable decisions in model design (e.g., avoiding overfitting at domain boundaries).
• Generality: The tools are applicable to any extreme value regression model (threshold exceedance or block maxima) that provides estimates of conditional distribution functions, regardless of the inference scheme used.

The authors note that the workflow assumes an independent hold-out sample for evaluation to ensure valid confidence bounds, though they acknowledge that bootstrap methods could be used if such data is unavailable. The paper concludes that these tools enable a more rigorous and nuanced assessment of extreme value models than global metrics alone.