Estimating probabilities of multivariate failure sets based on pairwise tail dependence coefficients

Technical Summary: Estimating Probabilities of Multivariate Failure Sets Based on Pairwise Tail Dependence Coefficients

Problem Statement
Estimating the probability that a multivariate random vector $X$ falls into an extreme "failure region" $C$ is a critical challenge in fields such as finance and climate science. A primary difficulty arises when failure regions contain few or no observed data points, rendering non-parametric estimation infeasible. While parametric models offer a solution, they face a dilemma: models with few parameters often oversimplify complex tail dependence structures, whereas models with many parameters become computationally intractable as the dimension $d$ increases (the "curse of dimensionality"). Existing semi- or non-parametric approaches are generally limited to low dimensions ($d \ge 5$) or narrow definitions of failure regions. This paper addresses the need for a general approach to estimate $P[X \in C]$ for moderate to high dimensions where $C$ is user-defined, relying solely on pairwise tail dependence information.

Methodology
The proposed approach utilizes the Tail Pairwise Dependence Matrix (TPDM), denoted $\Sigma_X$, which summarizes the tail dependence coefficients for all pairs of components in $X$. The methodology proceeds through the following steps:

1. Generalization of the TPDM: The authors generalize the TPDM definition (originally proposed for $\alpha=2$) to multivariate regularly varying vectors with any tail index $\alpha > 0$. They establish that the TPDM is invariant to the choice of norm and the specific tail index, and can be related to the angular measure of the vector.
2. Max-Linear Model Construction: The core of the method relies on the fact that any TPDM is a completely positive matrix. This implies the existence of a non-negative matrix $A$ such that $\Sigma_X = A A^T$. The authors propose an efficient iterative algorithm to obtain an approximate completely positive decomposition $\Sigma_X \approx A A^T$, where $A$ is a $(d \times d)$ matrix with non-negative elements. This corresponds to constructing a max-linear model $Y = A \times \max Z$, where $Z$ consists of independent Fréchet random variables.
   • The algorithm iteratively reduces the dimension of the TPDM by selecting a "path" of indices.
   • If the decomposition is exact, the resulting max-linear model matches the original TPDM perfectly.
   • If the decomposition is approximate (which occurs when the path does not yield an exact factorization), the constructed model satisfies $\Sigma_X \preceq \Sigma_Y$. For certain failure regions (specifically those defined by non-decreasing functions), this property ensures that the estimated failure probability is a conservative overestimate of the true probability.
3. Handling Non-Regular Variation: The method extends beyond multivariate regular variation by assuming the vector belongs to the max-domain of attraction of a multivariate Generalized Extreme Value (GEV) distribution. In such cases, marginal standardization is applied to transform the data to a common Fréchet(2) margin, allowing the TPDM-based approach to proceed.
4. Statistical Estimation: The TPDM is estimated from data using intermediate order statistics (Hill estimator for tail indices and empirical angular measures for the matrix entries). The decomposition algorithm is then applied to the estimated TPDM to recover the parameter matrix $\hat{A}$. Failure probabilities are subsequently estimated either analytically (for specific regions like sums or maxima) or via Monte Carlo simulation from the constructed max-linear model.

Key Contributions

• Algorithmic Efficiency: The paper introduces a computationally efficient algorithm to obtain approximate completely positive decompositions of the TPDM. Unlike existing methods that are computationally intensive or limited to low dimensions, this approach scales to moderate dimensions (e.g., $d \approx 30$).
• Conservative Estimation: The authors demonstrate that when an exact decomposition is not possible, the resulting approximation provides a conservative estimate for failure probabilities of specific regions, a desirable property in risk management.
• Flexibility: The method accommodates user-defined failure regions and does not require the underlying distribution to be strictly multivariate regularly varying, provided marginal standardization is feasible.
• Theoretical Properties: The paper establishes the invariance properties of the generalized TPDM and proves that the decomposition is exact if the underlying vector follows a max-linear model with a triangular coefficient matrix.

Results
The proposed method was evaluated through numerical experiments and applied to two real-world datasets:

1. Numerical Experiments: Simulations using max-linear, logistic, and Brown–Resnick models showed that the estimated failure probabilities closely matched the true values, even when the data-generating process differed from the max-linear assumption. The variability of estimates was generally low, and approximate decompositions yielded results nearly identical to exact ones.
2. Financial Industry Portfolios: Applied to daily negative log-returns of 30 industry portfolios, the method successfully estimated Value-at-Risk (VaR) and other failure probabilities for weighted sums and minimums of portfolio returns. The results were consistent with empirical estimates, and the computational time for approximate decompositions was significantly lower than for exact ones.
3. Wind Gusts in the Netherlands: Applied to daily maximal wind gusts at 35 weather stations, the method handled data that required marginal standardization due to non-regular variation. The analysis revealed that failure probabilities for coastal stations were significantly higher (up to six times) than for inland stations, despite similar pairwise dependence coefficients, highlighting the method's ability to capture complex multivariate structures.

Significance and Claims
The authors claim that this work provides a practical, general framework for estimating multivariate failure probabilities in moderate to high dimensions where traditional parametric models are too complex and non-parametric methods fail due to data sparsity. By relying solely on the TPDM, the method balances model flexibility with computational tractability. The paper emphasizes that while summarizing tail dependence via bivariate measures entails a loss of information, numerical experiments suggest this loss has a limited impact on the accuracy of failure probability estimates. The approach is presented as a viable tool for risk assessment in scenarios involving extreme events with few historical observations.