Heavy Tails and Predictive Ability Testing

Technical Summary: Heavy Tails and Predictive Ability Testing

Problem Statement
The paper addresses a critical flaw in the standard methodology for evaluating and comparing forecast accuracy in economics and finance, specifically the Diebold–Mariano (DM) test and the Superior Predictive Ability (SPA) test. These widely used tools rely on a Central Limit Theorem (CLT) that assumes forecast loss differentials possess finite variance. However, empirical evidence indicates that many financial variables (e.g., asset returns, exchange rates) and their derived forecast errors exhibit heavy tails, often with a tail index $\kappa < 2$, implying infinite variance.

The authors demonstrate that when loss differentials have infinite variance, the standard DM test statistic converges to a non-standard limit involving dependent non-Gaussian stable random variables rather than a Gaussian distribution. Consequently, using conventional standard normal critical values leads to severely distorted inference. In simulations, a nominal 5% test can reject a true null hypothesis up to 70% of the time, regardless of sample size. This issue is particularly acute when loss functions (such as squared error or tick loss) amplify the tail behavior of the underlying forecast errors.

Methodology
To resolve these inference failures, the paper develops a new theoretical framework and a robust inferential procedure:

1. New Stable Limit Theory: The authors derive a novel stable limit theorem for strongly mixing, infinite-variance time series processes. Unlike previous results that often rely on point process convergence, this theorem utilizes the convergence of hybrid characteristic function–Laplace transforms. It establishes that for a stationary, strongly mixing process $(X_t)$ with tail index $\kappa \in (1, 2)$, the self-normalized sum statistic $T_n = S_n / \gamma_n$ (where $S_n$ is the partial sum and $\gamma_n$ is the square root of the sum of squares) converges in distribution to a ratio $\xi_\kappa / \zeta_{\kappa/2}$. Here, $\xi_\kappa$ is a non-Gaussian $\kappa$-stable random variable and $\zeta_{\kappa/2}$ is a $(\kappa/2)$-stable random variable. The joint distribution is fully characterized by the tail index and the extremal dependence structure (spectral tail process) of the time series.

2. Subsampling-Based Inference: Building on this limit theory, the paper proposes a subsampling-based procedure to construct critical values. This method:

   • Is valid irrespective of whether the variance is finite or infinite.
   • Requires no estimation of long-run variances or tail indices.
   • Does not require knowledge of the convergence rate of the sample mean.
   • Uses subsample versions of the self-normalized statistic to approximate the limiting distribution empirically.

3. Extensions:

   • Undefined Mean ($\kappa < 1$): The authors show that the standard statistic $T_n$ does not diverge even when the mean is undefined ($E[|X_t|] = \infty$), rendering standard tests inconsistent. They propose a modified statistic $\tilde{T}_n$ that diverges under the alternative, allowing for non-trivial rejection frequencies.
   • Superior Predictive Ability (SPA): The framework is extended to the multivariate setting of Hansen's (2005) SPA test, deriving the limiting distribution for the maximum of dependent stable ratios.

Key Results

• Theoretical Limits: The limiting distribution of the test statistic under heavy tails is a ratio of dependent stable variables. This distribution is generally asymmetric and has quantiles that differ substantially from the standard normal distribution.
• Simulation Evidence: Monte Carlo experiments confirm that under heavy-tailed processes with skewness, the standard DM test suffers from massive size distortions (e.g., 70% rejection rates at a 5% level). In contrast, the proposed subsampling test maintains rejection frequencies close to the nominal level across various sample sizes and tail indices.
• Empirical Application: The authors apply the methodology to Value-at-Risk (VaR) forecasts for emerging-market foreign exchange rates (specifically the KRW/USD). The loss differentials were found to have a tail index of approximately 1.5 (infinite variance) and significant extremal skewness.
  • The standard DM test concluded that a sophisticated GARCH-based model (G-FZ) statistically dominated all nine competing models, including simple rolling-window forecasts.
  • The heavy-tail-robust subsampling test failed to reject the null of equal predictive ability between the G-FZ model and the rolling-window forecasts. This suggests that the apparent superiority of the complex model was an artifact of ignoring heavy tails and skewness in the loss differentials.

Significance and Contributions
The paper makes three primary contributions to the literature on forecast evaluation and heavy-tailed time series:

1. Limit Theory: It provides a new stable limit theorem for strongly mixing processes under infinite variance, extending the classical CLT to heavy-tailed settings with weaker assumptions than existing point-process-based results.
2. Inference Procedure: It introduces a fully data-driven subsampling procedure that yields valid inference for predictive ability tests without requiring knowledge of the tail index or long-run variance estimation.
3. Practical Impact: It demonstrates that standard forecast evaluation tools can lead to substantively incorrect conclusions in the presence of heavy tails. By accounting for heavy tails, the paper shows that simple non-parametric forecasting methods may perform as well as sophisticated parametric models, challenging the necessity of complex model specifications in certain risk management contexts.

The authors note that their work complements existing literature (e.g., Kim et al., 2021) by providing more general assumptions for the limit theory, extending the analysis to cases where the mean may not exist, and offering rigorous arguments for the validity of the subsampling approach in the context of forecast comparison.