On options-driven realized volatility forecasting: Information gains via rough volatility model

This paper demonstrates that augmenting the Heterogeneous Autoregressive (HAR) model with spot volatility estimators derived from traded options via a rough stochastic volatility model and deep learning surrogates significantly improves daily realized volatility forecasting accuracy and sustains superior performance across one-month horizons compared to traditional models and the VIX index.

The Gist

The Big Picture: Predicting the "Weather" of the Stock Market

Imagine you are a sailor trying to predict the weather for your journey. In the financial world, volatility is the weather—it's how choppy or calm the stock market is. If the market is calm, you sail smoothly. If it's stormy (high volatility), you risk capsizing.

For decades, sailors (investors) have tried to predict these storms using two main tools:

1. Looking at the past: Checking the waves from yesterday, last week, and last month. This is like looking at a logbook of past weather.
2. Looking at the clouds: Checking what the current sky looks like. In finance, this means looking at options prices (contracts that bet on future market moves). These prices contain "forward-looking" clues about what the market expects to happen.

The Problem: The old tools for reading the "clouds" (options) were a bit blurry. They missed some subtle, jagged details in how volatility actually moves.

The Solution: This paper introduces a new, sharper lens called the "Rough Volatility" model. It claims that market volatility isn't smooth like a gentle hill; it's "rough," like a jagged mountain range. By using this new lens, the authors built a better weather forecast for the stock market.

---

The Cast of Characters

To understand the experiment, let's meet the players:

• The HAR Model (The Veteran): This is the standard, reliable weatherman. It predicts tomorrow's storminess based on the average of yesterday's, last week's, and last month's storms. It's good, but it's a bit old-school.
• The VIX (The Crowd's Mood): This is a famous index that measures how scared or greedy the market feels right now. It's a great tool, but it's a "black box"—we know it's useful, but we don't know exactly why or how it calculates the future.
• The Old Models (Heston, Bates, SVCJ): These are the "classic" mathematical models that try to decode the options market. They are like trying to describe a mountain using a smooth, rounded drawing. They miss the sharp edges.
• The Rough Heston Model (The New Lens): This is the star of the show. It acknowledges that volatility is "rough" (jagged and unpredictable). It tries to capture the tiny, sharp details that the old models ignore.
• The Deep Learning AI (The Super-Computer): Here's the catch: The "Rough Heston" model is incredibly hard to calculate. It's like trying to solve a million-piece puzzle by hand. The authors used Deep Learning (AI) to build a "surrogate" or a "shortcut." Think of it as training a smart robot to memorize the answers to the puzzle so the computer doesn't have to solve it from scratch every time. This made the complex math fast enough to be useful.

---

The Experiment: Who Predicts Best?

The authors set up a race to see who could predict the stock market's volatility (the "waves") most accurately for the S&P 500 (the US stock market).

1. The Setup: They took historical data from 2011 to 2019 to train their models.
2. The Test: They tested the models on data from 2020 (a very turbulent year with the pandemic) to see who got it right.
3. The Comparison: They compared:
   • The Veteran (HAR only).
   • The Veteran + The Crowd's Mood (HAR + VIX).
   • The Veteran + Old Models (HAR + Heston/Bates).
   • The Winner: The Veteran + The New Rough Lens (HAR + Rough Heston).

The Results: Why the "Rough" Lens Won

The results were clear, and the "Rough Heston" model crushed the competition.

• Accuracy: The Rough Heston model made fewer errors than any other model. It was the most precise weatherman.
• Direction: It was better at predicting not just how big the storm would be, but which way the wind would blow (up or down). This is crucial for traders who need to know if they should put on their raincoats or take them off.
• Long-term: The advantage didn't just last for one day; it held up even when predicting a month ahead.

The Analogy:
Imagine you are trying to guess the temperature tomorrow.

• The Old Models say, "It will be 70 degrees."
• The Rough Model says, "It will be 70 degrees, but there's a sudden, jagged cold front coming in at 2 PM that the old models missed."
• Because the Rough Model caught that jagged detail, its prediction was much closer to reality.

Why Does This Matter?

1. Better Risk Management: Banks and investors can protect their money better if they know exactly how "rough" the market might get.
2. Smarter Trading: If you can predict the direction of the volatility, you can make more profitable trades.
3. The Power of AI: This paper shows that AI isn't just for replacing humans; it can be used as a "turbocharger" to make complex mathematical theories work in the real world.

The Bottom Line

The authors proved that volatility is rough, and ignoring that roughness makes your predictions worse. By combining a sophisticated "rough" math model with a smart AI shortcut, they created a superior tool for forecasting market storms. It's like upgrading from a paper map to a high-definition GPS that sees every pothole on the road.

Technical Summary

1. Problem Statement

The paper addresses the challenge of forecasting Realized Volatility (RV) with higher accuracy than traditional models allow.

• Limitations of Existing Models: The standard benchmark, the Heterogeneous Autoregressive (HAR) model, relies solely on backward-looking historical return data. While robust, it ignores forward-looking information embedded in derivative prices.
• The Gap in Option-Based Forecasting: Previous studies have attempted to incorporate option-implied information (e.g., VIX or estimators from traditional Stochastic Volatility (SV) models like Heston or Bates). However, these traditional SV models often fail to capture the "roughness" of volatility paths (high-frequency irregularity) observed empirically.
• Computational Barrier: The Rough Heston model, which captures these refined dynamics, is non-Markovian and computationally expensive to calibrate on large-scale option panels, limiting its practical application in forecasting frameworks.

2. Methodology

The authors propose an augmented forecasting framework that integrates model-based spot volatility estimators derived from a Rough Heston model into the HAR structure.

A. Data

• Underlying: S&P 500 (SPX) high-frequency intraday returns (5-minute intervals) from Jan 2011 to June 2021.
• Options: Traded option data from OptionMetrics used to infer latent spot volatility.
• Volatility Measure: Realized Variance (RV) calculated as the sum of squared 5-minute log-returns, transformed logarithmically to handle skewness.

B. Spot Volatility Inference (The Core Innovation)

To extract spot volatility ($V_t$) from option data, the authors employ a parametric inference framework (Andersen et al., 2015a) involving an iterative two-step procedure:

1. Step 1 (Latent State Recovery): Given structural parameters, solve a penalized Non-Linear Least Squares (NLS) problem to estimate the daily spot variance series ($V_t$) that minimizes the discrepancy between model-implied and market option prices, while penalizing deviations from non-parametric realized volatility.
2. Step 2 (Parameter Estimation): Optimize the structural parameters of the volatility model using the estimated spot variance series from Step 1.
3. Iteration: Repeat until convergence.

C. Deep Learning Surrogate for Rough Heston

A critical technical contribution is the solution to the computational complexity of the Rough Heston model:

• Challenge: The Rough Heston model involves fractional Riccati equations, making standard numerical pricing (Fourier inversion or Monte Carlo) too slow for the iterative inference loop required by large option panels.
• Solution: The authors train a Deep Learning surrogate (a Multi-Layer Perceptron) to approximate the option pricing function.
  • Training: The network learns the mapping from model parameters and contract variables (strike, maturity) to option prices using a synthetic dataset of millions of samples generated via the "Lifted Heston" (Markovian approximation) framework.
  • Application: During the inference loop, the lightweight neural network replaces the expensive numerical pricing engine, accelerating the estimation process by orders of magnitude while maintaining high accuracy (average percentage error < 0.5%).

D. Forecasting Framework

The authors construct HARX models where the baseline HAR model is augmented with exogenous predictors:

• Baseline: HAR (using daily, weekly, and monthly RV components).
• Benchmarks:
  • HAR-RV-VIX (adds non-parametric VIX).
  • HAR-RV-SV (adds spot volatility estimators from Heston, Bates, and SVCJ models).
• Proposed Model: HAR-RV-RHeston (adds the spot volatility estimator inferred from the Rough Heston model).

3. Key Contributions

1. Theoretical Advancement: Demonstrates that rough volatility dynamics (captured by the Rough Heston model) contain unique, incremental informational value for forecasting realized volatility that traditional SV models (Heston, Bates, SVCJ) and non-parametric indices (VIX) miss.
2. Methodological Innovation: Successfully bridges the gap between complex rough volatility theory and practical large-scale estimation by introducing a deep learning surrogate for the pricing function. This enables the application of rigorous parametric inference (Andersen et al., 2015a) to rough models on massive datasets.
3. Empirical Validation: Provides a comprehensive comparison showing that the Rough Heston-based estimator outperforms all traditional SV estimators and the VIX index in both in-sample fit and out-of-sample forecasting.

4. Key Results

The study evaluates performance using Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Quasi-Likelihood (QLIKE), and Mean Directional Accuracy (MDA).

• In-Sample Performance:

  • The HAR-RV-RHeston model achieved the highest adjusted $R^2$ (0.644), surpassing the baseline HAR (0.605), VIX-augmented (0.627), and traditional SV models (0.624–0.628).
  • The coefficient for the Rough Heston spot volatility was highly significant ($t$-stat = 13.767), indicating strong explanatory power.

• Out-of-Sample Forecasting (1-Day Ahead):

  • Accuracy: HAR-RV-RHeston yielded the lowest errors: MAE (0.2137), RMSE (0.2762), and QLIKE (0.0403). This represents a ~9% reduction in MAE compared to the baseline HAR.
  • Directional Accuracy: The model correctly predicted the direction of volatility changes (rise/fall) 73.60% of the time, a 5.6 percentage point improvement over the baseline HAR (68.00%).
  • Statistical Significance: Diebold-Mariano (DM) tests confirmed that the Rough Heston model significantly outperforms not only the baseline but also the VIX and all traditional SV models (Heston, Bates, SVCJ).

• Multi-Horizon Robustness:

  • The superiority of the HAR-RV-RHeston model persisted across forecasting horizons from 1 to 22 days. It consistently maintained lower error metrics and higher directional accuracy compared to benchmarks, proving that rough volatility information is valuable for both short-term trading and medium-term risk management.

5. Significance

• For Practitioners: The paper provides a robust, computationally feasible method to integrate "rough" volatility signals into trading strategies. The improved directional accuracy (MDA) is particularly valuable for algorithmic trading and risk management where predicting the direction of volatility is as critical as the level.
• For Academia: It validates the empirical relevance of rough volatility models beyond asset pricing (where they are already established) into the realm of volatility forecasting. It also sets a precedent for using deep learning not just as a direct predictor, but as a computational accelerator (surrogate) to enable complex parametric inference in finance.
• Future Implications: The authors suggest that extending this framework to even more complex models (e.g., Quadratic Rough Heston) could yield further gains, provided computational efficiency is maintained via similar deep learning techniques.