💰 quantitative finance

Modeling dependency between operational risk losses and macroeconomic variables using Hidden Markov Models

This paper proposes an extension of Hidden Markov Models that incorporates macroeconomic covariates via an auxiliary variable to effectively model the time-dependent heterogeneity and stress-test relationships in operational risk losses, utilizing the Expectation-Maximization algorithm for calibration and validation across various risk-event types.

Original authors: Nikeethan Selvaratnam, Dorinel Bastide, Clément Fernandes, Wojciech Pieczynski

Published 2026-04-24

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Nikeethan Selvaratnam, Dorinel Bastide, Clément Fernandes, Wojciech Pieczynski

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive ship (a bank) sailing through the ocean of the global economy. Your job is to keep the ship safe from "Operational Risk"—these are the unexpected storms, leaks, or crew mutinies that happen not because of the market crashing, but because of internal mistakes, hackers, fraud, or broken systems.

For a long time, captains tried to predict these disasters using a simple map: "If we sail this route, there's a 1% chance of a leak." They assumed the ocean was calm and predictable, like a flat swimming pool. But in reality, the ocean has currents, tides, and sudden storms that change the rules of the game.

This paper is about a new, smarter navigation system called a Hidden Markov Model (HMM) that helps the captain understand not just where the leaks are, but why they are happening right now, by looking at the weather outside.

Here is the breakdown of their idea, using simple analogies:

1. The Problem: The "Swimming Pool" vs. The "Ocean"

Traditional risk models treat the bank like a swimming pool. They assume that if a mistake happens today, the chance of one happening tomorrow is exactly the same. They ignore the fact that sometimes the whole ocean is stormy (a financial crisis), and sometimes it's calm.

The Flaw: When the economy is in a panic (like 2008), banks make more mistakes. When the economy is booming, they make fewer. Old models miss this connection. They are like a weatherman who only looks at the inside of the house and ignores the storm outside.

2. The Solution: The "Invisible Weatherman" (Hidden Markov Models)

The authors propose using a Hidden Markov Model. Think of this as an invisible weatherman inside the ship's engine room.

The Hidden States: The weatherman knows the ship is in one of two "modes": Calm Mode (low risk) or Storm Mode (high risk). You can't see the mode directly, but you can guess it by looking at the waves (the data).
The Magic: This weatherman doesn't just guess based on the ship's history; he also looks out the window at the macroeconomic variables (like the VSTOXX, which is a "fear gauge" for the stock market).

3. The New Trick: Connecting the Dots

The paper's big innovation is linking the ship's leaks (operational losses) with the outside weather (economic volatility).

The Analogy: Imagine you are trying to predict if your car will break down.
- Old Way: "Cars break down randomly."
- New Way: "Cars break down more often when it's freezing cold and the roads are icy."
The authors found that for Fraud (like employees stealing money or lying), the "ice" matters. When the stock market is panicking (high volatility), employees might get desperate or greedy, leading to more fraud. The model learns to say, "Oh, the market is scary today, so I should expect more fraud tomorrow."

4. The Experiment: Testing the Navigation System

The researchers tested this on a bank's data from 2000 to 2018. They looked at different types of risks:

Fraud (Internal & External): This is like the "ice on the road." The model worked great! When they added the "weather forecast" (market volatility), the predictions became much more accurate. The model could see the storm coming and warn the captain.
Physical Damage or System Failures: This is like a "broken lightbulb." Does a stock market crash make a lightbulb burn out? No. The model found that adding the "weather forecast" didn't help here. These risks are caused by things like earthquakes or bad wiring, not by the stock market.

5. The Results: Why It Matters

For Fraud: The new model is a superhero. It knows that when the economy is stressed, the risk of fraud spikes. It stops the bank from being too relaxed during a crisis or too paranoid during a calm period.
For Other Risks: It's a "one size does not fit all" situation. You don't need a weather forecast to predict a broken pipe, but you definitely need it to predict a mutiny.

The Bottom Line

This paper teaches us that risk isn't random; it's reactive.

By using this "Hidden Markov" navigation system, banks can stop guessing and start anticipating. They can look at the stock market's "fear gauge," realize the ship is entering "Storm Mode," and tighten their security against fraud before the losses happen. It's the difference between sailing blindfolded and having a smart assistant who whispers, "Captain, the storm is coming, brace the crew!"

1. Problem Statement

Operational Risk (OR) modeling faces significant challenges due to the heterogeneous, time-dependent nature of real-world loss data and the increasing regulatory demand for stress testing that links losses to macroeconomic conditions.

Limitations of Traditional Models: The standard Loss Distribution Approach (LDA), typically using a Compound Poisson Process (CPP), assumes that events occur independently with constant intensity (memoryless property). This fails to capture:
- Regime Switching: Operational losses often cluster during specific periods (e.g., financial crises) and exhibit "memory" (e.g., a fixed IT issue reduces future risk).
- Macroeconomic Dependency: Traditional models do not naturally incorporate external economic covariates (like volatility indices) to explain shifts in loss distributions.
- Stress Testing Needs: Regulators (Basel III/IV, CCAR, EBA) require models that can simulate how macroeconomic shocks propagate to operational losses, a capability static LDA models lack.

2. Methodology

The authors propose an extension of the Hidden Markov Model (HMM) to a multivariate observation framework to address these limitations.

A. Model Architecture

The model defines a bivariate Gaussian HMM $(X_n, Y_n)$ where:

Latent State ( $X_n$ ): A discrete-time homogeneous Markov chain with $K$ states (e.g., "Normal" vs. "Stress" regimes).
Observations ( $Y_n$ ): A 2-dimensional vector $(Y^1_n, Y^2_n)$ $(Y_{n}^{1}, Y_{n}^{2})$ :
- $Y^1_n$ : A macroeconomic covariate (specifically the VSTOXX, the Euro Stoxx 50 Volatility Index).
- $Y^2_n$ : The operational loss variable.
Dependency Mechanism: The latent state $X_n$ drives the joint distribution of both the loss and the macroeconomic variable. This allows the model to capture regime switches where both market volatility and operational losses change simultaneously.

B. Estimation and Calibration

Algorithm: The model parameters $\Xi = (\pi, A, \theta_1, \dots, \theta_K)$ $Ξ = (π, A, θ_{1}, \dots, θ_{K})$ are estimated using the Expectation-Maximization (EM) algorithm, specifically the Baum-Welch algorithm.
- E-Step: Computes forward ( $\alpha$ ) and backward ( $\beta$ ) probabilities to estimate the posterior probability of hidden states.
- M-Step: Updates transition matrices ( $A$ ), initial distributions ( $\pi$ ), and Gaussian emission parameters ( $\mu, \Sigma$ ) to maximize the log-likelihood.
Data Preprocessing: To handle the heavy-tailed nature of operational losses and ensure Gaussian stability, the authors apply a filtering procedure based on Inter-Quartile Range (IQR) to remove extreme outliers before calibration.
Prediction: The model predicts the 90th percentile (VaR) of future operational losses ( $Y^2_{N+1}$ ) by simulating paths based on the filtered posterior probabilities of the current state and the transition probabilities.

C. Experimental Design

Data: Anonymized operational loss data (Jan 2000 – June 2018) categorized by Basel II event types.
Variables: Loss data aggregated at Weekly, Monthly, and Quarterly frequencies. The macroeconomic variable is the VSTOXX.
Configuration: Models are tested with $K \in \{2, 3, 4\}$ hidden states, both with and without the VSTOXX covariate.
Evaluation: Performance is measured using Mean Squared Error (MSE) of the exceedances (difference between predicted 90th quantile and actual losses) and back-testing exception rates.

3. Key Contributions

Multivariate HMM Extension: The paper introduces a specific formulation where a macroeconomic covariate is treated as a direct component of the observation vector, rather than just an external regressor. This allows the latent state to capture the joint dynamics of market stress and operational losses.
Regime Detection in OR: It demonstrates that operational losses are not stationary but exhibit regime-switching behavior (e.g., high-loss clusters during crises) that standard Poisson models miss.
Covariate Utility Analysis: The study provides a granular analysis of when macroeconomic variables improve risk estimation. It proves that the inclusion of covariates is not universally beneficial but depends heavily on the specific event type and data frequency.
Frequency Sensitivity: The research highlights that the effectiveness of HMMs is highly sensitive to time aggregation. High-frequency (weekly) data often requires covariates to stabilize regime detection, whereas low-frequency (quarterly) data allows standard HMMs to capture regimes effectively.

4. Results

The empirical study focused primarily on Internal Fraud (ET1) and compared results across other event types (External Fraud, Physical Damage, System Failures, etc.).

Internal Fraud (ET1):
- Significant Improvement: Including VSTOXX significantly reduced the MSE of 90th quantile predictions, particularly at weekly and monthly frequencies.
- Mechanism: The covariate helped the model distinguish between short-term noise and genuine regime shifts, reducing the "chatter" (frequent state switching) seen in models without covariates at high frequencies.
- Visual Evidence: The filtered probabilities of the "high-loss" state aligned closely with periods of high market volatility (e.g., the 2007–2009 crisis) when VSTOXX was included.
Other Event Types:
- External Fraud (ET2): Showed similar improvements to Internal Fraud, suggesting a dependency on market sentiment.
- Physical Damage (ET5) & System Failures (ET6): The inclusion of VSTOXX did not improve (and sometimes worsened) prediction accuracy. These risks are driven by operational/technological factors rather than financial market volatility.
- Employment Practices (ET3) & Client Issues (ET4): No visible regime changes were detected; HMMs offered no advantage over simpler models.
State Space ( $K$ ):
- $K=2$ and $K=3$ provided the best balance between flexibility and interpretability.
- $K=4$ often led to overfitting, especially at lower aggregation levels (quarterly/monthly) where data points were scarce.

5. Significance and Implications

Regulatory Stress Testing: The framework offers a robust tool for regulators and banks to perform "what-if" scenarios. By linking operational losses to macroeconomic shocks (via the VSTOXX), institutions can better estimate capital requirements under stressed economic conditions.
Model Selection Guidance: The paper provides a data-driven rule for modelers: Do not blindly include macroeconomic covariates. They are essential for fraud-related risks (which correlate with market sentiment) but irrelevant for physical or IT failures.
Future Directions: The authors suggest extending the Gaussian emission assumption to heavy-tailed distributions (e.g., Student-t) to better capture extreme tail risks without the need for aggressive data filtering. They also propose exploring other covariates like interest rates, climate indicators, or geopolitical risk scores.

In conclusion, the paper successfully demonstrates that a Multivariate Hidden Markov Model is a superior framework for modeling operational risk in a stress-testing context, provided the model is tailored to the specific dependency structure of the risk event type and the temporal resolution of the data.