🤖 machine learning

Latent-Regime Bias Auditing for Volatility Forecasting

This paper introduces a model-agnostic audit framework that evaluates volatility forecasts across latent market regimes to reveal that models with competitive aggregate accuracy can still suffer from significant conditional biases and tail underpredictions, thereby advocating for regime-specific reliability assessments over average error metrics.

Original authors: Arthur Chagas, Pedro Bento, Yan Aquino, Arthur Buzelin, Wagner Meira Jr., Cristiano Arbex Valle

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Arthur Chagas, Pedro Bento, Yan Aquino, Arthur Buzelin, Wagner Meira Jr., Cristiano Arbex Valle

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a weather forecaster. For years, you've been judged on one simple number: your average accuracy. If you get the temperature right 90% of the time over a whole year, you're a hero. But here's the catch: what if your 10% of mistakes always happen when a hurricane is about to hit? You might be "accurate on average," but you are useless exactly when people need you most. This is the core problem in a field called volatility forecasting. In finance, "volatility" is just a fancy word for how much prices jump around. When markets are calm, prices barely move; when they are stressed, they swing wildly. Investors and risk managers need to predict these swings to protect their money. Traditionally, they've only looked at the "average error" of their prediction models. But just like the weather forecaster, a model can look great on paper while failing spectacularly during the most dangerous moments.

This paper, titled "Latent-Regime Bias Auditing for Volatility Forecasting," acts like a detective for these financial prediction models. The authors, a team from the Federal University of Minas Gerais in Brazil, argue that we need to stop just checking the final grade and start looking at when and why a model gets things wrong. They built a new "audit" system to see if a model remains reliable when the market is calm versus when it is in a panic.

The Detective's Toolkit: Finding Hidden Regimes

To understand how this works, imagine you are trying to sort a massive pile of photos of different people. If you just throw them all into one big bucket and look for patterns, you might get confused because some people are tall, some are short, and some are wearing hats. In finance, different assets (like Bitcoin, gold, or stock market funds) have their own unique "personalities." Bitcoin might swing wildly while gold stays relatively steady. If you mix them all together, your analysis gets messy.

The authors' first trick was to teach a computer to ignore who the asset is and focus only on how it is behaving. They used a clever technique called "adversarial learning." Think of this as a game of hide-and-seek between two computer programs. One program tries to guess which asset is in the photo (the "seeker"), while the other tries to scramble the photo so the seeker can't tell (the "hider"). By training the hider to win, the computer learns to strip away the specific identity of the asset and keep only the pure "mood" of the market.

Once they stripped away the identities, they grouped the market days into three "regimes" or moods: Calm, Intermediate, and Stress. These aren't universal rules for the whole world; they are relative to each asset. For Bitcoin, a "Stress" day might be a 10% drop, while for a stable gold fund, a "Stress" day might be a 1% drop. The key is that the computer learned to recognize these moods based on how the market felt at the time, not just by looking at a calendar.

The Audit: Who Fails When?

With these mood groups defined, the authors put dozens of different prediction models to the test. They looked at everything from simple math formulas (like the "HAR" models used by economists for decades) to fancy, complex AI systems like Transformers and Neural Networks.

Here is the big reveal: The models that looked the best on average were often the worst at handling stress.

The paper found that many models, including some of the most popular and complex AI ones, had a hidden flaw. They were great at predicting normal days but systematically underpredicted (guessed too low) the volatility during "Stress" regimes. It's like a weather app that says "sunny" every day, but when a storm actually hits, it still says "sunny" because it's trying to be right on average.

The authors introduced a new way to measure this called $RBA_{stable}^{m}$ . Think of this as a "reliability score." A high score means the model is hiding a big bias. They found that models with excellent average scores (low RMSE) often had terrible reliability scores. For example, a model might have an average error of 0.026, which looks great, but when the market is in a panic, it might be off by a huge margin, leaving investors exposed to massive losses.

The Verdict: Complexity Isn't a Cure-All

One of the most interesting findings is that making the models more complex didn't fix the problem. The authors tested advanced "Transformer" models (the same type of AI behind many modern chatbots) and found they didn't perform any better than simpler, older models when it came to predicting extreme market swings. In fact, some of the fancy AI models were even worse at handling the tail-end risks (the rare, extreme events) than the simple, old-school formulas.

The paper suggests that the financial world has been too focused on "average accuracy." If you are a risk manager, you don't care about the average; you care about the worst-case scenario. The authors show that by using their new audit framework, we can see that a model might be "accurate" overall but "unreliable" exactly when we need it most.

Why This Matters

This isn't just about math; it's about safety. If a bank uses a model to decide how much money to keep in reserve, and that model underestimates the risk during a market crash, the bank could run out of money. The authors aren't saying their new models are perfect or that they have solved the problem of predicting the future. Instead, they are offering a new tool to audit the tools we already have.

They conclude that we need to stop asking, "Which model is the smartest on average?" and start asking, "Which model stays reliable when the market is scared?" Their research suggests that the answer might not be the most complex AI, but rather a model that we understand well enough to know exactly where its blind spots are. By shifting the focus from "average error" to "conditional reliability," we can build a financial system that is safer, not just smarter.

Technical Summary: Latent-Regime Bias Auditing for Volatility Forecasting

1. Problem Statement

Volatility forecasts are critical for risk management, hedging, and portfolio allocation, particularly during periods of market instability. However, standard evaluation practices rely heavily on aggregate accuracy metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The authors argue that these metrics compress the temporal and conditional distribution of errors, potentially masking systematic failures that occur in specific, economically meaningful market states. A model may exhibit low average error while being conditionally unreliable—specifically, by systematically underpredicting volatility during stress regimes, which is the most costly error for risk management. The paper posits that volatility forecasting should be treated as a conditional reliability problem, asking not just if a model is accurate on average, but where and when its errors become biased or unstable.

2. Methodology

The paper proposes a model-agnostic audit framework designed to detect hidden regime-specific biases without altering the forecasting models themselves. The methodology proceeds through the following stages:

A. Asset-Relative Representation Learning

To evaluate forecasts across different assets (cryptocurrencies and ETFs) without the results being dominated by persistent asset-specific scales, the framework learns time-series representations of market-state windows:

Normalization: Features (returns, range-based volatility, implied volatility, volume, drawdown, etc.) are normalized within each asset using robust statistics (median and IQR) derived from the training period.
Embedding: A contrastive encoder (TS2Vec-style) learns window embeddings ( $z_{i,t}$ ) to align temporally overlapping views.
Adversarial Invariance: To prevent the embeddings from encoding static asset identity (which would confound regime detection), an adversarial asset classifier with gradient reversal is employed. The encoder is trained to make asset prediction difficult, thereby preserving dynamic market-state information while suppressing asset-identifying structure.

B. Construction of Aligned Regimes

Instead of treating regimes as universal market states, the framework constructs asset-relative audit partitions:

Clustering: K-means clustering is applied to the training embeddings for each asset individually to identify $K$ latent states.
Alignment: Since cluster labels are arbitrary across assets, they are aligned using a training-period stress score. This score combines realized volatility, drawdown, absolute returns, and stress-proxy variables.
Labeling: Clusters are ordered by their average stress score and labeled as Calm, Intermediate, and Stress. This ensures that "Stress" in one asset corresponds to a comparable level of stress in another, creating a unified audit space.

C. Audit Metrics

The framework evaluates forecasters ( $M$ ) using several metrics beyond standard RMSE/MAE:

Regime-Conditional Bias: The mean signed error within each aligned regime.
Hidden Bias Statistics:
- $RBA^{stable}_m$ : Quantifies how much the aggregate bias understates the largest asset-relative regime bias.
- $RCRG^{norm}_m$ : Normalizes the gap between aggregate and worst-case regime bias by the RMSE.
Tail and Economic Loss:
- Tail Underprediction: The rate of underprediction in the top decile of realized volatility outcomes.
- Asymmetric Loss: An asymmetric squared loss function ( $\ell_\lambda$ ) that penalizes underprediction ( $e < 0$ ) more heavily than overprediction, reflecting the economic cost of risk underestimation.
Statistical Inference: Block-bootstrap confidence intervals are used to determine if regime-specific biases are statistically significant.

3. Key Contributions

The paper outlines five primary contributions:

Problem Framing: Reframing volatility forecasting as an asset-relative, regime-conditional reliability auditing problem rather than a pure accuracy optimization task.
Diagnostic Tools: Introducing model-agnostic diagnostics for hidden regime bias, tail underprediction, and underprediction-sensitive economic loss.
Regime Construction: Developing a method to construct asset-relative regimes using normalized time-series embeddings, adversarial asset-invariance, and economic stress-score alignment.
Comprehensive Audit: Applying the framework to a broad set of models, including historical baselines, HAR/IV specifications, linear models, tree-based models, recurrent networks (LSTM/GRU), and Transformer-style architectures (iTransformer, PatchTST).
Empirical Evidence: Demonstrating that rankings based on aggregate accuracy (RMSE) often diverge from rankings based on conditional reliability and economic loss, indicating that "accurate" models can still be conditionally unreliable.

4. Experimental Results

The framework was applied to daily realized volatility forecasts for BTC, ETH, SPY, QQQ, IWM, GLD, and USO over the period 2021–2026.

Regime Validity (RQ1): The aligned regimes (Calm, Intermediate, Stress) were shown to be economically meaningful. Stress regimes exhibited higher realized/implied volatility, larger drawdowns, and higher concentrations of tail events compared to calm periods.
Hidden Bias (RQ2): Aggregate metrics were found to mask significant regime-specific biases. For instance, several models with competitive aggregate RMSE exhibited statistically significant underprediction or overprediction in specific regimes (e.g., LSTM and GRU models showed significant underprediction in intermediate stress regimes).
Divergent Rankings (RQ3 & RQ4):
- Reliability vs. Accuracy: Models ranking highly by RMSE (e.g., recurrent models) often dropped in rank when evaluated by conditional reliability metrics ( $RBA^{stable}$ ). Conversely, some linear and finance-native models (e.g., HAR-RV) improved in reliability rankings.
- Economic Loss: Rankings shifted further when using tail-weighted asymmetric loss. Transformer-style models (PatchTST, iTransformer), while competitive in some aggregate metrics, performed poorly under economic-loss criteria, showing weaker responsiveness to volatility extremes and higher tail underprediction.
Robustness (RQ5): The findings held across different forecasting horizons and regime construction variants (plain vs. adversarial embeddings), confirming that the observed conditional reliability gaps are not artifacts of a single clustering choice.

5. Significance and Claims

The paper claims that its framework shifts the evaluation paradigm from asking "which model is most accurate on average?" to "identifying the market regimes in which apparently accurate forecasts fail conditionally."

Limitations: The authors are modest about the scope, noting that the regimes are sample-dependent audit partitions rather than universal market states. The analysis is limited to daily realized volatility targets and a specific cross-asset panel.
Core Insight: The primary significance is the demonstration that architectural complexity does not guarantee conditional reliability. A model can be highly accurate on average while being dangerously unreliable in the specific stress states where risk management is most critical.
Implication: The paper suggests that volatility forecasting evaluation must move beyond aggregate error metrics to include regime-conditional diagnostics to ensure forecasts are reliable where they matter most. It does not propose a new forecasting model but provides a diagnostic tool to audit existing ones.