💰 quantitative finance

Regime-Gated Residual Mixture-of-Experts for Cross-Sectional Volatility Forecasting

This paper proposes RG-ResMoE, a regime-gated residual mixture-of-experts architecture that improves cross-sectional volatility forecasting accuracy and training stability by using regime information exclusively for expert routing rather than direct input, demonstrating that controlling how nonstationary regime data influences predictions is more critical than simply increasing model capacity.

Original authors: Junyi Ye, Gargi Vijay Borde

Published 2026-08-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junyi Ye, Gargi Vijay Borde

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather. You have a smart assistant who looks at the clouds, the wind, and the temperature to tell you if it will rain tomorrow. Usually, this works great. But sometimes, the world changes its mind. A sudden storm rolls in, or a heatwave hits, and the rules the assistant learned yesterday don't work today. In the world of finance, this is called "regime dependence." Markets have different "moods"—calm days, chaotic crashes, and slow recoveries—and the relationship between what happened yesterday and what happens tomorrow shifts depending on that mood.

For a long time, scientists have tried to build computer brains (neural networks) that can predict how much stock prices will bounce around, a concept called "volatility." The big question has been: how do you teach these computers to understand these shifting moods? The old way was to just hand the computer a list of facts, including a note about the current mood, and hope it figures it out. But this paper suggests that simply dumping extra information onto the computer isn't the right move. Instead, it asks a smarter question: should the computer use the mood to decide which part of its brain to use, or should it just use the mood as another piece of data to crunch?

This paper, titled "Regime-Gated Residual Mixture-of-Experts for Cross-Sectional Volatility Forecasting," dives into that exact puzzle. The researchers built a new type of computer model called RG-ResMoE. Think of it as a team of four specialized mechanics (the "experts") working on a car. In a standard setup, the team leader (the "gate") looks at the weather report (the "regime") and the car's engine noise (the "stock features") to decide how much each mechanic contributes to the final prediction. But the authors found that this often confuses the team, leading to mistakes and unstable training.

Instead, their new design works like this: One "base" mechanic is frozen in place and handles the standard, boring parts of the job using only the engine noise. The other four mechanics are there only to fix the mistakes the base mechanic makes. The team leader looks at the weather report and the engine noise, but instead of telling a single mechanic to take over, it acts like a traffic cop. It uses the weather report to decide how much each of the four "fix-it" mechanics should tweak the base mechanic's work. If it's a stormy day, the leader might tell the "storm expert" to add a little extra caution to the prediction. If it's a calm day, the "calm expert" might suggest a smaller adjustment.

The results were clear and consistent. When the researchers tested this on over 1,000 U.S. stocks and even replicated it on 1,500 Japanese stocks, their new method was the winner. It predicted future volatility more accurately than the old ways and, crucially, it was much more stable. The old method of just adding the "mood" data directly to the input often caused the computer to crash or learn the wrong things, especially during big market crashes like the one in 2020. The new method, however, stayed calm and accurate, even when the market was panicking.

The paper suggests that the secret sauce isn't just having more data or more experts; it's about how you use the information about the market's mood. By using the mood to guide the tiny corrections (the "residuals") rather than the main prediction, the model becomes both smarter and more reliable. This approach didn't just work on average; it shined brightest when it was needed most—during the most volatile and stressful times in the market. The authors conclude that for compact, efficient models, the best way to handle a changing world is to let the "mood" control the adjustments, not the main forecast itself.

Technical Summary: Regime-Gated Residual Mixture-of-Experts for Cross-Sectional Volatility Forecasting

Problem Statement

Financial volatility is inherently regime-dependent, with relationships between historical observations and future volatility shifting across market states (e.g., calm, crisis, recovery). While deep neural networks have improved volatility prediction by learning non-linear relationships from large cross-sectional equity panels, incorporating regime information remains challenging. Existing approaches typically introduce regime variables by appending them directly to model inputs or by switching between different predictors based on market states. However, these methods often conflate the presence of regime information with the integration pathway, making it unclear whether performance gains stem from the information itself or from how it enters the model. Furthermore, adding regime variables directly to inputs can destabilize training and degrade predictive performance in compact neural architectures.

Methodology

The authors propose RG-ResMoE (Regime-Gated Residual Mixture-of-Experts), a novel architecture designed to isolate the effect of the integration pathway. The study employs a rigorous, information-matched comparison framework where regime variables, model capacity, hyperparameters, and evaluation protocols are held constant, varying only the point of integration.

Data and Task

Datasets: A primary panel of 1,027 U.S. equities (2018–2025) and a replication panel of 1,552 Japanese equities.
Task: Predicting five-day forward realized volatility (annualized) for cross-sectional equity panels.
Inputs:
- Stock Features ( $x$ ): 16 features including realized volatility, returns, RSI, and log returns.
- Regime Variables ( $z$ ): Two indicators—market volatility (20-day rolling volatility of equal-weighted market returns) and idiosyncratic volatility (20-day rolling volatility of residual returns).

Architecture Design

The core innovation lies in separating the base prediction from regime-dependent corrections:

Frozen Base Network: A standard two-hidden-layer MLP predicts the base volatility level ( $\hat{y}_{base}$ ) using only stock features ( $x$ ). This network is trained first and then frozen.
Residual Experts: A set of $K=4$ expert networks (also MLPs) predict residual corrections ( $r_k(x)$ ) rather than full forecasts. These experts are initialized with zero-output layers to ensure the model starts as the base forecast.
Regime-Gated Routing: A gating network ( $g$ ) takes both stock features ( $x$ ) and regime variables ( $z$ ) as input. It outputs a soft probability distribution ( $\pi$ ) over the experts.
Final Prediction: The final forecast is the sum of the base prediction and the weighted sum of expert residuals:
$\hat{y} = \hat{y}_{base} + \sum_{k=1}^{K} \pi_k r_k(x)$
Crucially, the regime variables $z$ influence the prediction only through the routing weights $\pi$ , not by directly altering the input to the forecasting networks.

Training and Regularization

Training proceeds in two stages:

Train the base network.
Freeze the base and jointly train the experts and the gate.
The loss function includes:

Mean Squared Error (MSE) on the final prediction.
A shrinkage penalty on the magnitude of the aggregate residual correction to keep predictions close to the base.
A load-balancing penalty to prevent routing collapse (concentration on a single expert).

Key Contributions

Integration Pathway Discovery: The paper demonstrates that the location of regime information is decisive. Appending regime variables directly to inputs degrades accuracy and stability, whereas using them exclusively for gating residual corrections improves both.
Residual MoE Architecture: The authors show that in compact financial forecasting models, the value of Mixture-of-Experts (MoE) lies less in increasing total model capacity and more in controlling how non-stationary information modulates a stable base forecast via residual learning.
Soft vs. Hard Routing: Continuous soft routing (learned probability distributions) consistently outperforms hard routing (discrete assignments based on learned top-1 gates or hand-crafted market partitions).
Training Stability: The RG-ResMoE architecture significantly improves training stability compared to standard MoE and direct-input MLPs, eliminating "collapsed" training runs where models fail to converge.

Results

Evaluated on a rolling walk-forward framework (30 non-overlapping test windows) across 1,027 U.S. stocks:

Forecasting Accuracy: RG-ResMoE achieved the highest Information Coefficient (IC: 0.5469), lowest RMSE (0.2304), and highest out-of-sample $R^2$ (0.292) among all neural models. It significantly outperformed a capacity-matched large MLP (MLP-L) and standard MoE.
Stability: RG-ResMoE had 0 collapsed seeds out of 30, whereas Standard MoE collapsed in 24/30 runs, and MLP-L(+z) collapsed in 30/30 runs.
Risk Calibration: RG-ResMoE provided better Value-at-Risk (VaR) calibration, particularly during high-volatility periods, showing lower rejection rates in Kupiec's coverage tests compared to baselines.
Market Stress Performance: The model's advantage was most pronounced during periods of elevated volatility (e.g., the 2020 COVID crisis, where the IC gain was 6.7× the full-sample average).
Cross-Market Replication: The results were successfully replicated on the Japanese TSE Prime panel, confirming that the benefits of residual routing are not market-specific.

Significance and Claims

The paper argues that for compact neural volatility forecasters, the primary value of regime-aware MoE models is not in expanding model capacity, but in controlling how non-stationary regime information influences the prediction.

The authors claim a simple design principle: regime information is most effective when it guides residual routing rather than being treated as an ordinary predictive input. This approach allows the model to adapt to market states by modulating corrections to a stable base forecast, thereby improving both predictive accuracy and training robustness. The study suggests this principle extends beyond the specific RG-ResMoE architecture to guide the incorporation of market state information in future neural forecasting models.

Limitations noted by the authors include the focus on equity markets and a compact MLP backbone; the generalizability to other financial tasks, larger sequence models, or alternative state definitions remains an open area for future work.