Regime-aware financial volatility forecasting via in-context learning

This paper introduces a regime-aware in-context learning framework that leverages pretrained large language models to forecast financial volatility by dynamically adapting to nonstationary market conditions through oracle-guided, regime-specific demonstrations without requiring parameter fine-tuning.

The Gist

Imagine you are trying to predict the weather for tomorrow.

If you live in a place where the weather is always mild and predictable, you can just look at the last few days and guess, "It'll probably be sunny." This is how traditional financial models work. They look at past stock prices and assume the future will look a lot like the past.

But financial markets are more like the weather in a place with sudden, violent storms. One day it's calm, and the next, a hurricane hits. Traditional models get confused during these storms because they are too rigid. They keep predicting "sunny" even when the sky is turning black.

This paper introduces a new way to predict market chaos using Large Language Models (LLMs)—the same kind of AI that writes essays and chats with you. But instead of teaching the AI new math (which is slow and expensive), the authors teach it how to learn on the fly by looking at examples, just like a human student.

Here is the breakdown of their "Regime-Aware" system using simple analogies:

1. The Problem: The "One-Size-Fits-All" Mistake

Imagine a chef who cooks the same recipe for everyone. If you ask for a spicy dish, they give you mild soup. If you ask for mild soup, they give you a fire-breathing chili.

• Traditional Models: They are like this chef. They try to find one "average" pattern for the market. When the market is calm, they do okay. When the market goes crazy (high volatility), they fail miserably because they don't know how to switch recipes.

2. The Solution: The "Smart Librarian" Approach

The authors propose using an LLM as a Smart Librarian. Instead of memorizing a single recipe, the librarian has a massive library of past market days.

• In-Context Learning: When you ask the librarian, "What will the market do tomorrow?", the librarian doesn't just guess. They look at their library, find similar days from the past, and say, "Well, on days like this (calm days), the market did X. But on days like that (stormy days), the market did Y."
• The Catch: If you just ask the librarian to pick any random past day, they might pick a calm day to explain a stormy situation. That's bad.

3. The Secret Sauce: "Regime-Aware" Sorting

The paper's big innovation is Regime-Awareness.

Think of the market as having two distinct "moods" or Regimes:

1. The "Calm" Regime: Like a sunny Tuesday. Prices move slowly.
2. The "Storm" Regime: Like a hurricane. Prices swing wildly.

The authors built a system that acts like a Sorter:

1. The Oracle (The Teacher): First, they use a "teacher" (who knows the future answers) to look at past data and label every single day as either "Calm" or "Storm."
2. The Two Shelves: They organize the librarian's library into two separate shelves:
   • Shelf A: Only contains examples of "Stormy" days.
   • Shelf B: Only contains examples of "Calm" days.
3. The Smart Guess: When you ask for a prediction for tomorrow:
   • The system first checks the current weather (the last few days of data). "Hey, it looks like a storm is brewing!"
   • It then tells the librarian: "Go to Shelf A (Stormy days) and find me 5 examples of what happened on days that looked like this."
   • The librarian reads those 5 examples and says, "Ah, I see the pattern. When the market gets this crazy, it usually swings this way."

4. Why This is Better

• No Retraining: Traditional models need to be "re-taught" every time the market changes. This AI just changes its reading list. It's like a student who doesn't need to go back to school; they just need to open the right chapter of the textbook.
• Adaptability: When the market is calm, the AI looks at calm examples. When the market panics, it instantly switches to looking at panic examples.
• The Results: In their tests (using data from the S&P 500, NASDAQ, and currency markets), this "Smart Librarian" made significantly fewer mistakes than the old "One-Size-Fits-All" chefs, especially during the scary, high-volatility times.

The Bottom Line

This paper shows that we don't need to build a new, super-smart AI from scratch to predict the stock market. We can take an existing AI, give it a sorted library of past examples, and teach it to pick the right examples based on whether the market is currently calm or crazy.

It's the difference between guessing the weather by looking at an average of the whole year, versus checking the forecast for this specific season and this specific weather pattern.

Technical Summary

Here is a detailed technical summary of the paper "Regime-Aware Financial Volatility Forecasting via In-Context Learning".

1. Problem Statement

The paper addresses the challenge of financial volatility forecasting in non-stationary market conditions.

• Core Difficulty: Financial markets exhibit complex dynamics, including heavy-tailed return distributions and abrupt regime shifts (sudden transitions between low and high volatility). Traditional econometric models (e.g., GARCH) and standard deep learning sequence models often struggle to generalize during these turbulent periods because they rely on historical patterns that may become irrelevant after structural changes.
• Goal: To leverage Large Language Models (LLMs) as in-context learners to predict future realized variance (volatility) without parameter fine-tuning. The specific objective is to overcome the limitation where naive LLM prompting fails to adapt to high-volatility regimes, leading to significant prediction errors during market stress.

2. Methodology

The proposed framework, Regime-Aware In-Context Learning, operates entirely through prompt engineering and inference, avoiding any gradient-based updates to the LLM weights. The process consists of four main stages:

A. Initial Prompt-Based Prediction

The system converts historical financial time series (log returns and realized variance) into structured natural language prompts. A fixed pre-trained LLM ($M$) is prompted to generate an initial volatility estimate ($\hat{\nu}^{(0)}_{t+1}$) based on the input sequence.

B. Oracle-Guided Refinement Loop (Training Phase Only)

To construct high-quality training examples, the authors employ an iterative refinement process using "oracle" feedback (ground truth data available only during training):

1. The LLM generates an initial prediction.
2. An oracle provides feedback consisting of:
   • The ground-truth realized variance ($\nu_{t+1}$).
   • The initial prediction error ($e_t$).
   • A heuristic correction signal (e.g., suggesting adjustment toward a recent average).
3. The LLM refines its prediction based on this feedback.
4. This loop runs for a fixed number of iterations (e.g., 3) to produce a refined prediction $\hat{\nu}^{(*)}_{t+1}$.

C. Construction of Regime-Aware Demonstration Pool

The refined predictions are stored as demonstration tuples $T_t = (p_{conc}(x_t), \hat{\nu}^{(*)}_{t+1}, R_t)$, where $R_t$ is a volatility regime label (High or Low).

• Labeling: The regime is determined by comparing the next-day realized variance against a threshold $\tau$ (derived from the training set quantiles).
• Pooling: The dataset is split into two conditional pools: $D_{high}$ (high-volatility examples) and $D_{low}$ (low-volatility examples).

D. In-Context Forecasting (Inference Phase)

At test time, the LLM predicts volatility using conditional sampling from the demonstration pool:

1. Regime Estimation: The system estimates the current market regime ($\hat{R}_t$) using a simple heuristic (e.g., the mean of the last $m$ realized variances) based only on available historical data.
2. Conditional Sampling: Instead of random sampling, the system selects $K$ demonstration examples from the pool ($D_{high}$ or $D_{low}$) that match the estimated current regime.
3. Final Prediction: The LLM is prompted with the input sequence concatenated with the regime-matched demonstrations to generate the final forecast.

3. Key Contributions

1. Oracle-Guided Refinement Framework: A novel method to construct a high-quality demonstration pool by iteratively refining LLM predictions using ground-truth feedback during the training phase, without fine-tuning the model parameters.
2. Regime-Aware Conditional Sampling: The introduction of two strategies for selecting in-context examples:
   • Fixed Prior: Mixing examples based on a static label distribution.
   • Estimated Label: Dynamically selecting examples based on the estimated current market regime. This allows the LLM to adapt its reasoning to the specific volatility dynamics of the current moment.
3. Zero-Shot Adaptation: Demonstrating that a frozen LLM can effectively adapt to non-stationary financial environments purely through context selection, bridging the gap between the stability of long-memory models and the adaptability required for regime shifts.

4. Experimental Results

The framework was evaluated on three datasets: S&P500, NASDAQ, and EUR/USD, comparing against classical baselines (Rolling Mean, HAR, GARCH, GJR-GARCH) and a naive LLM "one-shot" approach.

• Performance Metrics: Evaluated using RMSE and MAE, with specific breakdowns for low-volatility ($MAE_{low}$) and high-volatility ($MAE_{high}$) regimes.
• Key Findings:
  • Naive LLMs: The "one-shot" LLM baseline performed poorly, especially during high-volatility periods, often under-reacting to sudden spikes.
  • Classical Models: GJR-GARCH performed best among classical models in high-volatility regimes but struggled with average error trade-offs. HAR performed well in low-volatility regimes.
  • Proposed Method: The Regime-Aware (Label Estimate) approach achieved the best overall performance.
    • On the S&P500, it reduced the overall forecasting error by approximately 27% compared to the best classical baseline (GJR-GARCH).
    • It significantly outperformed all baselines in high-volatility regimes (reducing $MAE_{high}$ by ~27% compared to GJR-GARCH), proving its ability to adapt to market stress.
    • It maintained competitive performance in low-volatility regimes, effectively balancing the trade-off between regime adaptation and average error.

5. Significance and Conclusion

This work establishes a new paradigm for financial time series forecasting using foundation models.

• Theoretical Insight: It demonstrates that the "regime-dependent failure mode" of standard in-context learning can be mitigated by explicitly injecting regime information into the context selection process.
• Practical Impact: The approach offers a robust, parameter-free solution for volatility forecasting that outperforms established econometric models, particularly during critical market periods where risk management is most vital.
• Future Direction: The authors suggest that retrieval-based demonstration selection could further enhance context extraction, moving beyond simple threshold-based regime labeling.

In summary, the paper proves that by combining oracle-guided refinement with regime-aware context selection, LLMs can serve as highly effective, adaptive tools for financial volatility forecasting without the need for expensive retraining.