💰 quantitative finance

FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance

The paper introduces FinStressTS, a parametric synthetic benchmark comprising 30 diagnostic environments that links financial time-series forecasting performance to specific structural mechanisms, revealing that classical models often outperform Transformers in volatile or jump-driven settings and highlighting the trade-offs between distributional alignment and data efficiency.

Original authors: Jiaze Sun, Kelvin J. L. Koa, Ruiyang Ni, Yize Liu, Haonan Chen, Ke-Wei Huang

Published 2026-06-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiaze Sun, Kelvin J. L. Koa, Ruiyang Ni, Yize Liu, Haonan Chen, Ke-Wei Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the weather. If you only show it one single, real history of the weather, you have a big problem: you can't tell if the robot failed because it's "stupid," because the weather was just weird that day, or because it didn't understand the difference between a cloud and a storm. You can't rewind time to see what would have happened if the wind had blown slightly differently.

This is exactly the problem researchers face with financial forecasting. Real stock market data is a "black box." When a complex AI model fails to predict the market, nobody knows why. Was it the sudden crash? The hidden volatility? The noise? Because everything happens at once in the real world, you can't isolate the specific cause of failure.

FinStressTS is a new tool created by researchers to fix this. Think of it as a "Flight Simulator for Stock Market Models."

Instead of using messy, real-world data, they built a video game-like environment where they control every single rule of the universe. They created 30 different "levels" or scenarios, each designed to test a specific financial behavior, like:

Volatility Clustering: When a market is calm, it stays calm; when it's crazy, it stays crazy for a while.
Regime Switching: Suddenly, the rules of the game change (like a market crash).
Heavy Tails: Rare, massive events (like a 100-year storm) happen more often than normal math predicts.
Zero-Inflated Jumps: Assets that sit still for a long time and then suddenly burst into activity.

In this simulator, the researchers know the "secret code" (the ground truth). They can see exactly how the market should behave. This allows them to run a model, see it fail, and immediately say, "Ah, this model failed because it couldn't handle the 'Regime Switching' level," rather than guessing.

What Did They Find?

They tested 15 different models, ranging from simple, old-school math formulas to the newest, most complex AI (Transformers). Here is what their "flight simulator" revealed:

1. The "Simple is Better" Rule
In most of these financial scenarios, simple models actually beat the fancy AI.

The Analogy: Imagine trying to navigate a bumpy road. A high-tech self-driving car with a massive computer might get confused by every little bump and overreact. A simple, sturdy bicycle (a basic math model) just follows the path and keeps going.
The Result: Simple models (like AR and HAR) were more accurate at predicting the next price than the complex Transformers. The fancy AI models tended to get "distracted" by the noise and overthink the problem.

2. The "Specialist" Problem
Models that are great at one thing often fail miserably at another.

The Analogy: A chess grandmaster is amazing at chess but might be terrible at playing poker.
The Result: A model that worked perfectly when the market was stable (stationary) completely fell apart when the market suddenly changed its rules (regime switching). There is no "one-size-fits-all" super-model.

3. The "Data Hunger" Issue
The complex AI models need way more data to learn than the simple ones.

The Analogy: A simple recipe might work perfectly with 100 ingredients. A complex molecular gastronomy dish might need 10,000 ingredients just to get the taste right, and even then, it might not be better.
The Result: The neural networks (AI) needed 2 to 3 times more data than the simple models just to catch up. In finance, where data is often scarce or changes quickly, this is a huge disadvantage.

4. The "Gambler's Fallacy" of Probability
Predicting the exact number is hard, but predicting the range of possibilities (probability) is even harder for AI.

The Analogy: A simple model might say, "It will probably rain." A complex AI might try to predict the exact shape of every raindrop but end up getting the overall weather wrong.
The Result: Simple probabilistic models (like DeepAR) were very good at estimating risk when things were stable. But when the market got weird (like sudden jumps or zero activity), the complex AI models got confused and gave bad risk estimates. Only models designed to be very flexible could handle the chaos.

The Big Takeaway

The paper concludes that financial forecasting isn't about building the biggest, most complex brain. It's about building the right tool for the specific job.

If you are trying to predict the stock market, you don't necessarily need a massive, data-hungry AI. Sometimes, a simple, robust model that understands the specific "stress test" (like volatility or sudden jumps) is actually the safer, more accurate bet. FinStressTS gives researchers a way to test these tools in a controlled lab before they ever touch real money, ensuring they know exactly why a model works or fails.

Technical Summary: FinStressTS

Problem Statement

Financial time-series forecasting is hindered by low signal-to-noise ratios, latent factors, and discontinuous jumps. A critical limitation of existing real-world benchmarks is the inability to attribute model failure. In historical data, generative mechanisms are unobservable entanglements of noise and structure; when a model underperforms, researchers cannot isolate whether the error stems from distributional miscalibration, an inability to adapt to regime changes, or insufficient data efficiency. Furthermore, real financial data offers only a single realized path, precluding the systematic assessment of tail-risk calibration or data efficiency under controlled conditions. Current evaluation infrastructure forces a choice between authenticity (real data) and diagnostic control (synthetic data), with existing synthetic baselines often relying on oversimplified Gaussian assumptions that fail to challenge modern architectures.

Methodology

The authors introduce FinStressTS, a mechanism-aware synthetic benchmark designed to bridge this gap. Rather than attempting to reproduce the full complexity of markets, FinStressTS adopts a controlled stress-testing philosophy.

Design Principles

Econometric Fidelity: All mechanisms are grounded in canonical stochastic processes established in econometric literature.
Diagnostic Control: Structural parameters (e.g., jump magnitude, regime frequency) are explicitly controlled to enable targeted stress testing.
Verifiable Ground Truth: The data-generating process (DGP) is fully specified, providing access to the true conditional distribution $P(y_{t+1}|H_t)$ , which is impossible with real data.
Multivariate Panel Structure: Environments generate panels of return series driven by shared latent drivers (factors, regimes) rather than arbitrary covariance matrices.

Mechanism Families

FinStressTS comprises 30 diagnostic environments organized around six canonical mechanism families, each targeting specific stylized facts of asset returns:

Volatility Clustering (GARCH-type): Models conditional heteroskedasticity and persistence.
Multi-scale Volatility Persistence (HAR-type): Captures long-memory effects across daily, weekly, and monthly horizons.
Heavy Tails and Outliers: Reproduces fat-tailed return behavior using Student- $t$ innovations.
Regime Switching: Represents abrupt, persistent shifts in market conditions via Markov-switching processes.
Self-Exciting Jumps (Hawkes-type): Captures event clustering where extreme events raise the likelihood of subsequent jumps.
Zero-Inflated Processes: Models intermittent activity with extended zero/near-zero periods, common in illiquid assets.

For each family, five diagnostic levels are defined by varying specific structural parameters (e.g., persistence strength, tail heaviness, regime duration) to isolate failure modes.

Evaluation Protocol

The benchmark evaluates 15 diverse models across two tasks:

Point Forecasting: Assessed using Normalized Mean Absolute Error (NMAE) across five diagnostic settings. Models range from classical econometrics (AR, HAR, VAR) to modern Transformers (PatchTST, iTransformer, Nonstationary Transformer) and decomposition models (DLinear, Autoformer).
Probabilistic Forecasting: Assessed using Continuous Ranked Probability Score (CRPS) in settings with known DGPs. Models include likelihood-based (DeepAR), flow-based (TSFlow), diffusion-based (TimeGrad, RATD), and quantile-based (QuantileFormer) architectures.
Data Efficiency: Learning curve analyses measure performance as a function of sample size to quantify the data requirements of different model classes.

Key Results

1. Mechanism Sensitivity and Inductive Bias

Model performance is governed by architectural inductive bias rather than capacity.

Robustness over Expressiveness: Simple autoregressive and linear models (AR, HAR, DLinear) consistently outperform complex Transformers in volatility-, tail-, and jump-driven environments. Deep models with high capacity tend to overfit idiosyncratic noise in low signal-to-noise regimes.
Local vs. Global Attention: PatchTST (local patch-based attention) generally outperforms global mixing models (iTransformer, TimeXer) in jump-driven environments, suggesting global attention mechanisms may dilute the signal required to predict sudden moves.
Decomposition Mismatch: Models relying on seasonal-trend decomposition (Autoformer, FEDformer) underperform, as financial returns lack the strong periodic structure found in domains like traffic or weather.

2. Distributional Alignment

Parametric Efficiency: Parametric probabilistic models like DeepAR achieve superior calibration in stationary settings (Cases 1–2) because their autoregressive Gaussian likelihood with time-varying scale is structurally isomorphic to GARCH dynamics.
Flexibility for Multimodality: In non-stationary or complex regimes (Case 4: Regime Switching; Case 6: Zero-Inflated Jumps), the unimodal Gaussian assumption of DeepAR fails. Flexible density models like TSFlow (using normalizing flows) outperform DeepAR when the true posterior is multimodal or zero-inflated.
Diffusion Limitations: Diffusion-based models (RATD) struggle with discontinuous mechanisms, degrading significantly under structural breaks where noise distributions shift instantaneously.

3. Data Efficiency

Early Saturation: Neural models often require substantially more data to match simple baselines. In volatility and tail-driven environments, performance saturates rapidly (around 40% of training data), suggesting that additional data primarily samples noise rather than learnable signal.
Regime Complexity: The exception is regime-switching environments, where learning curves remain steep even at large sample sizes, indicating that inferring latent structural changes is an information-dense task.
Calibration Cost: Probabilistic calibration is more data-intensive than point prediction. Flexible generative models require 2–3 $\times$ more data than parametric baselines to achieve competitive calibration.

Significance and Contributions

The paper claims that FinStressTS provides a "middle ground" that combines parametric control with financial realism, enabling "green-box" testing impossible with real data. Its primary contributions are:

Diagnostic Framework: A mechanism-aware benchmark with 30 environments that transforms opaque underperformance into interpretable diagnoses by linking errors to precise structural causes.
Controlled Evaluation: The ability to evaluate probabilistic calibration and data efficiency under known data-generating mechanisms, isolating factors that are confounded in historical data.
Empirical Insights: Evidence that model performance is highly mechanism-dependent; sophisticated neural architectures may underperform simpler alternatives when data is limited or regimes shift.
Open Infrastructure: An open-source framework allowing the community to conduct targeted stress tests, validate robustness claims, and systematically compare models under controlled conditions.

The authors position FinStressTS not as a leaderboard for ranking models, but as a diagnostic instrument to advance risk-aware forecasting by revealing the specific structural vulnerabilities of different architectural choices.