💰 quantitative finance

Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecasting

This paper demonstrates that while activation calibration has minimal impact on 8-bit post-training quantization for financial forecasting, it is the critical factor determining predictive performance at 4-bit precision, where percentile-based calibration significantly outperforms standard absolute-maximum methods to mitigate information loss.

Original authors: Junyi Ye, Ivy Gateri Wanjiku

Published 2026-08-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junyi Ye, Ivy Gateri Wanjiku

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the weather. You give it a super-smart brain (a neural network) that has studied years of rain, wind, and sunshine. This brain is incredibly powerful, but it's also huge and heavy, like a giant mainframe computer from the 1980s. To make it useful for a real-world job—like running on a tiny drone or a smartphone—you need to shrink it down. You can't just throw away its knowledge, so instead, you try to simplify how it stores numbers. This process is called quantization. Think of it like converting a high-definition movie into a compressed file: you lose a tiny bit of detail to save massive amounts of space and make it run faster.

However, there's a tricky part. While the robot's "knowledge" (its weights) is fixed once trained, its "thoughts" (its activations) change every time it looks at new data. To shrink these thoughts, you have to decide on a "safe zone" or a range of numbers the robot is allowed to use. If you set the safe zone too wide, you waste space on numbers that never happen. If you set it too narrow, you clip off the extreme numbers that actually do happen, like a sudden storm. This decision is called calibration. Usually, scientists just pick a range based on the data they have right now and hope it works for the future. But in the world of finance, where markets can go from calm to chaotic in a heartbeat, guessing the wrong range could mean the difference between a profitable trade and a disaster.

This paper, titled "Calibration Bets on the Past," dives deep into this exact problem for financial forecasting. The authors, Junyi Ye and Ivy Gateri Wanjiku, asked a simple but critical question: When we shrink financial AI models to run on cheap hardware, does the way we set that "safe zone" for the robot's thoughts actually matter? They didn't just guess; they ran a massive experiment. They took seven different types of AI models, trained them on stock market data from the S&P 500, and then tested them over eight different years (from 2018 to 2025). They simulated shrinking these models down to very low precision (4 bits, which is like having only 15 possible numbers to work with instead of millions) and watched what happened.

Here is what they found, and it's a bit of a plot twist. First, they discovered that if you only shrink the robot's "memory" (the weights) but keep its "thoughts" (activations) in high definition, or if you shrink both but keep them at 8 bits, the robot barely notices. It still predicts the market almost as well as the giant version. But, if you shrink the thoughts down to 4 bits using the standard, default method (which just looks at the biggest number seen so far and sets the limit there), the robot's performance crashes. In some cases, it lost up to 62% of its predictive power! It was as if the robot suddenly forgot how to read the weather map.

However, the story doesn't end with failure. The authors found that the crash wasn't inevitable; it was just a bad choice of "safe zone." By changing the calibration method to look at a specific percentile (like saying, "We'll only worry about the top 99% of numbers, not the absolute wildest outlier"), they could recover most of the lost power. For some models, this simple tweak brought back 94% of the performance. But here's the catch: the "perfect" safe zone changes depending on the market. A range that works great during a calm year might fail miserably during a market crash, and vice versa.

The paper concludes that setting this range isn't just a boring technical detail you set once and forget. It's a major strategic decision, like betting on the weather. If you want to run these financial models on low-power devices, you can't just use the default settings. You have to carefully tune the "safe zone" based on the market conditions, or be ready to switch back to a slightly larger (8-bit) setting if the market gets too wild. The authors show that with the right calibration, we can make these powerful financial AI models small and fast without losing their ability to predict the future.

Technical Summary: Calibration Bets on the Past

Problem Statement

Financial forecasting models are typically trained in full precision (FP32) but often require low-precision inference for production deployment to satisfy memory, latency, and hardware constraints. While Post-Training Quantization (PTQ) allows for the deployment of these models without retraining, the reliability of activation quantization remains a critical bottleneck. Unlike weights, which have fixed ranges determined by trained parameters, activation ranges must be estimated from a historical calibration dataset and then remain fixed during future inference.

This creates a fundamental deployment challenge: calibration relies on historical market data, while deployment occurs under future market conditions that may differ substantially. A wide activation range preserves rare extremes but reduces numerical resolution for typical values, whereas a narrow range improves resolution but increases clipping. In financial forecasting, where market regimes shift, the choice of a static activation range is a high-stakes decision that is poorly understood in the context of PTQ. Existing research has focused on vision and language models, leaving a gap in understanding how activation calibration affects predictive performance in time-series financial forecasting.

Methodology

The authors present a systematic study of activation calibration for PTQ in cross-sectional volatility forecasting on the S&P 500.

Dataset and Task: The study utilizes a daily panel of 501 S&P 500 constituents from June 2008 to December 2025. The task is to predict five-day-forward volatility. The performance metric is the Information Coefficient (IC), defined as the cross-sectional Spearman correlation between predicted and realized volatility scores.
Experimental Protocol: A walk-forward evaluation protocol is employed across eight test years (2018–2025). For each test year $Y$ , models are trained on $Y-8$ to $Y-2$ , validated on $Y-1$ , and tested on $Y$ . This setup ensures that calibration data (from $Y-1$ ) precedes the test period ( $Y$ ).
Models: Seven representative neural architectures are evaluated: DLinear, TSMixer, TimeMixer, vanilla Transformer, PatchTST, iTransformer, and SegRNN. A total of 560 independently trained models (7 architectures $\times$ 8 folds $\times$ 10 seeds) are analyzed.
Quantization Settings: The study compares several precision treatments:
- FP32: Full-precision baseline.
- W8A8 / W4A4: Static quantization of weights and activations (INT8/INT4) using a fixed calibration range.
- W4: Weight-only quantization (INT4 weights, FP32 activations).
- Dynamic INT8: Activation ranges recomputed during inference (used as a reference).
Calibration Strategies: Two primary strategies for setting the activation range $A$ $A$ are compared:
- Abs-Max: $A = \max(|x|)$ over the calibration set.
- Percentile: $A = P_p(|x|)$ , selecting the $p$ -th percentile of activation magnitudes (e.g., p99, p99.9).
Damage Metric: Quantization damage ( $\Delta$ ) is defined as the difference between the full-precision IC and the quantized IC.

Key Contributions

First Systematic Study: This is the first systematic investigation of activation calibration for PTQ in financial forecasting under a walk-forward evaluation protocol.
Decomposition of Degradation: The authors reveal that 4-bit activation degradation consists of two components:
- Range-Recoverable Loss: Damage caused by suboptimal range selection, which can be significantly mitigated by switching from abs-max to percentile calibration.
- Residual Loss: Architecture-dependent damage that persists even with optimal range selection, particularly in recurrent (SegRNN) and multi-scale (TimeMixer) models.
Market Regime Sensitivity: The study demonstrates that the preferred activation range is not static; it evolves across market periods. Narrow ranges (percentiles) perform well in typical conditions but suffer significantly when test-period market dispersion exceeds the calibration history (e.g., during stress events like the 2020 COVID shock).
Practical Deployment Guidelines: The paper provides a decision framework for selecting quantization strategies based on observed damage patterns, ranging from simple percentile tuning to fallback strategies like 8-bit activations or weight-only quantization.

Key Results

8-bit and Weight-Only Quantization: Both 8-bit activation quantization (W8A8) and 4-bit weight-only quantization (W4) introduce negligible predictive damage (typically <0.2% of FP32 signal) across most architectures.
Static 4-bit Activation (W4A4) Risks: Under the default abs-max calibration, static 4-bit activation quantization causes substantial performance degradation, ranging from 11% to 62% of the full-precision IC depending on the architecture.
- Most Affected: TimeMixer (62% loss) and SegRNN (59% loss).
- Least Affected: DLinear (minimal loss, though its baseline IC is near zero).
Recovery via Percentile Calibration: Replacing abs-max with percentile calibration (e.g., p99 or p99.9) recovers a significant portion of the lost performance:
- Transformer: Recovers 94% of the damage.
- TSMixer: Recovers 80%.
- SegRNN: Recovers 73%.
- TimeMixer: Recovers 53%.
- Despite recovery, recurrent and multi-scale models retain substantial residual sensitivity, suggesting that range selection alone cannot fully resolve their quantization issues.
Layer-Wise Sensitivity: For the Transformer architecture, a single layer (convolutional token embedding) accounts for 72% of the total W4A4 damage. Keeping this specific layer at 8-bit precision while quantizing the rest to 4-bit effectively eliminates the degradation.
Regime Mismatch: When the test market is more volatile than the calibration period (e.g., 2020), narrow percentile ranges clip critical tail activations, causing damage 1.5x to 3.6x higher than inside the calibration envelope. Conversely, in calmer periods (e.g., 2021), abs-max calibration becomes suboptimal because it sacrifices resolution for extremes that do not occur.

Significance and Claims

The paper argues that activation calibration is not merely a numerical implementation detail but a first-class deployment decision for reliable 4-bit PTQ in financial forecasting.

Strategic Implication: The choice of activation range directly determines whether a low-precision model retains its predictive utility. Default strategies (abs-max) are often insufficient for financial time-series due to the non-stationary nature of market data.
Robustness: While 8-bit quantization and weight-only 4-bit quantization offer robust deployment paths with minimal signal loss, 4-bit activation quantization requires careful calibration.
Actionable Guidance: The authors propose a deployment procedure:
1. Measure default W4A4 damage.
2. If damage is large, sweep percentile ranges on pre-deployment data to estimate recoverable vs. residual loss.
3. If residual loss is high, fall back to 8-bit activations, weight-only 4-bit, or layer-wise mixed precision.
4. Monitor the "calibration envelope" (the maximum dispersion observed during calibration) during deployment; if test conditions exceed this envelope, trigger recalibration or switch to a more conservative range (e.g., p99.9) or higher precision.

The study concludes that while PTQ is viable for financial forecasting, its success hinges on treating activation calibration as a dynamic policy decision rather than a fixed preprocessing step.