Aggregate Calibration Can Mask Stress-State Undercoverage: Bounded-Memory Forecast Intervals under Recurrent Financial Stress

In the world of finance, uncertainty is not a flaw; it is a feature. Markets do not move in straight lines. They lurch, they pause, and they sometimes erupt into periods of intense volatility that can wipe out savings or trigger global economic shifts. Because these dangerous moments are so consequential, investors and institutions rely on forecast intervals—ranges that predict where a price will land tomorrow. A good forecast does not just guess a single number; it provides a safety net, a band of likely outcomes that is wide enough to catch the truth most of the time. The standard for judging these safety nets is called calibration. If a forecast claims to be right 90 percent of the time, it should, in the long run, contain the actual price 90 percent of the time.

For decades, statisticians have assumed that if a system meets this average target, it is doing its job. However, financial markets have a peculiar habit: they are not random. They are episodic. A calm period can last for years, only to be followed by a sudden return of the chaos that existed a decade ago. This creates a hidden trap for the systems designed to predict them. If a computer program only remembers the most recent data to make its decisions, it might forget the lessons learned during the last crisis. When that specific type of crisis returns, the system, having never seen it in its short-term memory, might fail to widen its safety net, leaving investors exposed just when they need protection the most.

This is the precise problem researchers Ruihang Han and Li-Hsiang Lin set out to investigate. They built a controlled laboratory to test how different ways of storing and using historical data affect the reliability of financial forecasts. Their work focuses on a specific type of memory constraint: a system that can only hold a limited amount of past information. In the real world, computers cannot store every single day of market history forever; they must discard old data to make room for new data. The standard approach is to use a "first-in, first-out" method, where the oldest data is kicked out the moment the memory fills up, regardless of what that data represents. The researchers asked a simple but profound question: when a familiar type of market stress returns after a long absence, does a system that only remembers the recent past still have the right tools to handle it?

To find the answer, the team constructed a forecasting system designed to predict the daily range of prices for a wide variety of financial assets, including exchange-traded funds and a fixed set of fifty major U.S. stocks. They used data spanning from 2005 through 2025, a period that included multiple distinct eras of market calm and several sharp periods of high volatility. They defined "stress" using a well-known measure of market fear, the VIX index. When this index was high, the market was in a state of stress; when it was low, the market was calm. The researchers then ran their forecasting model under different memory rules, keeping everything else exactly the same. They compared a standard system that simply kept the most recent 250 days of data against a more sophisticated system that remembered not just the recent past, but also specific historical records from previous periods of high stress.

The results revealed a startling disconnect between the overall performance of the system and its performance during the moments that mattered most. The standard system, which only looked at the most recent data, appeared to be working perfectly when viewed from a distance. Over the entire evaluation period, it successfully predicted the price range 90 percent of the time, hitting its target exactly. However, when the researchers zoomed in on the 156 days when the market was under high stress, the picture changed dramatically. On those specific days, the standard system's accuracy plummeted. It failed to contain the actual price nearly a quarter of the time, dropping to a coverage rate of roughly 76 percent. The system was not broken; it was simply blind to the type of danger it was facing because that danger had not appeared in its short-term memory.

In contrast, the system designed to remember specific states of stress performed significantly better. By keeping a dedicated archive of past high-volatility days and recalling them when similar conditions returned, this system maintained a much higher level of accuracy during the crisis periods. It caught the true price range about 85 percent of the time during stress, a substantial improvement over the standard approach. The researchers found that this advantage was most pronounced right at the moment a stress period began, the exact time when the standard system was most likely to forget the lessons of the past. As the stress period continued and the standard system began to accumulate new data from the current crisis, the gap between the two systems narrowed, but the initial failure of the standard system was significant.

The study went further to ensure that this improvement was not just a result of having more data or a different way of weighing information. They created several control groups to test specific variables. One group had the same amount of memory as the improved system but only used recent data. Another group had access to old data but treated all old days as the same, without distinguishing between calm and stressful periods. Neither of these variations could match the performance of the system that specifically remembered the structure of past stress. This proved that the benefit came from the specific composition of the memory—knowing which historical days were relevant to the current situation—rather than simply having a larger database.

The researchers also examined how long this advantage lasted. They found that the benefit of remembering past stress was concentrated at the very beginning of a crisis. When a high-volatility period started, the system with the structured memory immediately knew how to widen its safety net because it had seen this pattern before. The standard system, however, had to wait for enough new data to accumulate in its short-term buffer before it could adjust. By the time the standard system caught up, the most dangerous initial phase of the crisis had often already passed. This timing difference highlights a critical flaw in relying solely on recent history: the most dangerous moments are often the ones that require the most immediate and accurate response, yet they are the ones most likely to be forgotten by a system that only looks backward a few months.

The implications of this work extend beyond the specific numbers of the study. It challenges the assumption that a system is reliable simply because it works well on average. In fields where rare but catastrophic events recur, average performance can be a misleading metric. A system can be perfectly calibrated for the 90 percent of the time when the market is calm and still fail catastrophically during the 10 percent of the time when it matters most. The researchers demonstrated that the way a system stores its history is just as important as the algorithm it uses to make predictions. If a system is forced to forget the past, it must be designed to remember the right things.

The study was conducted with rigorous controls, using the same forecasting model and the same market data for every test. The results were consistent across two different panels of assets, suggesting that the finding is robust and not limited to a specific type of stock or fund. The researchers also tested whether their findings held up when they changed the definition of stress or the size of the memory buffer, and the core pattern remained: systems that forget the specific nature of past crises are less reliable when those crises return. This is not a theoretical possibility but an observed reality in the data.

Ultimately, the paper argues that the design of a forecasting system must include a strategy for memory. It is not enough to simply store the most recent data points. For systems that operate in environments where conditions change and then return, the ability to retrieve relevant historical evidence is a fundamental part of the prediction process. The researchers showed that by structuring memory to preserve evidence of rare but recurring states, we can build systems that are not just accurate on average, but reliable when they are needed most. In a world where financial stress is episodic and recurrent, the most important data might be the oldest data, provided we have a way to find it again.