Rethinking respiratory disease forecasting: temporal heterogeneity between surveillance predictors and outcomes drives forecast instability
This study demonstrates that the unstable and seasonally variable relationships between diverse surveillance predictors and respiratory disease outcomes undermine the effectiveness of static forecasting models and long-term historical training, necessitating adaptive strategies and continual evaluation to improve forecast accuracy.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
Every winter, a familiar rhythm returns: viruses that cause colds, the flu, and other respiratory illnesses begin to circulate, pushing hospitals toward their limits and forcing communities to make difficult choices about safety and care. For public health officials, the goal is to see these surges coming before they arrive, allowing them to prepare hospitals, adjust staffing, and advise the public. To do this, scientists have built forecasting systems that act like weather reports for disease. Instead of tracking rain clouds, these systems track signs of infection in the community, such as how many people visit emergency rooms with coughs, how many virus tests come back positive, or even how much virus is washing through the city's sewer systems. The underlying idea has long been that if you look at enough history, you can find a steady pattern: that the relationship between these early warning signs and the number of people who eventually end up in the hospital remains constant over time. If this were true, a model trained on years of past data would simply get better and more accurate as it learned more.
However, a new study from Utah challenges this comforting assumption. Researchers there set out to test whether these relationships between early warning signs and hospital admissions actually stay the same, or if they shift and change as the viruses themselves change. They looked at data for two major respiratory threats: SARS-CoV-2, the virus that causes COVID-19, and influenza. They gathered a wide array of information, including daily counts of hospital admissions broken down by age, emergency room visits where patients showed symptoms of these illnesses, the percentage of virus tests that came back positive, and measurements of viral concentration in wastewater. By comparing these different streams of data against actual hospital numbers over several years, the team discovered that the connections between the warning signs and the outcomes are far from stable. The patterns that held true during one season or for one version of a virus often fell apart during the next.
The researchers found that when they looked at all the data from multiple years at once, the numbers suggested a strong, consistent link between the warning signs and hospitalizations. It appeared that as emergency room visits went up, hospital admissions would reliably follow. But this view was misleading. When the scientists zoomed in to look at specific time periods, such as when a new variant of the virus emerged or during a specific flu season, those strong links often vanished or even reversed. For instance, during the early waves of the pandemic, the data from children and adults did not move together; what was happening in one group told little about what was happening in the other. Later, as new variants appeared, the relationships shifted again. The same was true for the flu, where the connection between wastewater signals and hospital numbers changed from season to season. This means that a model built on the assumption that "more history equals better prediction" is actually training itself on patterns that no longer exist, potentially making its forecasts less accurate.
To see how this played out in practice, the team ran dozens of different forecasting models to predict hospital admissions. They tested two main ways of teaching these models: one method used a fixed, recent window of data, while the other kept adding older and older years of data to the mix, assuming that a larger history would help. The results were surprising. The models that kept piling on old historical data generally performed worse than those that focused only on the most recent weeks. In many cases, the models trained on the longest history of data failed to predict the turning points of an outbreak, such as when cases began to spike or when they started to drop. The models that adapted quickly to the current situation, ignoring the distant past, were often more reliable. This suggests that the world of respiratory viruses is too fluid for a single, static formula. The way the virus behaves, how people seek medical care, and how the virus shows up in our water and clinics are all in a state of constant flux.
The study also revealed that no single type of model or data source was a silver bullet. A method that worked perfectly well during one wave of the pandemic might fail completely during the next. Similarly, a model that predicted the flu peak accurately one year might miss the mark the following season. The researchers found that the best approach was not to rely on one "best" model, but to use a diverse group of many different models working together. By combining the predictions of many different approaches, the system could remain robust even when the underlying rules of the disease changed. This approach acknowledges that the future is not a simple replay of the past. Instead of assuming that the past holds the key to the future, public health forecasters need to constantly check if their tools are still working and be ready to adapt their methods as the virus evolves.
Ultimately, the work from Utah suggests that forecasting respiratory disease is less like predicting the weather, where the physics of the atmosphere remain largely consistent, and more like navigating a river that is constantly changing its course. The water flows differently depending on the season, the rain, and the terrain. The researchers concluded that to stay ahead of these viruses, public health systems must abandon the idea of a single, perfect model trained on all available history. Instead, they must build flexible systems that can evaluate their own performance in real time, discard old assumptions when they no longer fit, and rely on a variety of different perspectives to make sense of a shifting landscape. This requires a continuous partnership between scientists and public health officials, ensuring that the tools used to protect communities are as dynamic and responsive as the threats they are designed to track.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.