📄 public and global health

Probabilistic forecasting of prefecture-level influenza in Japan accounting for surveillance data release histories

This study evaluates short-term probabilistic forecasts for prefecture-level influenza in Japan using models that account for surveillance data release histories, finding that while combined models outperform persistence baselines, further prospective validation is required before they can reliably support public-health decision-making due to challenges in real-time lead time, threshold detection, and system changes.

Original authors: NAKAGAWA, R., Yamamura, N.

Published 2026-10-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: NAKAGAWA, R., Yamamura, N.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Public health officials do not merely need to know how many people are sick today; they need to know how many will be sick next week, and how certain they can be about that number. When influenza spreads, the difference between a manageable surge and an overwhelmed hospital system often comes down to timing. To prepare, health departments rely on surveillance systems that track doctor visits and test results, but these reports are never perfect. They arrive with a delay, and the numbers often change as more data is collected or as corrections are made. A forecast that looks accurate on a computer screen might fail in the real world if it assumes the officials have information they do not actually possess yet. The goal of modern forecasting is not just to predict a single number, but to provide a range of possibilities that honestly reflects this uncertainty, allowing leaders to see if a dangerous rise is coming before it happens.

In a recent study focused on Japan, researchers set out to test whether advanced computer models could predict the spread of influenza across the country's 47 prefectures while respecting these real-world delays and data revisions. The team, led by Ryutaro Nakagawa and Nami Yamamura, looked at six past flu seasons to build and refine their models, then tested them against the 2025/26 season. They compared three different approaches: a model that simply assumed the next week would look like the current one, a model that learned from data across all prefectures at once, and a more complex model that combined those two methods while also pulling in data from other countries and different types of health reports. The researchers were careful to simulate the exact conditions of the past, ensuring that when they made a prediction for a specific week, they only used the data that was actually available at that moment, ignoring any later corrections or updates.

The results showed that the more complex models were indeed better than the simple assumption that things would stay the same. When the researchers measured the accuracy of their predictions, the combined model was about 16 to 21 percent more accurate than the simple approach across the four weeks they tested. This improvement came from the model's ability to share information between different regions and to learn from a wider variety of health indicators. However, the study also revealed that this advantage was not as clear-cut as it first appeared. When the researchers looked at the raw numbers without mathematical adjustments, the difference between the best combined model and the simpler regional model was small and not statistically certain. In some specific situations, such as when flu cases were just starting to rise from very low levels, the complex model actually performed worse than the simpler one, failing to catch the early upward trend as quickly.

A critical finding of the study was that the models struggled to predict the exact upper limits of an outbreak during rapid growth. While the models were generally good at guessing the middle of the range, they sometimes underestimated how high the numbers could go when the virus was spreading quickly. This is a significant issue for public health, because missing the peak of an outbreak by even a small margin can mean the difference between having enough hospital beds and running out. The researchers also noted that Japan recently changed its surveillance system, which means that old warning thresholds used to trigger alerts are no longer directly comparable to the new data. Because of this, and because the models sometimes missed the early signs of a rise, the authors caution that these forecasts are not yet ready to be used as the sole basis for issuing official warnings or planning clinical surges.

The study concludes that while sharing information across regions helps improve forecasts, the current evidence is not strong enough to say these models are fully reliable for making high-stakes decisions. The researchers emphasize that before such tools can be trusted to guide public health actions, they need to be tested in a strictly controlled, forward-looking manner where the data availability is fixed in advance and the models are not tweaked based on what has already happened. Until a prospective test confirms that these models can consistently catch rising trends early and provide accurate warnings without false alarms, health officials should view them as helpful tools for understanding the situation rather than as definitive guides for action. The path forward requires more than just better algorithms; it demands a rigorous process that accounts for the messy reality of how data is collected, delayed, and revised in the real world.

Technical Summary: Probabilistic Forecasting of Prefecture-Level Influenza in Japan Accounting for Surveillance Data Release Histories

Problem Statement
Public health operations regarding influenza—specifically situational awareness, clinical surge planning, and the issuance of advisories—require more than accurate point estimates of future case counts. Operational utility depends on correctly identifying the direction of change, detecting rises from low levels early, and predicting when alert thresholds will be crossed, all while explicitly quantifying uncertainty. A critical challenge in this domain is that surveillance data available at the time of forecasting often differs from data retrieved later due to reporting delays and subsequent revisions. Evaluations based on later-curated data can inadvertently assume information unavailable in real-time, leading to overly optimistic assessments of forecast performance. Furthermore, existing models often fail to account for the specific data release history of the Japanese sentinel surveillance system, where provisional weekly reports are published without containing prior weeks' data, and where structural changes (such as the 2025 sentinel system update) alter reporting baselines.

Methodology
The study conducted a retrospective evaluation of short-term probabilistic forecasts for the number of influenza reports per sentinel site across Japan's 47 prefectures. The target variable was the value for weeks $t+h$ (where $h=1,2,3,4$ ) relative to the most recently observed week $t$ .

Data Handling: The authors reconstructed historical data conditions to respect the "release history." The target series utilized the first-published value for each week from the Infectious Disease Weekly Report (IDWR). Unlike the target series, some auxiliary series were treated as retrieved data, potentially including later revisions. The study explicitly distinguished between the observed target week, release date, retrieval date, and the version of values used for scoring.
Models Compared:
1. Persistence Forecast: A probabilistic baseline using the most recent observed value with intervals derived from past errors, corrected for current level and trend.
2. Ridge Regression: A model pooling data across all 47 prefectures, predicting log-changes using covariates including local levels, recent trends, national/neighboring trends, seasonality, and holidays.
3. Gradient-Boosting Quantile Regression (GBQR): A model learning from pooled examples across regions and multiple surveillance series (including 77 auxiliary series such as US ILINet, WHO FluNet, and other national surveillance).
4. Combination Model: An equal-weight average of the Ridge and GBQR predictions on the log scale.
Evaluation Period: Six seasons (2017/18 through 2024/25) were used for development. The 2025/26 season served as a temporal evaluation period with fixed specifications, though it was not a fully independent prospective test as its epidemic profile was partially known.
Metrics: Forecasts were scored using the Weighted Interval Score (WIS) on both original and log scales. Additional metrics included prediction interval coverage (80% and 95%), agreement in the direction of change, and discrimination of threshold exceedance.

Key Results

Overall Performance: In the 2025/26 season, the combination model outperformed the persistence forecast, achieving relative WIS values of 0.841, 0.811, 0.794, and 0.818 at horizons 1 through 4 (representing 16–21% improvement over persistence).
Model Comparison: While the combination model showed lower WIS than the Ridge regression alone, the difference on the original scale was not statistically significant (95% confidence intervals included zero). On the log scale, the combination showed statistically significant improvements over Ridge in the development period, but this did not fully translate to the original scale in the evaluation period.
Auxiliary Data: Adding auxiliary surveillance series improved the GBQR model during the development period. However, the specific causal contribution of individual data sources could not be isolated due to simultaneous changes in training examples, countries, and indicator types.
Interval Coverage: Prediction interval coverage for the 80% and 95% intervals generally fell below nominal levels at longer horizons (e.g., 75.1% and 91.8% for 80% and 95% at $h=4$ ).
Phase-Specific Weaknesses: In rapidly rising phases starting from low levels (pre-epidemic onset), the combination model performed worse than the Ridge model on the original scale. For example, at horizon $h=3$ in rising phases with low activity, the Ridge model had a relative WIS of 0.683 compared to 0.801 for the combination. This suggests that overall mean performance can mask failures in critical decision-making scenarios.

Significance and Claims
The paper claims that its primary contribution is not a novel algorithm, but rather a rigorous, joint examination of release history, cross-region learning, and auxiliary series integration within the specific context of Japanese sentinel surveillance.

Operational Reality: The authors emphasize that a marginal improvement in an overall score (WIS) does not automatically justify the operational complexity of adopting a more complex model. For public health use, the decisive factors are real lead time (accounting for reporting delays), the ability to detect rises from low levels, and the reliability of upper prediction intervals.
Limitations of Current Evidence: The study concludes that current evidence is insufficient to support the use of these forecasts for issuing advisories or planning. Key barriers include the under-coverage of intervals during rapid rises, the 2025 sentinel system change which renders pre-change thresholds non-comparable, and the retrospective nature of the evaluation.
Future Requirements: The authors assert that before such forecasts can be used for decision support, a strictly controlled prospective evaluation is required. This future validation must fix model specifications and data availability rules in advance, save forecasts with issue timestamps, and rigorously test performance across different epidemic shapes, particularly regarding early detection and threshold exceedance.

The paper maintains a modest stance, noting that while cross-region models outperformed simple persistence, the specific benefits of model combination and auxiliary data remain uncertain for real-world operational decision-making without further prospective validation.

Technical Summary: Probabilistic Forecasting of Prefecture-Level Influenza in Japan Accounting for Surveillance Data Release Histories

More like this