Local Influenza Forecasts Outperform State-Level Forecasts in the United States

Technical Summary: Local Influenza Forecasts Outperform State-Level Forecasts in the United States

Problem Statement
Accurate infectious disease forecasting is essential for public health planning, yet a critical gap exists in translating national and state-level insights to the local scale. In the United States, most influenza forecasts are generated at the state level, which can mask substantial sub-state heterogeneity in disease dynamics. This limitation is particularly pronounced in large states (e.g., Texas, California) where metropolitan areas experience distinct epidemic trajectories driven by demographic, mobility, and healthcare infrastructure differences. Consequently, decision-makers in large states often rely on highly aggregated forecasts that may not reflect local realities, while those in smaller states benefit from more geographically precise projections. The central question addressed is whether influenza forecasts generated using local Health Service Area (HSA) data are more accurate than those derived solely from state-level data when evaluated at the local scale.

Methodology
The study utilized weekly estimates of the percentage of Emergency Department (ED) visits attributable to influenza from the CDC's National Syndromic Surveillance Program (NSSP). The analysis focused on 173 large metropolitan Health Service Areas (HSAs) across 31 states, excluding regions with populations under 250,000 to mitigate stochastic noise associated with smaller sample sizes.

• Forecasting Models: The primary forecasting engine was a Gradient Boosting Quantile Regression (GBQR) model, selected for its strong performance in recent CDC FluSight evaluations. This model generates full predictive distributions across quantiles. For comparison, the authors also implemented an automated ARIMA model and a quantile baseline model.
• Experimental Design: Retrospective forecasts were generated for the 2022–2023, 2023–2024, and 2024–2025 influenza seasons. Models were trained on data from the other two seasons plus observations available up to the forecast date within the target season. Forecasts were produced weekly from October through March for horizons of 1 to 4 weeks.
• Evaluation Metrics: Performance was evaluated at the HSA level using three standard metrics:
  1. 95% Prediction Interval Coverage: Assessing probabilistic calibration.
  2. Mean Absolute Error (MAE): Assessing point forecast accuracy.
  3. Mean Weighted Interval Score (MWIS): A composite metric evaluating both accuracy and uncertainty quantification.
• Predictor Analysis: To identify where local forecasting offers the greatest value, the authors employed a Generalized Linear Model (GLM) to analyze the relationship between forecast improvement (difference in MWIS between local and state models) and regional characteristics. Predictors included the HSA-to-state population ratio, the proportion of the HSA population living in urban areas, and the number of Metropolitan Statistical Areas (MSAs) within a state.

Key Results

• Superiority of Local Forecasts: Local (HSA-level) forecasts consistently outperformed state-level forecasts across all horizons. At 1-, 2-, and 3-week horizons, local forecasts achieved lower MWIS values in 98.8%, 90.8%, and 78.6% of HSAs, respectively.
• Magnitude of Improvement: The mean weighted interval score for local forecasts was, on average, 39.2% lower than state-based forecasts at the one-week horizon, decreasing to 19.6% and 11.4% at two and three weeks, respectively.
• Calibration: Local forecasts demonstrated better calibration. At the one-week horizon, local models achieved a mean coverage rate of 0.91 compared to 0.59 for state models (targeting 0.95). This gap persisted, though narrowed, at longer horizons.
• Drivers of Improvement: The GLM analysis revealed that the performance advantage of local forecasting was most pronounced in:
  • HSAs representing a smaller share of their state's total population.
  • Highly urbanized HSAs.
  • States subdivided into a larger number of MSAs (indicating higher sub-state fragmentation).
• Geographic Variation: Projected gains were highest in fragmented, multi-metro states like California and Texas, and parts of the Southeast, while homogeneous, sparsely populated regions showed more limited benefits.

Significance and Claims
The paper claims that fine-scale modeling can substantially improve forecast accuracy for influenza, particularly in mid-sized to large jurisdictions within heterogeneous states. The authors argue that local forecasts are not only feasible but consistently more accurate than state-level projections when applied to local areas.

The significance of these findings lies in their potential to improve public health preparedness and response. Decisions regarding healthcare surge management, targeted outreach, and risk communication are often made at local or regional levels. By providing forecasts that capture local epidemic dynamics rather than smoothed state averages, public health officials can receive more timely and actionable guidance. The study suggests that targeted investment in local forecasting capacity is most likely to yield meaningful returns in large, demographically diverse states where epidemic timing and intensity vary significantly across regions.

Limitations
The authors note several limitations:

• The analysis is restricted to HSAs with populations >250,000; performance in smaller or rural jurisdictions may differ due to increased stochastic noise.
• The regression analysis used interpretable population-structure variables rather than maximizing predictive accuracy via complex machine learning, potentially missing higher-order interactions.
• The comparison assumes state forecasts are applied directly to local areas without recalibration, which may not reflect all operational practices.
• The study focuses on the percentage of ED visits, which serves as a normalized indicator of dynamics but is not a direct measure of disease burden (e.g., hospitalizations or deaths).