📊 statistics

Short-Term Forecasting of Monthly Syphilis Cases Using a Stability and Recency-Aware Selective Ensemble

This study proposes a stability and recency-aware selective ensemble (SRSE) framework optimized by Particle Swarm Optimization (PSO) that significantly outperforms individual and traditional ensemble models in forecasting monthly syphilis cases in Jiangsu Province, offering a robust tool for short-term disease surveillance.

Original authors: Shanhua Zhang, Xinxin Shen

Published 2026-10-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shanhua Zhang, Xinxin Shen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Public health officials face a constant challenge: predicting the future of infectious diseases. To allocate resources effectively, such as vaccines or medical staff, they need to know not just how many people are sick today, but how many will be sick next month. This task relies on analyzing patterns in historical data, a field known as time series forecasting. The core idea is that past trends often hold clues about what comes next, but the future is rarely a perfect copy of the past. Disease rates can shift due to changes in how people behave, how the virus spreads, or even how hospitals report cases. Because of these shifting dynamics, a single mathematical model that works well today might fail tomorrow. Researchers have long sought a way to combine the strengths of different prediction methods to create a more reliable forecast, but doing so without introducing errors or bias requires a careful, disciplined approach.

In a recent study, researchers from Jiangsu Province in China tackled this problem by developing a new method to forecast monthly syphilis cases. Syphilis is a bacterial infection that spreads through sexual contact and other routes, and while it is treatable, it remains a persistent public health concern. The team focused on data from 2021 to 2025, a period that provided a relatively short but critical window of monthly case counts. Their goal was to build a system that could look at the past four years of data and accurately predict the numbers for the following year. They did not rely on just one formula to make these predictions. Instead, they created a "selective ensemble," which is essentially a panel of different forecasting models working together. The innovation in their work lies in how they chose which models to include and how they decided how much trust to place in each one.

The researchers started by testing nine different statistical models, ranging from very simple approaches to more complex ones. Some models were basic, assuming that next month's numbers would simply be the average of the past or the same as the previous year. Others were more sophisticated, designed to detect trends, seasonal cycles, or specific patterns in how the numbers rose and fell. They ran these models against the historical data from 2021 to 2024, but they did not just look at the overall average performance. They realized that a model might be good at predicting the distant past but poor at predicting the most recent months. To address this, they introduced a "recency-aware" system. This system gave more weight to how well a model performed in the months closest to the present, effectively asking, "Is this model still working well right now?"

Furthermore, the team understood that consistency matters as much as accuracy. A model that is wildly accurate one month and wildly inaccurate the next is dangerous for planning. They added a "stability" check to their system, which penalized models that showed large swings in their error rates. By combining these two factors—recent performance and stability—they could calculate a score for each model. They then used a computer optimization technique, similar to how a flock of birds might search for food, to find the perfect mix of settings for their system. This process allowed them to select the best six models from their original nine and assign them specific weights based on their scores. The final forecast was a weighted average of these top performers, rather than a simple average of all of them.

When the researchers tested their new system against the actual data from 2025, which they had kept hidden during the training phase, the results were clear. The new method, which they called PSO-SRSE, outperformed every single individual model and even beat a simple average of all the models combined. In terms of the average number of cases it missed, the new system was off by about 109 cases per month. In comparison, the best single model was off by nearly 140 cases, and a standard average of all models was off by over 114. The new method also reduced the percentage of error significantly, landing at just under 4 percent. These numbers suggest that by carefully selecting which models to trust and how much to trust them, the researchers created a more robust tool for public health planning.

The study also examined how reliable their optimization process was. They ran their computer search algorithm thirty times to see if it would find the same best settings each time. The results were remarkably consistent, showing that the method did not depend on a lucky guess or a random fluctuation in the data. However, the authors noted that their work was based on data from a single province over a short period. While the method proved effective for this specific dataset, they cautioned that it would need to be tested on longer time series and in different regions to confirm its broader usefulness. They also pointed out that future work should look at how well the system handles different timeframes and how it can quantify the uncertainty of its predictions. For now, the study offers a promising step forward, demonstrating that a smart combination of simple and complex tools, guided by recent performance and stability, can provide a clearer view of the future for disease surveillance.

Technical Summary: Short-Term Forecasting of Monthly Syphilis Cases Using a Stability and Recency-Aware Selective Ensemble

Problem Statement
Accurate short-term forecasting of syphilis cases is critical for disease surveillance and public health resource allocation. However, forecasting infectious disease trends faces challenges such as shifting transmission dynamics, reporting practice variations, and limited sample sizes. Traditional approaches often rely on a single "best-performing" model, which may lack adaptability, or simple ensemble methods that assign equal weights, potentially diluting the contribution of superior models. Furthermore, standard model evaluation often fails to account for the temporal decay of predictive relevance, where older validation data may not reflect current forecasting capabilities. This study addresses these issues by developing a framework specifically tailored for short disease surveillance time series (48 monthly observations) to improve point forecasting accuracy and stability.

Methodology
The study proposes a Stability and Recency-Aware Selective Ensemble (SRSE) framework, validated on monthly syphilis case data from Jiangsu Province (January 2021–December 2024) with an independent test set for 2025. The methodology consists of four core components:

Candidate Model Library: To mitigate overfitting risks associated with limited data, nine parsimonious statistical models were selected as candidates: Mean, Naive, Seasonal Naive, Simple Exponential Smoothing (SES), Holt, Damped Holt, Exponential Smoothing State Space (ETS), ARIMA, and Fourier-AR. These cover diverse assumptions regarding level, trend, seasonality, and autocorrelation.
Rolling-Origin Validation with Recency Weighting: An expanding-window rolling-origin strategy was employed to generate out-of-sample forecasts iteratively. Unlike standard validation, this approach introduces an exponential time-decay mechanism. Validation errors from rolling origins closer to the end of the training period are assigned higher weights ( $\gamma$ ), ensuring the model selection reflects recent predictive performance rather than historical averages.
Stability-Aware Model Screening and Weighting:
- Screening: Models are ranked by a composite loss function ( $L_m$ ) that combines the recency-weighted average error ( $\bar{E}_m$ ) and a stability penalty ( $S_m$ ), defined as the standard deviation of errors across rolling windows. Only the top $K$ models are retained.
- Weighting: Non-negative ensemble weights are derived inversely from the composite loss, adjusted by a parameter ( $\eta$ ) to control weight disparity.
Hyperparameter Optimization: The four key hyperparameters ( $K$ , $\gamma$ , $\lambda$ , $\eta$ ) were jointly optimized using Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and Sparrow Search Algorithm (SSA). The objective function was the minimization of the Cross-Validation Mean Absolute Percentage Error (CV-MAPE) derived strictly from the training period to prevent data leakage.

Key Contributions

SRSE Framework: A novel ensemble approach that jointly incorporates recency-weighted validation errors, cross-origin prediction stability, and candidate model screening. This reduces the influence of outdated or unstable forecasts on the final ensemble.
Leakage-Free Evaluation: Implementation of a strict expanding-window rolling-origin strategy where all model selection, stability assessment, and weighting are based solely on out-of-sample forecasts from the training period (2021–2024), reserving 2025 data exclusively for independent testing.
Comparative Optimization Analysis: A systematic comparison of PSO, GA, and SSA for hyperparameter tuning. The study demonstrates that PSO achieves superior repeat-run stability (near-zero standard deviation across 30 runs) and optimal CV-MAPE compared to GA and SSA under identical settings.

Results
The study evaluated the models on the independent 2025 test set using MAE, RMSE, MAPE, and sMAPE.

Performance: The PSO-optimized SRSE (PSO-SRSE) achieved the lowest errors among all compared models: MAE of 109.39, RMSE of 132.35, MAPE of 3.89%, and sMAPE of 3.87%.
Comparative Advantage: PSO-SRSE outperformed the unoptimized SRSE (MAPE 4.16%), the equal-weighted ensemble of all nine models (MAPE 4.03%), and the best individual candidate model, ARIMA(1,0,0) (MAPE 4.83%).
Model Composition: The final ensemble selected by PSO-SRSE retained six models (ARIMA, Mean, SES, Naive, ETS, Damped Holt), with ARIMA receiving the highest weight (19.14%) due to its low recency-weighted error and stability.
Sensitivity: Sensitivity analysis indicated the model is most sensitive to the number of retained models ( $K$ ) and the weight adjustment parameter ( $\eta$ ), while remaining relatively robust to variations in the stability penalty ( $\lambda$ ) and time-decay coefficient ( $\gamma$ ).

Significance and Claims
The authors claim that the proposed PSO-SRSE framework offers a robust method for short-term point forecasting of disease surveillance time series with limited sample sizes. By explicitly addressing the temporal relevance of validation data and the stability of model performance across different time windows, the method improves upon traditional single-model and static ensemble approaches. The study concludes that while the method shows promise for monthly syphilis surveillance, its application is currently limited to the specific regional dataset used, and future work should involve external validation across different regions, diseases, and longer time series to confirm generalizability.

Technical Summary: Short-Term Forecasting of Monthly Syphilis Cases Using a Stability and Recency-Aware Selective Ensemble

More like this