A new recurrent neural network for forecasting seasonal time series

Technical Summary: A New Recurrent Neural Network for Forecasting Seasonal Time Series

Problem Statement
Seasonal time series forecasting requires specialized models capable of handling periodic structures. While statistical methods like SARIMA and exponential smoothing (ETS) are traditional standards, and Artificial Neural Networks (ANNs) offer non-linear modeling capabilities, existing ANN approaches often rely on external decomposition techniques or simply include lagged variables as inputs to capture seasonality. The authors note that standard ANNs do not inherently utilize the logic of exponential smoothing, where feedback on model forecasts can enhance performance. Furthermore, traditional gradient-based learning in ANNs is susceptible to getting trapped in local optima, and standard error metrics (like MSE) treat all historical errors equally, potentially neglecting the critical importance of recent observations in forecasting.

Methodology
The paper proposes a novel Recurrent Artificial Neural Network (RNN) specifically designed to model seasonal dynamics without prior data decomposition. The architecture and learning process are defined as follows:

• Network Architecture: The proposed network consists of two distinct operational paths synthesized by a combination parameter ($\beta$):

  1. Convolutional Neuron Topology: This component processes $p$ lagged observations ($x_{t-j}$) using weights and biases, integrated with a logistic activation function to introduce non-linearity.
  2. Seasonal Recurrent Smoothing: This component mimics exponential smoothing logic. It takes current and seasonal lagged inputs ($x_{t-1}$ and $x_{t-s}$) and incorporates feedback from historical network outputs ($\hat{x}_{t-1}$ and $\hat{x}_{t-s}$). This path utilizes smoothing coefficients ($\alpha_1, \alpha_2$) and a weight factor ($\alpha_3$) to model level and seasonal components.
     The final output is a weighted sum of these two paths.

• Learning Algorithm (PSO): To avoid local optima traps common in gradient-based methods, the network parameters (weights, biases, smoothing coefficients, and combination factors) are optimized using Particle Swarm Optimization (PSO).

  • Objective Function: A new objective function, Weighted Mean Squared Error (WMSE), is introduced. Unlike standard MSE, WMSE assigns exponentially higher weights to recent observations, prioritizing the accuracy of the most recent data points.
  • Training Strategies: The training process employs an early stopping mechanism to prevent overfitting and a dynamic restarting strategy to re-initialize the swarm if the optimization stagnates, ensuring the search for a global optimum.

• Experimental Setup: The model was evaluated on six diverse seasonal time series datasets:

  1. Logarithmic international airline passengers (Monthly, $s=12$).
  2. Australian beer consumption (Quarterly, $s=4$).
  3. Ankara air pollution ($SO_2$) (Monthly, $s=12$).
  4. Turkey electricity consumption (Monthly, $s=12$).
  5. Giresun City air pressure (Monthly, $s=12$).
  6. Giresun City air temperature (Monthly, $s=12$).

  Performance was benchmarked against Long Short-Term Memory (LSTM), Pi-Sigma Artificial Neural Network (PS-ANN), and Simple Recurrent Neural Network (SRNN). Each model underwent hyperparameter tuning via a validation set, and final performance was assessed on a test set over 30 independent runs to ensure statistical reliability.

Key Results
Across all six datasets, the proposed network demonstrated superior forecasting performance compared to the benchmark models:

• Accuracy: The proposed method consistently achieved the lowest Mean Root Mean Squared Error (RMSE) on the test sets. For instance, in the airline passenger dataset, it achieved a mean RMSE of 0.089 compared to 0.271 for LSTM and 0.317 for SRNN.
• Stability: The proposed network exhibited significantly lower standard deviations in error metrics across the 30 runs, indicating greater robustness against random initialization compared to the volatile performance of SRNN and PS-ANN.
• Statistical Significance: Non-parametric Wilcoxon signed-rank tests were conducted to compare the distributions of errors. The results yielded $p$-values well below the $10^{-3}$ threshold for all comparisons, confirming that the performance improvements of the proposed network over the benchmarks are statistically significant and not due to chance.

Significance and Claims
The authors claim that this study bridges the gap between classical exponential smoothing philosophies and modern deep learning. By integrating seasonal smoothing logic and dynamic feedback loops directly into the neural network architecture, the model captures complex non-linear dynamics and seasonal dependencies without requiring pre-processing decomposition.

The significance of the work lies in:

1. Architectural Innovation: Creating a recurrent structure that inherently respects seasonal periods through feedback mechanisms similar to exponential smoothing.
2. Optimization Advancement: The use of PSO with a custom WMSE objective function and restart strategies effectively addresses the challenges of local optima and the need to prioritize recent data trends.
3. Empirical Validation: The consistent outperformance across economic, environmental, and utility datasets suggests the model is a robust, mathematically sound alternative for complex seasonal forecasting problems.

The paper concludes that the proposed network offers a highly stable and accurate solution for seasonal time series, with future work potentially exploring extensions to multiple and nested seasonal periods.