A Temporal Deep Feature Transfer Hybrid of LSTM and Gradient Boosting Machine for Multi Granularity Smart Grid Electricity Demand Forecasting

Technical Summary: A Temporal Deep Feature Transfer Hybrid of LSTM and Gradient Boosting Machine for Multi-Granularity Smart Grid Electricity Demand Forecasting

Problem Statement
Accurate electricity demand forecasting across various temporal granularities (daily and monthly) is critical for the efficient management of smart grids, influencing generation dispatch, transmission management, and demand response. However, existing methods face significant challenges when predicting heterogeneous consumer groups (e.g., agricultural, commercial, industrial, residential, street lighting).

• Classical Limitations: Traditional statistical models like ARIMA, Exponential Triple Smoothing (ETS), and Weighted Moving Average (WMA) rely on assumptions of linearity and stationarity. These assumptions are frequently violated in smart grid data due to non-linear consumer behaviors, industrial cycles, and category-level heterogeneity. The study notes that these methods yield high error rates, with MAPE values exceeding 340% to over 550% in certain scenarios.
• Deep Learning Limitations: While Long Short-Term Memory (LSTM) networks excel at temporal modeling, standard hybrid approaches often feed raw features or scalar outputs from the LSTM's final layer directly into downstream regressors (like Gradient Boosting Machines). This approach fails to fully utilize the rich temporal information encoded in the intermediate layers of the LSTM, leading to sub-optimal regression precision.

Methodology: The TDF-LSTGBM Architecture
The paper proposes TDF-LSTGBM, a novel two-stage supervised learning pipeline designed to decouple temporal representation learning from final regression. The core innovation is a Temporal Deep Feature Transfer (TDF) mechanism.

1. Stage 1: Stacked LSTM Temporal Encoder (Feature Extraction)

   • Architecture: A stacked LSTM encoder consisting of two layers, each with 32 hidden units and a dropout rate of 0.3.
   • Training: The model is pre-trained using the Adam optimizer (learning rate 0.001) on normalized demand data with a Mean Squared Error (MSE) loss function for 300 epochs.
   • TDF Extraction: Instead of using the final scalar output, the architecture extracts the activation vector from the penultimate dense layer (a 32-dimensional vector, $\phi_i$). This vector encapsulates the learned temporal patterns, seasonal dependencies, and demand dynamics.
   • Sequence Construction: Data is processed using sliding windows ($W=12$ for monthly, $W=7$ for daily) to create supervised input-output pairs.

2. Stage 2: Gradient Boosting Machine (GBM) Regressor

   • Feature Augmentation: The 32-dimensional temporal feature vector ($\phi_i$) is concatenated with exogenous categorical and temporal variables: Consumer Category (encoded as integers 0–4), Year, and Month Index. This creates an augmented 35-dimensional input vector ($z_i$).
   • Regression: A Gradient Boosting Machine (GBM) with 100 shallow decision trees (max depth 3) and a learning rate of 0.1 performs the final regression. The GBM corrects residuals and leverages the structured feature space to handle non-linearities that the LSTM alone might miss.

3. Data Generation and Validation

   • The study utilizes two datasets generated via MATLAB simulation to represent realistic smart grid demand for five consumer categories.
   • Daily Dataset: 2,040 records per day.
   • Monthly Dataset: 1,066 records per month.
   • The data is split chronologically (80/20) to preserve temporal causality.

Key Contributions

• Penultimate-Layer Transfer: The primary novelty is the extraction of the LSTM's penultimate-layer activation as a structured feature vector for transfer to a GBM. This differs from previous hybrids that use raw inputs or final scalar outputs.
• Multi-Granularity Pipeline: The framework successfully handles both daily and monthly forecasting within a single architectural paradigm without requiring distinct model tuning for each granularity.
• Category-Conditioned Forecasting: The model explicitly incorporates consumer category labels and temporal indices, addressing the heterogeneity of demand patterns across different sectors (e.g., agricultural vs. industrial).
• Long-Term Projection: The architecture supports stable, category-conditioned 5-year projections (2026–2031), directly applicable to infrastructure planning.

Results
The TDF-LSTGBM model was evaluated against five baselines: ARIMA, WMA, ETS, Vanilla LSTM, and a standard LSTM+XGBoost hybrid.

• Performance Metrics:
  • MAPE: The proposed model achieved a 3.64% MAPE for daily forecasting and 3.66% for monthly forecasting.
  • Comparison: This represents a 14.2× improvement over the best classical baseline (ARIMA, ~51.96% MAPE). It also outperformed Vanilla LSTM (7.68% MAPE) by a factor of 2.1× and the standard LSTM+XGBoost (6.12% MAPE) by 1.7×.
  • RMSE/MAE: The model achieved an RMSE of 0.0873 and an MAE of 0.0283 across both granularities, demonstrating cross-resolution stability.
• Visual Analysis: Forecast plots indicate that the model closely tracks actual demand, including peak events, without the progressive error accumulation seen in other models. The GBM stage effectively corrects peak underestimations common in pure LSTM models.
• Ablation: The study confirms that each component (the penultimate layer transfer, the specific GBM hyperparameters, and the feature augmentation) contributes uniquely to the final accuracy.

Significance and Claims
The paper claims that TDF-LSTGBM addresses critical gaps in current smart grid forecasting by systematically handling multi-granularity time series and consumer-category disaggregation. By transferring deep temporal features rather than scalar outputs, the model achieves superior precision in heterogeneous environments where classical linear models fail.

The authors position the model as a robust tool for infrastructure planning and demand response scheduling, evidenced by its ability to generate stable 5-year projections without drift. The study acknowledges limitations, including the use of simulated data, the absence of meteorological variables in the current feature set, and the scope being limited to a single country's context. Future work is proposed to integrate weather variables, incorporate self-attention mechanisms, and explore federated learning architectures for privacy-preserving forecasting.