Artificial Lemming Algorithm-Optimized CrossKAN for Multi-Energy Load Forecasting

Technical Summary: Artificial Lemming Algorithm-Optimized CrossKAN for Multi-Energy Load Forecasting

Problem Statement
Accurate short-term forecasting of multi-energy loads (electricity, heating, and cooling) is critical for the operation of integrated energy systems, particularly as renewable generation increases. These loads exhibit complex nonlinear, time-varying, and cross-variable dependencies driven by weather, operating conditions, and user behavior. Existing approaches face specific limitations:

• Statistical and traditional ML methods (e.g., ARIMA, SVM) often rely on linearity or stationarity assumptions that fail to capture coupled load dynamics.
• Deep learning models (e.g., RNNs, Transformers) can model long-range dependencies but often incur high computational and memory costs, restricting their deployment in scheduling and demand response scenarios.
• Lightweight architectures (e.g., CrossLinear) offer efficiency but typically employ linear forecasting heads, which may be insufficient for capturing nonlinear phenomena such as peak-valley transitions and threshold effects.
• Feature selection remains challenging, as single association measures often fail to simultaneously characterize nonlinear statistical dependence, model-specific predictive contribution, and temporal stability.

Methodology
The authors propose a unified framework integrating three distinct components: Triple-Correlation Analysis (TCA) for feature selection, CrossKAN for forecasting, and the Artificial Lemming Algorithm (ALA) for hyperparameter optimization.

1. Triple-Correlation Analysis (TCA) for Input Selection:
   To address the limitations of single-metric feature selection, TCA ranks candidate exogenous variables using a composite score derived from three perspectives:

   • Nonlinear Statistical Dependence: Measured via the Maximal Information Coefficient (MIC).
   • Model-Derived Predictive Contribution: Quantified using SHapley Additive exPlanations (SHAP) values from a Random Forest model.
   • Temporal Stability: Evaluated using the mean absolute correlation coefficient within a sliding window.
     These normalized scores are weighted (MIC: 0.4, SHAP: 0.4, TVC: 0.2) to produce a unified ranking for input selection.

2. CrossKAN Architecture:
   CrossKAN is a lightweight forecasting model that modifies the CrossLinear architecture.

   • Encoder: It retains the lightweight cross-correlation embedding (to model interactions between endogenous loads and exogenous variables) and patch-embedding (to encode local temporal variations) of CrossLinear.
   • Forecasting Head: The fixed linear projection of CrossLinear is replaced with a Kolmogorov–Arnold Network (KAN) layer. This head utilizes learnable one-dimensional spline functions to map encoded temporal features to future loads, concentrating nonlinear modeling capacity at the output without introducing a heavy self-attention backbone.

3. Artificial Lemming Algorithm (ALA) Optimization:
   ALA, a population-based metaheuristic inspired by lemming migration and foraging behaviors, is employed as an external optimizer. It searches for optimal configurations of three key hyperparameters:

   • Hidden-layer width (controlling nonlinear capacity).
   • Embedding dimension (controlling temporal representation compression).
   • Learning rate (governing training convergence).
     The algorithm utilizes strategies including long-distance migration (global exploration), burrowing (local exploration), foraging (spiral search), and predator avoidance (Lévy flight escape) to minimize the Mean Squared Error (MSE) on the training set.

Key Contributions
The paper outlines four primary contributions:

1. CrossKAN Design: A model that retains the efficient cross-correlation and patch-embedding backbone of CrossLinear but replaces the linear head with a KAN-based nonlinear mapping, enabling the learning of complex nonlinear dependencies without a self-attention mechanism.
2. Triple-Correlation Analysis (TCA): A novel feature selection framework that integrates MIC, SHAP importance, and sliding-window correlation to rank inputs based on statistical dependence, predictive contribution, and temporal stability.
3. ALA Integration: The application of the Artificial Lemming Algorithm to jointly optimize learning rate, hidden-layer width, and embedding dimension, operating externally to the forecasting network.
4. Comprehensive Evaluation: A rigorous evaluation across electricity, heating, and cooling loads on two datasets (a primary integrated-energy dataset and Building Data Genome 2), comparing the framework against multiple baselines and optimization algorithms.

Results
The framework was evaluated on two datasets: a primary dataset from Arizona State University (2023) and Building Data Genome 2 (BDG2, 2016–2017).

• Feature Selection Impact: On the primary dataset, TCA-based feature screening reduced forecasting errors significantly. For electricity loads, MAE, RMSE, and MAPE decreased by 15.43%, 18.07%, and 15.30%, respectively, compared to using all candidate variables.
• Optimizer Performance: ALA outperformed other optimization algorithms (SSA, RIME, WOA, PSO, CPO) in selecting hyperparameters for CrossKAN, achieving the lowest MAE (3.524), RMSE (5.145), and MAPE (2.513%) on electricity load forecasting.
• Forecasting Accuracy (Primary Dataset): CrossKAN achieved the lowest error metrics across all three load types compared to the strongest baselines (including FEDformer, iTransformer, DLinear, and CrossLinear). Relative to the strongest baseline for each metric, reductions ranged from 13.2% to 17.6% (e.g., electricity MAE reduced by 14.90%, heating MAPE by 17.58%).
• Cross-Dataset Evaluation (BDG2): On the BDG2 dataset, CrossKAN achieved the lowest MAE and RMSE for electricity, heating, and cooling loads among all tested models, demonstrating robustness across different data sources, though the authors note the evaluation did not use leave-site-out splits.

Significance and Claims
The paper claims that concentrating nonlinear modeling in the forecasting head while preserving a lightweight encoder is an effective strategy for multi-energy load forecasting. The results suggest that this approach improves accuracy while maintaining computational efficiency suitable for scheduling and demand response applications.

The authors maintain a modest scope regarding their findings:

• The improvements are validated within the specific datasets and evaluation settings used.
• The BDG2 evaluation demonstrates cross-dataset robustness but does not establish generalization to unseen buildings or regions due to the lack of leave-site-out validation.
• The study focuses on point-forecast errors and does not quantify predictive uncertainty or performance under extreme-load conditions.
• Future work is identified as necessary for independent-site validation, probabilistic forecasting, and sensitivity analysis of the feature-fusion weights and search ranges.