Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting

1. Problem Statement

Short-term load forecasting (STLF) is critical for electricity grid operations, including unit commitment and economic dispatch. While deep learning architectures have evolved from LSTMs to Transformers and State Space Models (SSMs), there is a lack of systematic guidance for grid operators on which architecture to deploy. Existing studies are often limited to:

• Narrow Scope: Evaluations on only 1–5 grids, failing to capture the diversity of US Independent System Operators (ISOs).
• Inconsistent Protocols: Comparisons often use different preprocessing, hyperparameters, or data splits, making cross-architecture conclusions unreliable.
• Input Dependency: It remains unclear how architectural performance changes when moving from load-only inputs to weather-augmented inputs, despite weather being a primary driver of demand.
• Task Generalization: It is unknown if rankings hold across different signal types (e.g., solar, wind, prices) beyond aggregate load.

2. Methodology

The authors conducted a comprehensive, controlled benchmark across six major US ISOs (CAISO, ISO-NE, MISO, PJM, ERCOT, NYISO) and one additional grid (SWPP) for weather experiments.

A. Data and Setup

• Dataset: EIA-930 hourly load data (2022–2025).
• Forecast Windows: 24, 48, 72, 96, and 168 hours.
• Context Length: Fixed at $L=240$ hours (10 days).
• Metrics: Normalized MSE (%), MAPE (%), and signed-error tails (P0.5/P99.5).
• Conditions:
  1. Load-Only: Historical load + temporal embeddings.
  2. Weather-Augmented: Load + temporal + meteorological covariates (temperature, humidity, wind, GHI, cloud cover) with thermal-lag alignment.

B. Evaluated Architectures

Five models from three families were adapted with domain-specific components (temporal embeddings, bidirectional encoding, and modular weather fusion):

1. State Space Models (SSMs):
   • S-Mamba: A minimalist encoder-decoder using selective state space mechanisms ($O(n)$ complexity).
   • PowerMamba: Introduces series decomposition (Trend/Seasonal) and bidirectional processing to handle disparate frequencies (e.g., duck curve).
2. Transformers:
   • PatchTST: Uses channel-independent patching. It treats each variate independently, sharing weights across channels to extract temporal patterns without explicit cross-variable attention.
   • iTransformer: Uses cross-variate tokenization. It embeds entire time series variables as tokens, allowing self-attention to model correlations between variables (e.g., Load vs. Temperature).
3. Recurrent Networks:
   • LSTM: A 2-layer bidirectional LSTM serving as the recurrent baseline.

C. Weather Integration Strategy

Crucially, the authors designed architecture-matched fusion strategies to ensure fair comparison:

• SSMs/LSTM: Early summation or concatenation of weather tokens before the core processing.
• PatchTST: Interleaved cross-attention where load patches query weather patches (preserving channel independence).
• iTransformer: Weather variables are treated as additional variate tokens, allowing global cross-variate attention.

3. Key Contributions

1. First Large-Scale US Grid Benchmark: A systematic comparison of SSMs, Transformers, and RNNs across six diverse US operators under identical protocols.
2. Architecture-Weather Interaction Analysis: Demonstrated that the benefit of weather data is architecturally determined, not just a function of model capacity.
3. Capacity-Controlled Experiments: Proved that the superior performance of certain models with weather data is due to architectural design (e.g., cross-variate attention) rather than simply having more parameters.
4. Multitype Generalization: Extended the benchmark to solar, wind, wholesale prices, and ancillary services, revealing that optimal architecture depends on the signal structure (periodicity vs. chaos).

4. Key Results

A. Load-Only Forecasting (No Weather)

• Winner: PatchTST was the overall leader, winning 15 out of 30 evaluations (ISO $\times$ Window).
• Runner-up: SSMs (PowerMamba/S-Mamba) were highly competitive, particularly on ISO-NE, PJM, and NYISO.
• Loser: iTransformer performed poorly (2nd highest error) because its cross-variate attention mechanism degenerates to near-identity when only a single variate (load) is present.
• LSTM: Performed the worst, degrading significantly at longer horizons.

B. Weather-Augmented Forecasting

• Ranking Reversal: When weather data was added, iTransformer improved its accuracy 3x more efficiently than PatchTST.
  • iTransformer average $\Delta$MAPE improvement: -1.62%
  • PatchTST average $\Delta$MAPE improvement: -0.52%
• Reasoning: iTransformer's ability to explicitly model correlations between load and weather tokens allows it to leverage exogenous data effectively. PatchTST's channel-independent design limits its ability to mix information across variables.
• SSM Performance: With weather, SSMs (specifically S-Mamba) became the strongest overall models on several grids (SWPP, ERCOT, NYISO), combining high accuracy with $O(n)$ inference efficiency.

C. Capacity-Controlled Findings

The authors scaled PatchTST to match iTransformer's parameter count and vice versa.

• Result: The weather advantage of iTransformer persisted even when shrunk to fewer parameters, while scaling PatchTST up did not improve its weather utilization. This confirms the advantage is architectural, not capacity-driven.

D. Multitype Generalization (Solar, Wind, Price)

• PatchTST dominates signals with strong diurnal periodicity (Solar, Ancillary Services) because its fixed-length patches align with 24-hour cycles.
• SSMs dominate signals with chaotic/aperiodic fluctuations (Wind, Wholesale Prices) due to their adaptive state dynamics.
• iTransformer remains weak on single-variate tasks but excels on multi-variate tasks (like price) where cross-variable correlation is key.

E. Grid Characteristics & Weather Value

• Ceiling Effect: Weather integration yields the highest value on grids with high baseline forecast difficulty (e.g., ERCOT, ISO-NE) and high climate variability.
• Low Value: On grids with low baseline error (e.g., MISO, PJM) or mild climates (CAISO), adding weather provides diminishing returns because temporal patterns already capture most variance.

5. Significance and Practical Implications

This paper provides actionable guidelines for grid operators:

1. Input-Dependent Selection:
   • Load-Only: Deploy PatchTST or SSMs.
   • Weather-Available: Deploy iTransformer or Weather-Aware SSMs (S-Mamba/PowerMamba).
2. Signal-Specific Tuning: Do not use a single model for all tasks. Use PatchTST for solar/periodic loads and SSMs for wind/price forecasting.
3. Investment Prioritization: Prioritize weather integration investments for isolated or smaller grids (e.g., ERCOT, SWPP) where baseline errors are high, rather than large, geographically diverse interconnections where spatial smoothing already reduces weather impact.
4. Efficiency: SSMs offer a compelling trade-off, delivering top-tier accuracy with linear inference complexity, making them ideal for real-time edge deployment.

The study concludes that there is no "one-size-fits-all" model; the optimal architecture is strictly determined by the availability of exogenous data and the intrinsic periodicity of the target signal.