Technical Summary: Evaluating Time Series Foundation Models for Electricity Price Forecasting

Problem Statement

Short-term electricity price forecasting (EPF) is a critical component of modern power system operations but remains highly challenging due to strong nonstationarity, multi-scale variability (seasonality, spikes, extreme events), and a heavy reliance on complex structural and contextual system dynamics. While Time Series Foundation Models (TSFMs) have demonstrated strong zero-shot forecasting capabilities across various domains, their generalization in covariate-driven, non-stationary settings like EPF is underexplored.

Existing benchmarks for TSFMs often face four specific limitations when applied to EPF:

1. Contamination Risk: Many TSFMs are pretrained on energy-related data, making it difficult to determine if benchmark datasets overlap with pretraining corpora, potentially inflating performance estimates.
2. Nonstationarity: Distributional shifts and price spikes can undermine model generalization.
3. Tail Behavior: Accurate modeling of extreme events (tails) is critical for risk-aware decision-making in power systems but is often overlooked in average accuracy metrics.
4. Covariate Dependence: Historical prices alone are often insufficient; accurate EPF relies heavily on exogenous covariates (load, fuel prices, weather, generation), yet many evaluations fail to adequately test covariate support.

Methodology

Benchmarking Framework

The authors propose a two-dataset benchmarking framework designed to mitigate contamination risk and enable fair evaluation:

1. GEFCom2014-P: A standardized, widely used probabilistic EPF benchmark from the 2014 Global Energy Forecasting Competition.
2. GridStatus2025: A newly curated dataset derived from the GridStatus API (2022–2025). This dataset was selected specifically because its evaluation period (2025) post-dates the publicly disclosed pretraining data for major TSFMs (Chronos 2, TimesFM 2.5) or relies on synthetic data (TabPFN-TS), thereby minimizing temporal overlap and contamination.

Models Evaluated

The study benchmarks four TSFMs, evaluating both covariate-free and covariate-supported variants:

• Chronos 2
• TimesFM 2.5
• TabPFN-TS
• TOTO 1.0

These are compared against three classes of baselines:

1. Statistical Models: Seasonal Naïve, Auto-ARIMA, Auto-ETS, Auto-Theta.
2. Task-Trained Deep Learning: DeepAR, TFT, TiDE, TSMixer, DLinear, NLinear.
3. Domain-Specific EPF Methods: LEAR, DNN, and CING-LEAR (specifically designed to leverage electricity market information).

Evaluation Protocols

• Metrics: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Average Quantile Loss (aQL) across nine quantiles.
• Analysis Dimensions:
  • Point and Probabilistic Forecasting: Assessing overall accuracy and distributional calibration.
  • Tail Behavior: Analyzing performance at extreme quantiles (e.g., Q(0.01), Q(0.99)) to evaluate robustness to price spikes and negative prices.
  • Distributional Shifts: Dedicated analysis of model performance during price spike events (defined as observations outside the 5th and 95th percentiles).
  • Statistical Significance: Diebold-Mariano (DM) tests are used to verify if covariate support leads to statistically significant improvements.

Key Results

1. Competitive Zero-Shot Performance with Covariate Dependence

• On GEFCom2014-P, TSFMs achieved competitive zero-shot performance relative to many competition submissions but did not reach the top-5, with the best TSFM (TOTO 1.0) ranking eighth. This suggests that while pretraining offers general capability, domain-specific design remains advantageous.
• On GridStatus2025, TSFMs generally outperformed statistical and general-purpose deep learning baselines. Crucially, covariate-supported variants (e.g., Chronos 2 w., TabPFN-TS w.) consistently ranked among the strongest models.
• Covariate-free variants performed substantially worse, confirming that exogenous information is critical for EPF. Diebold-Mariano tests statistically confirmed that covariate usage significantly reduces forecast loss for Chronos 2 and TabPFN-TS.

2. Domain-Specific Methods Remain Highly Competitive

• CING-LEAR, a domain-specific method, achieved the second-best individual performance on GridStatus2025, outperforming most TSFMs and general-purpose models.
• This indicates that for EPF, task-specific inductive biases and feature engineering tailored to electricity pricing signals remain highly effective.

3. The Power of Ensembles

• A simple ensemble, constructed by averaging the forecasts of the best-performing TSFM (Chronos 2 w.) and the best domain-specific method (CING-LEAR), achieved the strongest overall performance across all metrics (MAE, RMSE, aQL).
• This suggests that TSFMs and domain-aware methods capture complementary predictive information, and combining them yields significant gains over either approach alone.

4. Tail Behavior and Robustness

• Tail Performance: Statistical models showed significant degradation in lower-tail regions (e.g., negative prices), whereas TSFMs produced more balanced tail forecasts. Domain-specific methods (CING-LEAR) were highly competitive across all tail quantiles.
• Price Spikes: During spike events, models with covariate support (Chronos 2 w., CING-LEAR) and their ensemble demonstrated greater robustness compared to models without covariates.
• Asymmetry: Statistical methods tended to perform better in upper-tail (high prices) but struggled with abrupt downward movements, whereas TSFMs offered more balanced distributional robustness.

5. Contamination Sensitivity

• TOTO 1.0 exhibited strong performance on GEFCom2014-P but degraded substantially on GridStatus2025. The authors attribute this to potential differences in market conditions and data characteristics, highlighting the necessity of contamination-conscious evaluation protocols.

Significance and Claims

The paper claims that while TSFMs are highly promising for EPF, their success is critically dependent on covariate support and they do not consistently surpass methods specifically tailored to the domain. The study emphasizes that:

1. Contamination-aware evaluation is essential for structured domains like EPF to avoid inflated performance estimates.
2. Domain-informed design remains vital for complex systems with strong structural dependencies.
3. Ensembles combining the broad generalization of TSFMs with domain-aware models represent a fruitful path for achieving state-of-the-art predictive skill.

The authors conclude that future work should focus on fine-tuning TSFMs for EPF-specific objectives (such as tail accuracy) and developing stronger ensembles that leverage the strengths of both foundation models and domain-specific architectures.