💻 computer science

An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

This paper presents an empirical benchmark of nine deep learning models for smart meter energy forecasting, revealing that while deep learning outperforms classical baselines, lightweight architectures offer comparable accuracy at lower computational costs and that architectural differences only significantly impact performance on longer forecasting horizons and heterogeneous datasets.

Original authors: Behnaz Kavoosighafi, Maria Eidenskog, Wiktoria Glad, Katerina Vrotsou

Published 2026-08-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Behnaz Kavoosighafi, Maria Eidenskog, Wiktoria Glad, Katerina Vrotsou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Every modern power grid relies on a delicate balance between the electricity being generated and the electricity being used. If too much power is drawn at once, the system can become unstable; if too little is used, energy is wasted. To keep this balance, utility companies need to know exactly how much electricity homes will consume in the coming hours or days. For decades, they have used smart meters—digital devices that record energy use with high precision—to gather this data. These meters produce a constant stream of numbers, creating a detailed history of when a household turns on a heater, runs a dishwasher, or sleeps. The challenge for scientists is to turn these past numbers into reliable predictions for the future. While simple statistical rules work well for some patterns, the complex, shifting habits of real people often require more sophisticated tools. In recent years, researchers have turned to deep learning, a branch of artificial intelligence that can learn intricate patterns from vast amounts of data, to solve this forecasting puzzle.

A team of researchers at Linköping University in Sweden set out to test how well these advanced computer models actually perform when faced with real-world smart meter data. They did not just look at how accurate the predictions were; they also examined how much computing power each model required and whether some models worked better for certain types of households than others. The team gathered data from two distinct sources: one from thousands of homes in London and another from a smaller group of homes in the United States. They selected nine different deep learning models to test, ranging from relatively simple designs that rely on basic mathematical patterns to highly complex systems that use attention mechanisms to focus on specific parts of the data history. They also included older, traditional statistical methods as a baseline to see if the new, complicated tools were truly necessary.

The researchers trained these models to look at a window of past energy usage and predict what would happen next. They tested the models under various conditions, changing how much past history the models were allowed to see and how far into the future they had to predict. One of their key discoveries was that giving a model more history does not always make it smarter. While accuracy improved as the models were fed more past data, this benefit hit a ceiling. After a certain point, adding more historical information provided no extra value, suggesting that the models had already learned the essential patterns of the household's behavior. Conversely, as the researchers asked the models to predict further into the future, the accuracy naturally declined. This is a fundamental limit of forecasting: the further out you look, the more uncertainty creeps in, and the harder it becomes to be precise.

When comparing the different types of models, the study found that the most complex, computationally expensive designs did not always win. In many cases, lighter, simpler architectures achieved nearly the same level of accuracy as the heavyweights but required significantly less computing power to run. This is a crucial finding for utility companies, which may need to run thousands of forecasts simultaneously. The researchers also looked at whether the models treated different groups of people fairly. They analyzed performance across various demographic categories, such as income levels and housing types. While some complex models appeared to perform slightly better for the most under-represented or irregular groups, the difference was not strong enough to be considered a consistent rule. The data suggested that for most households, the choice of model mattered less than the specific conditions of the forecast.

Ultimately, the study concluded that there is no single "best" model for every situation. The ideal choice depends on the specific task at hand. If a utility company needs a quick forecast for the next day using a short history of data, a model based on patching together small segments of time might be the most efficient. If the goal is to look further ahead or if the data is highly variable, other designs like those using inverted attention or encoder-decoder structures might offer a better balance of speed and accuracy. The research provides a practical guide for engineers and planners, showing that while deep learning offers powerful new tools, the most effective solution is often the one that fits the specific constraints of the problem rather than the most complex tool available. By understanding these trade-offs, energy managers can build systems that are not only accurate but also efficient and fair for all types of households.

Technical Summary: An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

Problem Statement
Accurate forecasting of residential energy consumption is critical for the efficient operation of power systems, supporting demand response, infrastructure planning, and maintenance. While high-resolution data from smart meters has enabled the adoption of data-driven methods, the comparative performance of modern deep learning architectures on real-world smart meter data remains under-explored. Existing benchmarks often rely on general-purpose datasets or fail to jointly evaluate the interplay between model architecture, historical context length, prediction horizon, and computational efficiency. Furthermore, there is limited understanding of how these models perform across diverse geodemographic and household subgroups, despite the inherent heterogeneity of energy consumption patterns.

Methodology
The authors present an empirical benchmark evaluating nine modern deep learning forecasting models against classical statistical baselines (Naïve seasonal and SARIMA) on two distinct, high-resolution smart meter datasets:

Low-Carbon London (LCL): 5,561 households with half-hourly resolution (resampled to hourly for the benchmark), accompanied by ACORN geodemographic categories.
Pecan Street: 73 households in the US with 15-minute resolution, including appliance-level data aggregated to household load.

Models Evaluated:
The study covers a spectrum of architectural families:

Linear: DLinear.
MLP-based: LightTS, TSMixer, TiDE.
CNN-based: SCINet.
Memory-based: FiLM.
Transformer-based: Crossformer, PatchTST, iTransformer.

Experimental Protocol:

Setup: Models were trained using the Time Series Library (TSLib) under a consistent sliding-window protocol.
Variables: The study systematically varied the historical context length (input sequence length) and the prediction horizon (forecast length).
Metrics: Performance was assessed using standard error metrics (MSE, MAE, MAPE), variance explained ( $R^2$ ), correlation ( $r$ ), cosine similarity, and Fréchet distance (FD). Computational efficiency was measured via training and inference time.
Analysis: The authors conducted subgroup analyses across geodemographic categories (LCL) and building types (Pecan Street) to assess fairness and generalization. A composite metric ( $\mathcal{L}$ ) combining normalized MAE and training time was used to evaluate the accuracy-efficiency trade-off.

Key Results

Deep Learning vs. Classical Baselines: Deep learning models consistently outperformed classical baselines (Naïve seasonal and SARIMA) across primary accuracy metrics (MSE, MAE, $R^2$ ) on both datasets. The performance gap was particularly pronounced on the Pecan Street dataset, where classical baselines yielded negative $R^2$ values, indicating an inability to capture high household-level variability.
Historical Context Saturation: Extending the historical input context improves forecasting accuracy, but only up to a saturation point. Beyond this threshold, additional historical data provides diminishing returns.
Prediction Horizon Degradation: As the prediction horizon increases, forecasting accuracy consistently declines for all models.
Architecture vs. Efficiency:
- Lightweight Advantage: Lightweight architectures (e.g., MLP-based models like TSMixer and TiDE) achieved competitive accuracy at significantly lower computational costs compared to complex Transformer models.
- Context-Dependent Performance: No single model dominated all configurations.
  - On the LCL dataset, PatchTST and iTransformer performed well at shorter horizons and varying context lengths, while lighter models (TSMixer, TiDE) became more favorable at longer horizons.
  - On the Pecan Street dataset, TiDE and PatchTST generally dominated, with TiDE's encoder-decoder design proving robust to the dataset's higher variability and finer temporal resolution.
Subgroup Fairness: A subgroup analysis revealed that model choice has limited impact on predictive performance for most population segments. While Transformer-based models appeared to offer an advantage for the most under-represented or behaviorally irregular groups (e.g., specific ACORN categories), this pattern was not statistically robust due to small subgroup sizes and did not generalize consistently across datasets.

Significance and Claims
The paper claims to provide practical, evidence-based guidance for selecting forecasting models in operational smart meter energy applications. Its primary contributions are:

Establishing a comprehensive benchmark of nine diverse deep learning architectures on real-world, high-resolution energy data.
Demonstrating that architectural differences become significant primarily at longer forecasting horizons and on more heterogeneous datasets.
Highlighting that lightweight models offer a superior accuracy-efficiency trade-off for many scenarios, challenging the assumption that increasingly complex architectures are always necessary for structured signals like energy consumption.
Offering a nuanced view on fairness, noting that while attention-based models may show promise for under-represented groups, current evidence does not support a statistically robust generalization of this advantage.

The study concludes that model selection should be driven by the specific constraints of the application (horizon length, data heterogeneity, and computational budget) rather than a default preference for the most complex architectures.

Technical Summary: An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

More like this