🤖 machine learning

Beyond Weather Correlation: A Comparative Study of Static and Temporal Neural Architectures for Fine-Grained Residential Energy Consumption Forecasting in Melbourne, Australia

This study demonstrates that for fine-grained (5-minute) residential energy forecasting in Melbourne, temporal autocorrelation captured by LSTM networks significantly outperforms static weather-driven MLP models, particularly highlighting the dominance of consumption history over meteorological data even in solar-integrated households.

Original authors: Prasad Nimantha Madusanka Ukwatta Hewage, Hao Wu

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Prasad Nimantha Madusanka Ukwatta Hewage, Hao Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess what a family is doing in their house right now, and you want to predict exactly how much electricity they will use in the next 5 minutes.

This paper is like a scientific showdown between two different "guessing machines" (AI models) to see which one is better at this job. The researchers set up a race in Melbourne, Australia, using real data from two houses: one that just uses power from the grid, and another that also has solar panels on the roof.

Here is the story of what they found, explained simply.

The Two Contenders

The "Weather Watcher" (MLP):
Think of this model as a meteorologist who only looks at the daily weather report. It knows the high temperature, the humidity, and that it's 2:00 PM. It tries to guess the electricity usage based only on these facts. It's like trying to guess what a person is eating for dinner just by knowing the season and the time of day, without knowing if they are hungry or what's in the fridge.
The "Memory Keeper" (LSTM):
Think of this model as a super-observant neighbor who has been watching the house for the last two hours. It doesn't care about the weather outside; it only cares about what the house just did. If the lights turned on at 6:00 PM and the oven started at 6:15 PM, this model uses that immediate history to guess what happens next. It's like predicting the next move in a game of chess by looking at the last few moves, rather than the weather outside the stadium.

The Big Surprise

The researchers ran a test to see who could predict the electricity usage for the next 5 minutes more accurately.

The Weather Watcher (MLP) failed miserably for the standard house. In fact, it was so bad that it was worse than just guessing the average amount of electricity used all day.
- Why? Because daily weather is too slow. Knowing it's 30°C (86°F) all day doesn't tell you if the family is currently watching TV (low power) or running the air conditioner and the oven (high power). The "stochastic" (random) nature of human life—like deciding to boil a kettle or turn on a hair dryer—happens too fast for daily weather data to catch.
The Memory Keeper (LSTM) won easily. It got it right about 88% of the time.
- Why? Because human behavior has a rhythm. If you just turned on the kettle, you are likely to turn on the toaster next. The model learned the "flow" of the house's energy usage. It realized that the past few minutes are the best predictor of the next few minutes.

The Solar Twist (The Plot Twist)

There was a second house with solar panels. Here, the "Weather Watcher" did a bit better (scoring a 41% success rate).

Why?
Think of solar panels as a predictable machine. When the sun is high and it's hot, the panels always produce power. The "Weather Watcher" was actually good at guessing how much power the solar panels would make, but it was still bad at guessing what the people in the house were doing.

The Analogy: If you are trying to guess the total weight of a truck, and you know the truck is carrying a heavy, predictable load of bricks (solar), you can guess the weight fairly well. But if you don't know if the driver is also carrying a bag of feathers (the family's random appliance use), your guess will still be off. The model was mostly guessing the "bricks," not the "feathers."

The Takeaway for the Real World

This study teaches us a valuable lesson for the future of our power grids:

Don't rely on the weather forecast for minute-by-minute power needs. If you want to manage electricity for a specific house every 5 minutes, looking at the daily weather report is useless.
History is the best teacher. To manage power efficiently (like turning on a battery or an air conditioner at the perfect time), you need to look at what the house just did.
The "Smart Grid" needs memory. For Australia's new 5-minute electricity pricing rules to work, the systems need to be like the "Memory Keeper"—they need to learn the habits of the house, not just watch the sky.

In a Nutshell

If you want to know what a house will do in the next 5 minutes, don't ask the weatherman; ask the house what it did in the last 2 hours. The "Memory Keeper" AI proved that remembering the immediate past is far more powerful than knowing the daily forecast.

1. Problem Statement

The paper addresses the challenge of Short-Term Load Forecasting (STLF) at the residential level with 5-minute temporal resolution. While weather variables (temperature, humidity, etc.) are widely accepted as key drivers of electricity demand, their relative utility compared to temporal autocorrelation (the sequential memory of past consumption) remains underexplored at fine granularities.

The core research question is: At a 5-minute forecasting resolution, is the current day's weather information or the immediately preceding consumption history more informative for predicting future load?

This is critical for:

Smart grid management and demand response programs.
Integration of Distributed Energy Resources (DERs), specifically rooftop solar PV.
Compliance with Australia's National Electricity Market (NEM) 5-minute settlement rules.

2. Methodology

Dataset

Source: Real-world smart meter data from two households in Melbourne, Victoria, Australia, merged with official Bureau of Meteorology (BOM) daily weather observations.
Timeframe: March 2023 – April 2024 (14 months).
Samples: ~117,000 to ~119,000 records per household (5-minute intervals).
Households:
- House 3: Grid-only dwelling (standard consumption).
- House 4: Rooftop solar PV-integrated dwelling (net consumption includes negative values during export).
Features:
- MLP Input: 6 daily weather variables (Max Temp, Rainfall, 9 am/3 pm Temp & Humidity) + Time-of-day decimal.
- LSTM Input: Univariate consumption sequence only (24-step sliding window = 2 hours).

Models Compared

Multilayer Perceptron (MLP): A static feedforward neural network. It maps the 7-dimensional static feature vector (weather + time) to a single scalar prediction. It has no access to historical consumption values.
- Architecture: Input(7) → Dense(64) → Dropout → Dense(32) → Dropout → Dense(1).
Long Short-Term Memory (LSTM): A recurrent neural network designed to capture temporal dependencies. It processes a 2-hour (24-step) sliding window of past consumption to predict the next step.
- Architecture: Input(24, 1) → LSTM(50) → Dropout → Dense(1).
- Note: The LSTM was intentionally run without weather features to isolate the predictive power of temporal autocorrelation.

Evaluation Metrics

$R^2$ (Coefficient of Determination): Primary metric for cross-model comparison. $R^2 < 0$ indicates performance worse than a trivial mean predictor.
RMSE & MAE: Error metrics in physical units (Watts).
Baselines:
- Naïve Persistence: Predicts the current value for the next step ( $\hat{y}_{t+1} = y_t$ ).
- Seasonal Naïve: Predicts the value from the same time slot the previous day.

3. Key Contributions

First Empirical Comparison: Provides the first head-to-head comparison of MLP (weather-driven) vs. LSTM (sequence-driven) for 5-minute residential forecasting in the Southern Hemisphere using official BOM data.
Quantitative Signal Decomposition: Demonstrates that temporal autocorrelation dominates weather features at 5-minute granularity.
Solar PV Asymmetry Discovery: Identifies that weather-based models perform significantly better on solar-integrated homes, not because they predict load better, but because they implicitly predict solar generation (which is weather-dependent).
Benchmarking: Establishes persistence baselines and seasonal stratification metrics for the Australian context, filling a gap in regional STLF literature.
Reproducibility: Offers a reproducible protocol merging public BOM data with real smart meter data.

4. Key Results

Performance Comparison (Test Set $R^2$ )

Model	House 3 (Grid Only)	House 4 (Solar PV)
MLP (Weather + Time)	-0.055 (Worse than mean)	0.410
LSTM (History Only)	0.883	0.865
Naïve Persistence	0.878	0.851

House 3 (Grid Only): The MLP failed completely ( $R^2 = -0.055$ ), proving that daily weather features cannot resolve intra-day stochasticity. The LSTM achieved $R^2 = 0.883$ , a 93.3 percentage point improvement over the MLP.
House 4 (Solar PV): The MLP improved to $R^2 = 0.410$ , but the LSTM still outperformed it by 45.5 percentage points ( $R^2 = 0.865$ ).

Visual & Analytical Findings

MLP Behavior: Predictions were essentially flat (following the daily mean), failing to capture morning/evening peaks or transient appliance spikes.
LSTM Behavior: Closely tracked actual load profiles, capturing diurnal patterns and peaks.
Solar Effect: The MLP's success on House 4 is attributed to the deterministic nature of solar generation (which correlates with weather/time), not the stochastic household load. The model was effectively forecasting solar output, not consumption.

5. Significance and Implications

Scientific Insight

The study conclusively proves that at sub-hourly (5-minute) resolutions, individual household behavior (stochastic appliance usage, occupancy shifts) creates variance that daily weather data cannot explain. Temporal autocorrelation is the dominant signal. Weather features only become useful when the target variable includes weather-dependent components (like solar generation) or at coarser time scales (e.g., hourly/daily).

Practical Applications

Smart Grid Design: For 5-minute settlement and demand response, relying on weather data alone is insufficient. Systems must utilize historical consumption sequences (via LSTM or similar temporal models).
Solar Integration: The "solar asymmetry" highlights a methodological pitfall: forecasting "net load" on solar homes using weather data conflates load prediction with solar generation prediction. Future models should disaggregate these components.
Future Roadmap: The authors propose a PhD-level research path including:
- Hybrid models (Weather-Augmented LSTM).
- Transformer-based architectures (PatchTST, TFT).
- Federated Learning for privacy-preserving multi-household training.
- Disaggregation of grid draw vs. solar generation.

Conclusion

The paper establishes that historical consumption data is necessary and sufficient for high-resolution residential forecasting, rendering static weather features largely ineffective for predicting 5-minute load fluctuations in grid-only homes. While weather aids in forecasting solar generation, it does not replace the need for temporal sequence modeling in smart grid applications.