A Deep Learning Framework for Heat Demand Forecasting using Time-Frequency Representations of Decomposed Features

This paper proposes a novel deep learning framework that utilizes Continuous Wavelet Transform to generate time-frequency representations of decomposed features, enabling Convolutional Neural Networks to achieve significantly higher accuracy (reducing error by 36–43%) in day-ahead district heating demand forecasting compared to state-of-the-art baselines across multiple European datasets.

The Gist

Imagine a city's heating system as a giant, complex kitchen. The chefs (the heating plant) need to know exactly how much soup (heat) to cook every hour of the next day. If they cook too little, people freeze; if they cook too much, they waste expensive fuel and energy.

For a long time, chefs tried to guess the future by looking at yesterday's soup pot. They'd say, "It was cold yesterday, so it will probably be cold today." But this is like trying to predict the weather by only looking out the window for five minutes. It misses the bigger picture: the changing seasons, the sudden storms, and the fact that people behave differently on holidays.

This paper introduces a super-smart "Time-Traveling Chef" that uses a new kind of magic to predict heat demand with incredible accuracy. Here is how it works, broken down into simple concepts:

1. The Problem: The "Noisy" Signal

Heat demand is messy. It goes up and down like a rollercoaster. It has:

• The Trend: The slow, steady change as winter gets colder or warmer.
• The Rhythm: The daily beat (people wake up, go to work, come home, sleep) and the weekly beat (weekends are different).
• The Noise: Random spikes caused by a broken sensor, a sudden snowstorm, or a holiday.

Old computer models tried to read this messy rollercoaster line directly. They often got confused by the noise and missed the big patterns.

2. The Secret Sauce: The "Musical Score" (Time-Frequency)

The authors realized that looking at the raw data is like trying to understand a symphony by staring at a single vibrating string. Instead, they used a tool called the Continuous Wavelet Transform (CWT).

The Analogy: Imagine you are listening to a song.

• Old Method: You just hear the volume going up and down over time.
• New Method: They turn the sound into a musical score (a sheet of music).
  • The horizontal axis is time.
  • The vertical axis is the pitch (frequency).
  • This allows the computer to see exactly when a high note (a sudden spike) happens and how long a low note (a slow trend) lasts.

By turning the data into this "image" or "score," the computer can use a Convolutional Neural Network (CNN)—the same type of AI that recognizes cats in photos—to "read" the patterns in the heat demand like it's reading a picture.

3. Breaking It Down: The "Deconstructed Soup"

Before turning the data into a musical score, the AI first decomposes the data. Think of it like taking a complex stew and separating it into its ingredients:

• The Broth (Trend): The long-term temperature changes.
• The Herbs (Seasonality): The daily and weekly rhythms.
• The Chunks (Residuals): The random, weird bits.

The AI learns that the "Broth" is mostly driven by the weather (how cold it is outside), while the "Herbs" are driven by human habits (work vs. weekend). By studying these ingredients separately, the AI understands the recipe much better than if it just tasted the whole stew at once.

4. The Results: A Crystal Ball

The researchers tested this new "Time-Traveling Chef" against:

• Old Statisticians: (Like SARIMAX) who use simple math formulas.
• Modern AI: (Like Transformers and LSTMs) which are very popular right now.
• Foundation Models: Giant AI models trained on massive amounts of data.

The Outcome:
The new method was the clear winner. It reduced the prediction error by 36% to 43% compared to the best existing methods.

• Why it matters: It's not just about being "right." It's about being right when it counts. The new model is great at predicting peaks (when everyone turns on their heaters at 6 PM) and holidays (when the city goes quiet).
• Real-world impact: Because the prediction is so accurate, the heating plant can turn down the temperature slightly without anyone freezing. This saves massive amounts of fuel and cuts carbon emissions.

5. The "Holiday" Twist

One tricky part of heating is holidays. On Christmas, people don't follow their normal schedule.

• The AI learned that simply knowing "It's Christmas" isn't enough.
• It needed to know: "Last year, on Christmas, people used this much heat."
• By feeding the AI specific data from previous holidays, it learned to handle these rare events much better than the other models, which usually just guessed based on a normal Tuesday.

Summary

This paper is about teaching a computer to listen to the music of the city rather than just counting the notes. By turning messy heat data into a clear "musical score" and separating the ingredients of the data, they built an AI that can predict exactly how much heat a city needs for the next 24 hours.

The Bottom Line: It's a smarter, more efficient way to keep us warm while wasting less energy and money. It's like upgrading from a guess-and-check thermostat to a crystal ball that knows exactly what the weather and the neighbors will do tomorrow.

Technical Summary

1. Problem Statement

District Heating Systems (DHS) are critical for sustainable energy distribution, yet their efficient operation relies on accurately balancing supply with fluctuating demand. Current challenges include:

• Complexity: Heat demand exhibits non-linear, non-stationary patterns influenced by consumer behavior, weather, and calendar events.
• Limitations of Existing Models: Traditional statistical methods (e.g., SARIMAX) and standard Deep Learning (DL) architectures (e.g., LSTMs, Transformers) often struggle to capture high-frequency volatility, long-range dependencies, and the specific coupling between exogenous factors (weather) and demand.
• Evaluation Gaps: Many existing studies rely on short-term datasets (weeks or months) or homogeneous user groups, failing to demonstrate robustness across diverse climates, seasons, and heterogeneous consumer profiles over multi-year horizons.
• Feature Engineering: There is a lack of systematic understanding regarding how decomposed time-series components (trend, seasonality, residuals) interact with weather variables to drive demand.

2. Methodology

The authors propose a novel framework that transforms the forecasting problem from a 1D time-domain task into a 2D time-frequency image recognition task.

A. Data Preprocessing & Feature Engineering

• Data Sources: The framework was trained and validated on multi-year hourly data from three Danish District Metered Areas (DMAs), plus public datasets from Flensburg (Germany) and Aalborg (Denmark).
• Outlier Handling: A Savitzky-Golay smoothing approach was applied to studentized residuals to remove physically implausible values without distorting the underlying trend.
• Time-Series Decomposition: Input features (consumption and weather) were decomposed into Trend, Seasonal, and Residual components using additive decomposition.
  • Key Insight: Decomposition revealed that while raw temperature correlates strongly with demand, the Trend component of temperature drives the long-term demand trend, while the Residual captures short-term volatility.
• Feature Selection:
  • Endogenous: Lagged consumption at 24 hours ($c_{24}$) and 168 hours ($c_{168}$). The 168-hour lag was crucial for stabilizing forecasts during pattern transitions (e.g., weekends/holidays).
  • Exogenous: Ambient temperature ($t_{amb}$) was identified as the most impactful single feature. Other temperature metrics (min, max, feels-like) were added in decomposed forms.
  • Calendrical: Explicit cyclical time features (sine-cosine encoding) were found to degrade performance due to redundancy with the time-frequency representation; however, specific holiday embeddings (categorical and lagged) were used to handle rare events.

B. Time-Frequency Transformation (CWT)

Instead of feeding raw time series directly into the network, the authors apply the Continuous Wavelet Transform (CWT) to the decomposed feature vectors.

• Scalograms: Each feature is converted into a 2D "scalogram" (a time-frequency image) where rows represent frequency scales and columns represent time steps.
• Input Tensor: These scalograms are concatenated to form a 3D input tensor ($M \times N_{history} \times N_{features}$), effectively treating the time-series data as a multi-channel image.

C. Model Architecture: Wavelet-CNN

The core model is a Convolutional Neural Network (CNN) designed to process these image-like scalograms:

• Convolutional Blocks: Three stacked 2D convolutional layers (32, 64, and 128 filters) extract hierarchical temporal-periodic patterns.
• No Pooling: Unlike standard CNNs, pooling layers are excluded to preserve high-fidelity time-frequency localization and phase coherence, which is critical for predicting the exact timing of daily peaks.
• Linear Head: The flattened feature maps pass through two dense layers (1024 neurons each) with Leaky ReLU activation and Dropout (0.1) to prevent overfitting, followed by a final output layer predicting the next 24 hours ($N_{horizon}=24$).

3. Key Contributions

1. Novel Time-Frequency Framework: The first application of CWT-based scalograms combined with a CNN for hourly district heating demand forecasting, enabling the model to learn complex temporal-periodic patterns inaccessible to standard 1D models.
2. Systematic Feature Analysis: A rigorous ablation study demonstrating that:
   • Decomposing features significantly improves accuracy.
   • Adding a weekly lag ($c_{168}$) is essential for robustness during pattern shifts.
   • Explicit cyclical time encoding is redundant when using time-frequency representations, but specific holiday features are necessary for rare events.
3. Comprehensive Benchmarking: A state-of-the-art evaluation against 14 baselines, including classical statistics (SARIMAX), tree-based models (XGBoost), RNNs (LSTM), modern Transformers (Informer, PatchTST, TimesNet), and Foundation Models (Chronos-2, TTM).
4. Multi-Dataset Validation: Validation across four distinct geographic locations (three Danish DMAs, one German city, one Danish residential dataset) over multi-year periods (up to 7 years), proving generalizability.

4. Results

The proposed framework achieved state-of-the-art performance across all datasets:

• Accuracy Improvement: The model reduced the Mean Absolute Error (MAE) by 36% to 43% compared to the strongest baseline (TimeMixer).
  • DMA A: MAE reduced from ~173 kWh (TimeMixer) to 99.76 kWh.
  • DMA B: MAE reduced from ~257 kWh to 164.81 kWh.
  • DMA C: MAE reduced from ~269 kWh to 152.60 kWh.
• Robustness: The model demonstrated superior ability to track volatile demand peaks and troughs, particularly during high-error weeks where other models (especially Transformers) produced overly smoothed forecasts or significant phase lags.
• Longitudinal Stability: A rolling-window evaluation over 7 years (Flensburg dataset) showed consistent performance with minimal degradation, confirming the model's ability to adapt to accumulating historical data.
• Holiday Handling: While aggregate metrics showed marginal gains, granular analysis revealed that holiday-specific features significantly reduced errors on specific high-impact days (e.g., Christmas, New Year) where baseline models failed.

5. Significance

• Operational Impact: The high accuracy and low variance allow utility operators to reduce safety margins on supply temperatures, directly lowering distribution losses and carbon emissions. It enables more precise integration of renewable energy sources.
• Efficiency: The study challenges the notion that massive Transformer architectures are required for time-series forecasting. It demonstrates that a lightweight, purpose-built CNN with time-frequency inputs can outperform heavy foundation models and complex attention mechanisms.
• Reproducibility: The work provides a robust, open-source framework validated on diverse, real-world data, offering a blueprint for intelligent control in modern energy systems.

In conclusion, this paper establishes a new benchmark for heat demand forecasting by leveraging the synergy between time-series decomposition, continuous wavelet transforms, and convolutional neural networks, delivering a solution that is both highly accurate and operationally robust.