⚡ electrical engineering

Data Scarcity in Gas Load Profiling: Generalized Proxy-Guided Load and Temporal Disaggregation

This paper introduces a Generalized Proxy-Guided Load and Temporal Disaggregation framework that overcomes severe data scarcity in gas load profiling by transforming low-frequency utility data into high-resolution thermal profiles through a four-stage process validated on multi-unit residential buildings.

Original authors: Lucas Krome, Mahtab Aboufazeli, Soosan Beheshti, Bala Venkatesh

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lucas Krome, Mahtab Aboufazeli, Soosan Beheshti, Bala Venkatesh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out exactly how much natural gas a building uses every hour to heat its rooms and heat its water. You want this data to help design better, greener heating systems. But here's the problem: the utility company only sends you a bill once a month. It's like trying to guess the exact speed of a car at every second of a trip, but you only know the total distance driven and the time it took.

This paper introduces a clever "mathematical detective" framework to solve this mystery. The authors call it the Generalized Proxy-Guided Load and Temporal Disaggregation framework. Here is how it works, broken down into simple steps:

The Core Problem: The "Foggy" Bill

Most buildings only have a monthly gas bill (low resolution) but no hourly gas meter. Without hourly data, you can't tell when people are using hot water versus when the furnace is running to heat the house. Standard computer programs that learn from data (AI) usually need huge amounts of detailed data to work, which these buildings don't have.

The Solution: Using Electricity as a "Shadow"

The researchers realized that while they don't have hourly gas data, they do have hourly electricity data. They decided to use electricity as a proxy (a stand-in or a shadow) to guess what the gas usage looks like.

Think of it like this: If you see a shadow moving in a specific pattern, you can guess how the person casting it is moving, even if you can't see the person directly.

The Four-Step Detective Process

1. Cleaning the Shadow (Weather Normalization)
Electricity usage isn't just about people being home; it's also about the weather. If it's hot, people turn on AC, which spikes electricity use but has nothing to do with gas or hot water.

The Analogy: Imagine trying to hear a whisper in a noisy room. The "noise" is the AC kicking on. The researchers used a mathematical filter to "mute" the weather-related noise. They stripped away the parts of the electricity bill caused by cooling, leaving behind a clean "behavioral signal" that shows when people are actually using lights and appliances. This clean signal becomes their Occupancy Proxy.

2. Grouping the Clues (Unified Segmentation & Pooling)
Since the data is sparse (only one gas bill a month), looking at just one building is like trying to solve a puzzle with only three pieces.

The Analogy: The researchers gathered data from 11 different buildings and treated them like a single, giant puzzle. They looked specifically at the "non-heating" months (summer) when the gas is only used for hot water. By mixing (pooling) the data from all 11 buildings, they could see a clear pattern of how people use hot water, even if individual buildings had very little data.

3. Customizing the Recipe (Local Calibration)
Once they figured out the general pattern of hot water usage from the group, they had to make it fit each specific building.

The Analogy: Imagine a master chef creates a perfect soup recipe for a crowd (the group pattern). But every kitchen has slightly different pots and stoves. The researchers applied a "calibration factor" to adjust the master recipe for each specific building, ensuring the math matched that building's unique reality.

4. Splitting the Bill (Temporal Disaggregation)
Finally, they took the total monthly gas bill and split it back into two distinct streams:

The Hot Water Stream: Driven by the "behavioral shadow" (when people are home and using appliances).
The Heating Stream: Driven by the weather (how cold it is outside).
They used an optimization algorithm to distribute the total monthly gas amount into every single hour of the month, ensuring the total adds up perfectly to the bill while following the logic of when heating and hot water actually happen.

The Results: Did It Work?

The team tested this on 11 apartment buildings over 18 months.

The Verdict: The method was highly accurate. When they compared their reconstructed hourly gas data against the actual total monthly bills, the error was very small (about 6.37% on average).
The "Shadow" Test: They also tested their method on a dataset where they did have real, high-speed gas data (the AMPds dataset). Even without seeing the real gas data during the training, their "shadow" method correctly guessed the patterns of heating vs. hot water usage with about 10-11% error.

The Bottom Line

This paper proves that you don't need expensive, invasive gas meters installed in every home to get detailed hourly data. By using existing electricity data as a "behavioral shadow" and applying smart math to clean and group the data, you can reconstruct a high-quality, hour-by-hour picture of gas usage. This allows city planners and engineers to design better decarbonization strategies for existing buildings without needing new hardware.

Technical Summary: Generalized Proxy-Guided Load and Temporal Disaggregation

Problem Statement
The decarbonization of the building sector, particularly through the electrification of heating systems, is hindered by a critical lack of granular thermal load profiles. While advanced forecasting models (ranging from deep neural networks to physics-based tools like EnergyPlus) require high-resolution data, such data is rarely available for existing building stocks. High-resolution gas metering falls outside standard utility business requirements, leaving researchers with only low-frequency, irregular, and sparse monthly billing data. Conventional machine learning approaches are impractical in this context due to data scarcity, while traditional inverse modeling methods (e.g., degree-day methods) lack the intra-day resolution necessary for precise equipment sizing and stochastic occupant behavior modeling.

Methodology
The paper proposes the Generalized Proxy-Guided Load and Temporal Disaggregation framework, a four-stage process designed to reconstruct high-resolution (hourly) gas consumption profiles from low-frequency monthly gas bills and hourly electricity data. The methodology is predicated on decomposing total natural gas consumption ( $G_{total}$ ) into two distinct physical regimes: space heating ( $G_{heat}$ ) and domestic hot water ( $G_{dhw}$ ), where the latter serves as the system's baseload driven by stochastic occupant behavior.

The framework operates through the following sequential steps:

Generalized Occupancy Proxy Extraction via Weather Normalization:
To isolate behavioral signals from hourly electricity data ( $E$ ), the authors apply Multivariate Weather Normalization using a Multiple Linear Regression (MLR) framework. This step removes seasonal cooling loads (confounded by air conditioning) by regressing electricity against meteorological variables (wind speed, humidity, cooling degree hours) and temporal priors. The resulting Occupancy Proxy ( $E_N$ ) is a clean behavioral signal representing lighting and appliance usage, derived by zeroing out thermal components in the regression model.
Unified Segmentation and Normalized Pooling:
To address the statistical insignificance of sparse individual records, the framework identifies "non-heating" periods (defined as months where average Heating Degree Hours < 5). Data from these periods across 11 multi-unit residential buildings are pooled. To ensure comparability, electricity is normalized by floor area, and gas consumption is normalized by the building's specific volume-to-area ratio ( $\phi$ ). This creates a unified dataset ( $D_{pool}$ ) that reveals robust trends invisible at the individual building level.
Unified Baseload Parameter Estimation and Local Calibration:
A generalized Ordinary Least Squares (OLS) model is trained on the pooled dataset to establish a global relationship between the occupancy proxy and domestic hot water consumption ( $G_{dhw}$ ). To account for building-specific variations in appliance efficiency, a Local Calibration step is applied. A scalar calibration factor ( $\kappa_i$ ) is derived for each building by minimizing the residual between the universal prediction and observed gas consumption during the building's most representative non-heating month.
Component-Wise Temporal Disaggregation:
The final stage reconstructs hourly profiles for both components using a dual-stream optimization approach.
- Baseload Reconstruction: Driven by the electrical proxy features ( $X_{dhw}$ ), modeling occupant behavior.
- Heating Reconstruction: Driven by meteorological features ( $X_{heat}$ ), modeling thermodynamic demand.
  An optimization algorithm minimizes the error between aggregated hourly predictions and known monthly targets, ensuring the synthesized profiles strictly conserve the energy mass of the original monthly bills while adhering to specific temporal dynamics.

Key Results
The framework was validated on two datasets: an 18-month operational dataset from 11 multi-unit residential buildings and the high-resolution Almanac of Minutely Power dataset (AMPds).

Proxy Effectiveness: The weather-normalized occupancy proxy ( $E_N$ ) demonstrated a mean Pearson correlation of 0.74 with non-heating gas consumption, a significant improvement over raw electricity data (mean $r = 0.28$ ), confirming the successful isolation of behavioral signals.
Baseload Accuracy: The unified pooling and calibration strategy achieved a Mean Squared Percentage Error (MSPE) of 8.58% for the baseload period across the portfolio.
Total Load Reconstruction: The reconstructed total gas consumption ( $\hat{G}_{total}$ ) achieved an average portfolio-wide MSPE of 6.37% and a Normalized Root Mean Square Error (NRMSE) of 0.20. The model successfully tracked both high-frequency baseload variations and heating peaks without significant temporal lag.
High-Resolution Validation: When tested on the AMPds dataset (minute-level data), the framework effectively decoupled heating and baseload components, yielding component-wise MSPEs of approximately 10–11%, demonstrating robustness even at minute-level intervals.

Significance and Claims
The paper claims that this framework effectively bridges the resolution gap between low-frequency utility data and the high-fidelity training sets required for data-driven decarbonization modeling. By utilizing hourly electricity as a dynamic, non-intrusive proxy for occupant behavior, the method overcomes the data scarcity that typically prevents the application of advanced AI analytics in building retrofits.

The authors position the framework as a scalable solution for generating granular training data without the need for sub-metering infrastructure. The significance lies in its ability to transform sparse, monthly billing records into reliable, high-resolution thermal profiles, thereby enabling precise equipment sizing and the development of sustainable electrification strategies for existing building stocks. The paper concludes that this approach offers a practical tool to accelerate building electrification retrofits while maintaining physical consistency with energy mass balance.