🤖 machine learning

Toward a foundational thermal model for residential buildings

This paper introduces a physics-informed transformer architecture that embeds domain knowledge to achieve accurate, generalizable thermal predictions across diverse residential buildings and climates without building-specific calibration, demonstrating promising zero-shot transferability on the CityLearn dataset.

Original authors: Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to teach a computer to predict how a house will feel temperature-wise tomorrow. Right now, most computers need to be "trained" on every single house individually. It's like hiring a personal tutor for every student in a school; you teach Student A how they react to the sun, then you have to start over and teach Student B, who might live in a different house with different windows and insulation. This is slow, expensive, and doesn't scale well.

This paper proposes a new way to build a "Universal Building Brain" that can understand the physics of any house without needing a specific tutor for each one.

Here is the breakdown of their approach, using simple analogies:

1. The Problem: The "One-Size-Fits-None" Trap

Currently, if you want to control a building's heating or cooling efficiently, you need a model that knows that specific building's quirks.

The Old Way: Think of it like trying to learn a new language by memorizing every single sentence a specific person says. If you meet a new person, you have to start from scratch.
The New Goal: The authors want a model that learns the grammar of heat. If you understand the grammar (physics), you can understand any new speaker (building) immediately, even if you've never met them before.

2. The Solution: The "Physics-Infused" Translator

The authors built a new type of AI called the Physics Transformer. Instead of just staring at numbers and guessing patterns, they forced the AI to "think" like a physicist. They did this with two main tricks:

Trick A: Learning the "Speed" of Change (Derivative Enrichment)
- The Analogy: Imagine you are driving a car. A normal AI might try to memorize exactly where the car is at every second. The authors' AI, however, focuses on how fast the car is accelerating or braking.
- Why it works: It's easier to learn the rule "if I press the gas, I speed up" than to memorize every possible speed the car could be at. Similarly, the AI learns how fast a house heats up or cools down, rather than just memorizing that "House A is 72°F." This rule applies to almost every house, making it easier to transfer knowledge to new buildings.
Trick B: Predicting the "Next Step" (Euler-based Residual Prediction)
- The Analogy: Instead of trying to guess the exact temperature of a house at 5:00 PM from scratch, the AI asks, "What is the temperature right now, and how much will it change in the next hour?"
- Why it works: It's like predicting the weather. It's hard to predict the exact temperature tomorrow, but it's easier to predict that "it will get 2 degrees warmer because the sun is out." By focusing on the change rather than the absolute number, the AI learns universal rules of heat transfer that work for a house in Texas just as well as a house in Vermont.

3. The "Static" Memory

The AI also has a "memory card" it looks at before making a prediction. This card contains the permanent facts about the building: how big it is, how thick the walls are, and how many windows it has.

The Analogy: Before guessing how a person will react to a spicy meal, you first check their "profile card" to see if they are generally sensitive to spice. The AI checks the building's profile to know if it's a "slow cooker" (thick walls) or a "fast heater" (thin walls).

4. The Results: The "Zero-Shot" Magic

The team tested this on 247 simulated houses across three very different climates: hot Texas, mild California, and cold Vermont.

The "Zero-Shot" Test: They trained the AI on just two houses and then asked it to predict the temperature for a house it had never seen before.
The Result: The AI did an amazing job. It was almost as accurate as models that had been trained on 32 different houses.
The Comparison: They compared their "Physics AI" to a massive, generic AI (called TimesFM) that had been trained on millions of time-series datasets. The generic AI failed miserably on buildings (getting the temperature wrong by over 2°C). The Physics AI, with its specific "heat grammar," was much more accurate.

5. The Catch (Limitations)

The authors are honest about the limits of their work:

It's a Simulation: They tested this on computer-generated houses, not real-world buildings. Real houses have messy, unpredictable human behavior (like opening a window because someone is hot) that the simulation might not fully capture.
It's Only Houses: They only tested single-family homes. They haven't tried this on huge office buildings with complex, multi-zone heating systems yet.
Climate Limits: They tested it in three US climates. They aren't sure yet if it would work perfectly in a tropical jungle or a desert without some extra tuning.

The Bottom Line

This paper suggests that to build a truly "smart" building AI, we shouldn't just throw more data at a generic computer. Instead, we should build the computer's brain to understand the basic laws of physics (heat, insulation, change over time) from the start. By doing this, the AI can learn from a tiny amount of data and instantly understand new buildings it has never met before.

Technical Summary: Toward a Foundational Thermal Model for Residential Buildings

Problem Statement
The building energy community currently lacks a "foundational thermal model"—a single, pre-trained model capable of generalizing across diverse buildings, climates, and control strategies without building-specific calibration. Existing approaches face significant hurdles: physics-based models require extensive, building-specific calibration; data-driven models often demand large datasets and generalize poorly; and grey-box RC models struggle with nonlinearities. Furthermore, recent Time-Series Foundation Models (TSFMs) exhibit limited generalizability for building energy tasks and their scale often mismatches control latency requirements. The central research question is how to construct a transferable thermal model capable of zero-shot generalization to unseen buildings.

Methodology: The Physics Transformer
The authors propose a "Physics Transformer," a decoder-only transformer architecture designed to embed domain knowledge directly into the neural network structure rather than relying solely on massive data pre-training. The architecture incorporates three key physics-informed principles:

Derivative Enrichment: To explicitly capture rate-of-change dynamics inherent in heat transfer, the model augments the input space with first-order temporal derivatives of the covariates. These are computed using second-order centered difference schemes.
Euler-Based Residual Prediction: Instead of predicting absolute temperature values, the model predicts the temperature change ( $\Delta T_t$ ) at the next timestep. This formulation is equivalent to explicit Euler integration of an ordinary differential equation (ODE). By learning thermal response patterns (e.g., "this heat input causes a 0.5°C/hour rise") rather than absolute temperature ranges, the model focuses on universal dynamics that are more consistent across buildings than specific setpoints.
Static Feature Enrichment: The model integrates time-invariant building characteristics (e.g., floor area, aspect ratio, window-to-wall ratio, R-values, internal gain density) extracted from simulation models. These static features are projected and added to the temporal representation after encoding, allowing the model to modulate predictions based on building-specific physics without learning these relationships purely from temporal patterns.

The architecture utilizes Rotary Position Embedding (RoPE) attention to encode temporal dependencies, enabling generalization to sequence positions unseen during training. The model is trained on the CityLearn dataset, which includes simulated thermal responses for 247 residential buildings across three climate zones (Texas, California, Vermont) under five distinct HVAC control strategies.

Key Contributions

Physics-Informed Architectural Principles: The paper identifies and validates derivative enrichment and Euler-based residual prediction as inductive biases that embed heat transfer physics into neural architectures, enabling data-efficient transfer learning.
Empirical Evidence for Zero-Shot Transfer: The study provides the first systematic evaluation of thermal model transferability across both buildings within a climate and across different climate zones, demonstrating successful zero-shot transfer without fine-tuning.
Benchmarking the Foundation Model Paradigm: The work compares physics-informed designs against prevailing foundation model approaches (scale + generic architecture), challenging the notion that massive generic pre-training is sufficient for building energy tasks.

Results
Evaluated on the CityLearn dataset, the Physics Transformer achieved state-of-the-art performance:

Accuracy: The model achieved one-step prediction RMSE of 0.30°C in Texas and 0.29°C in Vermont, outperforming traditional baselines (LSTM, XGBoost) and fine-tuned Time-Series Foundation Models (TimesFM).
Data Efficiency: The model demonstrated remarkable sample efficiency. Training on as few as two buildings yielded performance comparable to an LSTM trained on 32 buildings, representing a 16-fold reduction in data requirements.
Zero-Shot Transfer: Models trained on a small subset of buildings (e.g., 2–4 buildings) generalized effectively to unseen buildings and even to different climate zones (e.g., a Texas-trained model applied to Vermont) without any fine-tuning.
Baseline Comparison: General-purpose foundation models (TimesFM-200M) failed catastrophically on building thermal prediction (RMSE > 2°C) without domain-specific adaptation, whereas the physics-informed approach succeeded with minimal data.

Significance and Claims
The authors position this work as a step toward the vision of foundational thermal models, establishing architectural principles rather than claiming a complete, universal solution. They argue that for building thermal dynamics, domain-specific inductive biases outperform massive-scale generic architectures, even when the latter are fine-tuned on in-domain data.

The paper claims that embedding heat transfer physics through derivative enrichment and residual prediction constitutes a necessary foundation for universal building energy models. The results provide empirical evidence that zero-shot transfer is achievable for thermal dynamics, offering a proof-of-concept for models that are universal, transferable, and practically deployable.

Limitations and Scope
The authors remain modest regarding the scope of their claims. They acknowledge that:

The evaluation is limited to single-family residential buildings; commercial buildings with multi-zone HVAC systems and complex controls remain untested.
All training and evaluation rely on EnergyPlus simulations rather than real-world data, creating a "sim-to-real" gap that may require future domain adaptation.
The climate diversity is limited to three U.S. zones (hot-humid, marine, cold), excluding extreme tropical, arid, or subarctic conditions.

The paper concludes that while these architectural principles are necessary, scaling them to broader building populations and validating them on real-world data are essential next steps toward truly foundational models.