Multimodal Modular Chain of Thoughts in Energy Performance Certificate Assessment

This paper introduces Multimodal Modular Chain of Thoughts (MMCoT), a cost-efficient framework utilizing Vision-Language models to improve automated Energy Performance Certificate (EPC) pre-assessment by decomposing the estimation into structured reasoning stages, which demonstrated statistically significant accuracy gains over standard prompting on a UK residential dataset.

The Gist

Imagine you want to know how energy-efficient your house is, but you don't have a certified expert with a clipboard, and you certainly can't afford to pay them hundreds of dollars to come visit. In many parts of the world, getting this "Energy Performance Certificate" (EPC) is a luxury or simply impossible because there isn't enough data or money to do it properly.

This paper introduces a clever new way to solve that problem using Artificial Intelligence. Think of it as a "Digital Detective" that can look at a few photos of a house and guess its energy rating with surprising accuracy.

Here is the breakdown of how it works, using some everyday analogies:

1. The Problem: The "Black Box" Mistake

Usually, when we ask an AI to guess something complex (like an energy rating), we just show it a picture and say, "What's the rating?"

• The Flaw: This is like asking a student to solve a complex math problem without showing their work. The AI guesses the final answer based on a gut feeling. If the house looks old, it might guess "Low Energy," but it might miss that the old house has brand-new solar panels. It treats every clue separately and often gets confused.

2. The Solution: The "Assembly Line" Detective (MMCoT)

The authors created a system called MMCoT (Multimodal Modular Chain of Thoughts). Instead of asking the AI for the final answer immediately, they force it to act like a detective on an assembly line, solving the mystery step-by-step.

Imagine a team of specialists passing a case file down a line:

• Station 1 (The Architect): The AI looks at the outside of the house and guesses the age. (e.g., "This looks like a 1920s building.")
• Station 2 (The Window Inspector): The AI looks at the windows, but now it knows the house is from the 1920s. It uses that context to guess if the windows are single or double-glazed.
• Station 3 (The Heating Expert): The AI looks at the heater. It knows the age and the windows, so it can make a smarter guess about the heating system.
• Station 4 (The Lighting Scout): It checks the lightbulbs, keeping all the previous clues in mind.
• Station 5 (The Judge): Finally, the AI takes all those previous guesses and the photos to make the final Energy Rating.

The Magic Trick: The system doesn't just guess; it passes notes. The answer from Station 1 becomes a "hint" for Station 2. This is called Chain Propagation. It's like a relay race where the baton (the information) is passed smoothly from runner to runner, ensuring the final result is based on a complete story, not just a snapshot.

3. The "Photo Cheat Sheet" (Multimodal Few-Shot)

Sometimes, the AI gets stuck. For example, it might struggle to tell the difference between a 1950s house and a 1960s house just by looking.

• The Fix: The system can show the AI a "cheat sheet" of reference photos. It's like showing a student a picture of a "1950s house" and saying, "See? Look for these specific brick patterns." This helps the AI anchor its guess before moving to the next step.

4. Why This Matters

The researchers tested this on 81 real houses in the UK.

• The Result: The "Assembly Line" detective (MMCoT) was much better than the "Gut Feeling" detective (standard AI).
• The Error: When the AI did get it wrong, it usually only missed by one step (e.g., guessing a "C" rating when it was actually a "D"). It rarely made huge mistakes (like guessing "A" when it was "G").
• The Cost: Doing this costs pennies per house. A human expert costs £60–£120. This makes it possible to check thousands of homes for the price of one.

The Bottom Line

This paper isn't trying to replace the official government inspectors. Instead, it offers a low-cost, early-warning system.

Think of it like a health checkup app on your phone. It can't replace a full medical exam by a doctor, but if you take a photo of a rash and answer a few questions, it can tell you, "Hey, that looks serious, you should see a doctor," or "That looks fine, don't worry."

For buildings, this tool can tell a landlord, "Your house looks like it has poor energy efficiency; you should probably get a real inspection and maybe fix those windows." It brings the power of energy assessment to places where it was previously too expensive or too data-poor to exist.

Technical Summary

1. Problem Statement

The assessment of building energy performance via Energy Performance Certificates (EPCs) is critical for sustainability policy and carbon markets. However, current assessment methods face significant barriers:

• Scalability and Cost: Traditional regulatory assessments (e.g., RdSAP in the UK) require trained experts to conduct on-site surveys, costing £60–£120 per property and taking 30–60 minutes. This is unfeasible for large-scale deployment, especially in developing regions or for properties where owners cannot afford the cost.
• Data Scarcity: Data-driven machine learning approaches require massive labeled datasets, which are unavailable in many regions.
• Limitations of Current VLMs: While Vision-Language Models (VLMs) offer zero-shot capabilities, standard instruction-based prompting fails to handle the multi-attribute, hierarchical nature of EPC assessment. VLMs tend to treat attributes independently, leading to poor accuracy in multi-class energy rating tasks (e.g., distinguishing between 7 EPC grades A–G) compared to simple classification tasks.

Goal: To develop a low-cost, automated pre-assessment tool that estimates EPC ratings using only limited visual information (photos) in data-scarce environments, without requiring model retraining.

2. Methodology: Multimodal Modular Chain of Thoughts (MMCoT)

The authors propose MMCoT, a framework that decomposes the complex EPC assessment task into a staged, modular reasoning pipeline. Instead of a single end-to-end prediction, MMCoT performs sequential inference over interrelated building attributes, propagating structured information between stages.

Core Architecture

The system operates as a staged inference pipeline using a unified prompt template with two key functional components:

1. Chain Propagation: Intermediate attribute predictions from earlier stages are explicitly injected as contextual inputs (structured prompts) into subsequent stages.
2. Multimodal Few-Shot Exemplars: Selective inclusion of reference images (few-shot examples) at specific stages where visual discrimination is difficult and cannot be resolved by standard assumptions.

The Five Stages

The pipeline processes a property's images through five sequential tasks:

1. Building Age Estimation:
   • Input: Exterior façade image.
   • Mechanism: Uses multimodal few-shot exemplars (images of different eras) to anchor the model's perception, as age cannot be derived from standard assumptions.
   • Output: Categorical age range (e.g., 1900–1930).
2. Window Characteristics Estimation:
   • Input: Close-up window image + Propagated building age.
   • Mechanism: Uses age as a "weak prior" to guide the model (older buildings likely have single glazing), but visual evidence takes precedence. No few-shot examples used here.
   • Output: Glazing type (Single/Double/Triple).
3. Heating System Identification:
   • Input: Heating component images + Propagated age and window data.
   • Mechanism: Uses multimodal few-shot exemplars (e.g., radiators vs. electric heaters) because heating systems are not age-dependent in standard assumptions and require explicit visual identification.
   • Output: System type (Fireplace, Boiler, Electric).
4. Lighting Efficiency Estimation:
   • Input: Interior lighting image + Propagated age, windows, and heating data.
   • Mechanism: Uses propagated attributes as context but relies on visual cues for low-energy lighting proportions.
   • Output: Efficiency range (0–100% low-energy lighting).
5. EPC Rating Inference:
   • Input: All previous images + Propagated attributes from all four stages.
   • Mechanism: Aggregates all intermediate findings. The model treats previous predictions as soft guidance (weak priors) to infer the final A–G rating, ensuring consistency across the logical chain.

3. Key Contributions

1. Novel Framework (MMCoT): Introduces a modular Chain of Thoughts approach specifically tailored for building energy assessment, moving beyond single-shot prompting to a structured, multi-stage reasoning process.
2. Hybrid Prompting Strategy: Combines Chain Propagation (logical dependency) with Multimodal Few-Shot Learning (visual anchoring). The paper demonstrates that few-shot examples are only necessary for stages where standard regulatory assumptions fail (Age and Heating), optimizing resource usage.
3. Data-Efficient Design: The method operates effectively in few-shot and zero-shot regimes, requiring no fine-tuning of the underlying VLM (GPT-4o), making it suitable for regions with no historical labeled data.
4. Cost-Efficiency: The inference cost is approximately $0.05 per property, a drastic reduction compared to expert-led surveys.

4. Experimental Results

The framework was evaluated on a dataset of 81 residential properties in the UK (images matched to ground-truth EPC records).

• Performance vs. Baselines: MMCoT significantly outperformed:
  • RdSAP Baseline: (Rule-based, using same limited inputs) showed much lower accuracy.
  • Zero-shot CLIP & GPT: (Instruction-only prompting) showed lower accuracy and recall.
  • Linear Probing: (Lightweight ML on frozen CLIP) showed marginal gains but failed to match MMCoT.
• Metrics:
  • Accuracy & Recall: MMCoT achieved the highest scores in both metrics.
  • Mean Absolute Error (MAE): Reduced from 1.988 (RdSAP baseline) to 0.741 (MMCoT).
  • Error Distribution: Errors were predominantly adjacent (e.g., predicting 'C' when the truth was 'D'), indicating the model captures the ordinal structure of EPC ratings well. Extreme classes (A and G) were harder to distinguish.
• Ablation Studies:
  • Chain Propagation: Statistically significant improvement in final EPC estimation.
  • Multimodal Few-Shots: Showed positive trends in age estimation but were not statistically significant in isolation; however, they contributed to the overall system robustness.
  • Propagation Control: Replacing propagated answers with random strings caused a significant drop in performance, proving that the information flow between stages is critical.

5. Significance and Implications

• Scalable Decision Support: MMCoT provides a viable, low-cost tool for preliminary screening, policy analysis, and awareness raising in data-scarce regions where expert assessments are unavailable.
• Methodological Advancement: The study proves that restructuring VLM reasoning to mimic human procedural workflows (decomposing tasks and propagating context) is more effective than direct end-to-end prediction for complex, multi-attribute domains.
• Limitations & Future Work: The current dataset is small (81 properties) and region-specific (UK). Future work aims to expand to diverse global datasets and incorporate additional data sources (e.g., satellite imagery) to improve fine-grained classification at extreme efficiency levels.

In conclusion, the paper demonstrates that modular, prompt-based reasoning allows Vision-Language models to overcome the limitations of data scarcity and complex attribute dependencies, offering a practical pathway for automated, low-cost building energy assessment.