Technical Summary: OptAgent – An Agentic AI Framework for Intelligent Building Operations

1. Problem Statement

The building sector accounts for over one-third of global energy use, and achieving net-zero emissions by 2050 requires decarbonizing thousands of buildings daily. Current building operations rely heavily on human-intensive engineering workflows, expert knowledge, and manual effort, which cannot scale to meet these demands. While Large Language Models (LLMs) offer potential for automation, existing applications in building domains often conflate Generative AI with Agentic AI, resulting in systems that are limited to passive question-answering or isolated single-agent tasks.

Three critical gaps hinder the deployment of autonomous building operations:

1. Lack of Systematic Frameworks: Most studies focus on ad-hoc single-agent implementations rather than robust, hierarchical multi-agent coordination architectures capable of handling the complexity of building energy systems.
2. Absence of Comprehensive Benchmarks: There is a scarcity of rigorous evaluations regarding success rates, planning accuracy, token consumption, and computational costs for agentic workflows in this domain.
3. Limited Toolsets and Environments: Current agentic systems often rely on static simulation engines (e.g., EnergyPlus) that lack real-time adaptability and closed-loop integration with Distributed Energy Resources (DERs), HVAC control, and grid interactions. Furthermore, existing tools are often designed for human engineers rather than AI agents, leading to execution failures.

2. Methodology

The authors propose OptAgent, an end-to-end framework coupling a multi-agent Agentic AI decision layer with a Physics-Informed Machine Learning (PIML) runtime environment.

2.1 The Agentic AI Layer (The "Brain")

The framework utilizes a hierarchical multi-agent system implemented via the Model Context Protocol (MCP):

• Architecture: It consists of a Concierge Agent (user interface), an Orchestrator Agent (global planner), and a cluster of 11 Specialist Agents (domain-specific roles for building, HVAC, DER, simulation, etc.).
• Intelligence Modes: The study evaluates three coordination strategies:
  • Centralized Single-Stage (C-1): The orchestrator generates a full execution plan with all tool schemas in one pass.
  • Centralized Two-Stage (C-2): The orchestrator first performs lightweight routing to identify agents/tools, then generates detailed parameterized instructions in a second pass to mitigate "prompt explosion."
  • Decentralized (D): The orchestrator provides high-level goals, while specialist agents autonomously select tools and parameters.
• Dynamic Adaptation: The system includes an "HR-like" mechanism where the orchestrator can dynamically generate new specialist agents or revise existing agent cards if capability gaps are detected, allowing the system to adapt to evolving tool registries without manual redesign.

2.2 The PIML Environment (The "Body")

The decision layer interacts with BESTOpt, a modular, physics-informed runtime environment.

• Capabilities: BESTOpt integrates building thermal dynamics, HVAC systems, DERs, grid interactions, and occupancy behavior.
• Structure: It employs a hierarchical modular structure (Cluster → Domain → System/Building → Component) and a unified data typology (State, Action, Disturbance, Observation).
• Interaction: Agents interact with BESTOpt via 72 standardized MCP tools, enabling closed-loop execution where agents perceive system states, apply control actions, and receive physical feedback.

2.3 Evaluation Design

The authors conducted a large-scale benchmark involving 3,975 runs across:

• 50 Test Cases: Categorized by complexity (Single/Multi-Agent × Single/Multi-Tool).
• 25 Model Configurations: Pairings of 5 LLM tiers (from 1.7B to API-class) for orchestrator and specialist roles.
• Metrics: Accuracy (tool selection, agent selection, plan steps, parameter extraction), timing, token consumption, and inference cost.

3. Key Contributions

1. First Multi-Agent Framework for Building Operations: The study presents the first end-to-end multi-agent agentic AI framework specifically designed for intelligent building operations, featuring a top-level orchestrator and 11 specialist agents.
2. Integration with BESTOpt: The framework is integrated with a physics-informed modular environment via 72 domain-specific MCP tools, supporting coordinated analysis across building, HVAC, DER, and grid domains.
3. Comprehensive Benchmarking: The paper provides a systematic evaluation of agentic workflows, quantifying the impacts of intelligence mode design, model size, task complexity, and orchestrator-specialist coordination on performance.

4. Results and Findings

4.1 Case Study Performance

In a representative case study assessing the impact of HVAC COP upgrades and battery capacity increases, the framework successfully coordinated multi-step simulations. Results showed that while COP upgrades improved efficiency, they did not alter thermal comfort trajectories without control changes. However, adding a pre-cooling strategy reduced temperature violations and improved PV self-consumption by 20.8%, though it increased total grid import and operating costs due to rule-based DER control limitations.

4.2 Benchmark Evaluation Insights

• Intelligence Mode: The Centralized Two-Stage (C-2) mode achieved the best trade-off between accuracy and cost. It outperformed Single-Stage (C-1) in accuracy (e.g., tool selection accuracy of 0.69 vs. 0.22) and reduced orchestrator token usage by 16%. Decentralized (D) modes shifted reasoning burden to specialists, increasing total runtime by 36% and specialist token usage by 146%.
• Model Configuration: Orchestrator capability dominates system performance. Upgrading the orchestrator yielded significant gains even with lightweight specialists. Conversely, scaling specialist agents with a weak orchestrator provided limited improvement. Notably, mid-sized models (e.g., 4B) exhibited lower reliability than smaller (1.7B) or larger models, often attempting "expert" reasoning without sufficient capability, leading to workflow drift.
• Task Complexity: Performance followed an "all-or-nothing" distribution. When the orchestrator correctly planned the workflow, execution succeeded; early planning errors led to total breakdown. Multi-tool reasoning was identified as the primary bottleneck, causing significant accuracy drops in complex Multi-Agent Multi-Tool (MAMT) tasks.
• Coordination Dynamics: Specialist agents adhered to orchestrator plans in 82.7% of cases. Deviations (mostly tool removals) consistently reduced accuracy. Interestingly, when specialists were larger than the orchestrator, they were more likely to override plans, but these overrides degraded performance.

5. Significance and Lessons Learned

The paper concludes that OptAgent establishes a scalable, physics-grounded foundation for deploying agentic AI in decarbonized building operations. The authors derive six key lessons for future system design:

1. Larger Models Are Not Always More Reliable: Infrastructure constraints (timeouts, resource contention) can make extra-large models less reliable than smaller, stable models. Systems must be compute-aware.
2. Architecture Matters More Than Prompts: Structural design (e.g., two-stage planning) is more critical for scalability than prompt optimization. Hierarchical organization is necessary to manage prompt explosion as tool ecosystems grow.
3. Orchestrator-Specialist Alignment: Investing in a strong orchestrator is more effective than uniformly scaling all agents. Mid-tier specialists attempting to "re-reason" strong orchestrator plans often introduce errors.
4. Agent-Native Tool Design: Tools designed for humans (complex nested schemas) fail with agents. Tools must be simplified, flattened, and typed specifically for LLM interpretability.
5. Human-Agent Co-Adaptation: Reliability depends on how users frame tasks. Systems require interaction designs (e.g., clarification loops) to align human intent with executable workflows.
6. Dynamic and Self-Evolving Systems: Static agent pools are insufficient for long-term autonomy. Frameworks must support dynamic role generation and structural adaptation based on execution feedback.

The work highlights that while agentic AI offers a paradigm shift from rule-based automation to context-aware decision-making, realizing its potential in building operations requires careful attention to system architecture, tool design, and the interplay between model capabilities and operational constraints.