HearthNet: Edge Multi-Agent Orchestration for Smart Homes

HearthNet is an edge-based multi-agent orchestration system that deploys persistent, role-specialized LLM agents on home hubs to enable robust, natural language control of smart devices through decentralized coordination, shared state management, and strict authorization boundaries.

The Gist

Imagine your smart home is a busy, chaotic restaurant kitchen.

Right now, most smart homes run on a simple rulebook: "If the sun goes down, turn on the lights." This works fine until things get complicated. What if you say, "I'm working from home," and the system needs to dim the lights, lower the music, and turn off the TV all at once? Current systems often crash, get confused, or do the wrong thing because they don't have a "brain" that remembers what happened five minutes ago or who gave the order.

HearthNet is a new system designed to fix this. It turns your smart home from a chaotic kitchen into a well-orchestrated theater production.

Here is how it works, using simple analogies:

1. The Cast of Characters (The Agents)

Instead of one giant brain trying to control everything, HearthNet uses a small team of specialized "actors" living on your home's local hardware (like a small computer or your phone).

• The Director (Root Agent): This is the boss. When you say, "I'm working from home," the Director doesn't touch the lights directly. Instead, they break the request down into tasks and hand them to the specialists.
• The Specialists (Manager Agents):
  • Jeeves: The expert on everything connected to your main hub (lights, thermostat, speakers).
  • Darcy: The expert on your phone and apps. Some smart devices (like a specific TV app) can't be controlled by the main hub; they need someone to physically tap the screen. Darcy does this.
• The Librarian (Observer Agent): This is the most important character for safety. The Librarian doesn't give orders. Their only job is to watch everything, write it down in a permanent, unchangeable diary (a Git repository), and make sure everyone is telling the truth.

2. The Rules of the Game (How They Talk)

In a normal smart home, if the internet cuts out, the system forgets everything. In HearthNet, the team communicates via a whisper network (MQTT).

But here is the magic trick: They don't rely on memory.
If the Director gets a power outage and restarts, they don't say, "Oh no, I forgot what we were doing!" Instead, they walk over to the Librarian's diary, read the last entry, and pick up exactly where they left off. This means the system never gets "confused" by crashes.

3. The Safety Check (The "Lease" System)

This is the paper's biggest innovation. Imagine the Director wants to turn on the lights. They can't just shout, "Lights on!"

1. The Proposal: The Director asks the Specialist (Jeeves), "Can we turn on the lights?"
2. The Check: Jeeves checks the Librarian's diary to see if the lights are already on or if someone else is trying to turn them off.
3. The "Lease": If everything looks good, the Director hands Jeeves a digital ticket (a lease). This ticket says: "You are allowed to turn on the lights, but ONLY for the next 5 seconds, and ONLY if the state of the house hasn't changed since I wrote this ticket."
4. The Execution: Jeeves tries to use the ticket. If the house state changed (e.g., someone manually turned the lights off in the meantime), the ticket becomes stale (like an expired coupon), and the lights won't turn on. This prevents the system from doing dangerous or conflicting things.

4. The Live Show (The Demonstrations)

The paper shows three real-life scenarios where this system shines:

• Scene 1: The "Work From Home" Command. You say, "I'm working from home." The Director figures out what that means: dim the lights, lower the volume, and turn off the TV. The team coordinates perfectly, and the Librarian writes down exactly who did what and when.
• Scene 2: The Conflict. Imagine you set a timer for "Evening Relax" to dim the lights, but you just manually turned them bright to read a book. The system sees this conflict. Because the Librarian has a timeline of events, the Director knows your manual action (just now) is more important than the timer (from 10 minutes ago). The system cancels the timer and keeps the lights bright.
• Scene 3: The Crash Recovery. Imagine one of the agents (Jeeves) crashes and restarts. When it wakes up, it tries to use an old command it was holding. The Director checks the Librarian's diary, sees that the house state has changed, and says, "No, that ticket is expired!" The agent is forced to get a fresh ticket before acting. This stops the system from accidentally turning on the oven because it "forgot" you turned it off.

Why Does This Matter?

Current smart homes are like paper maps; if you lose your place, you're lost.
HearthNet is like a GPS with a live feed; it knows exactly where you are, what happened five minutes ago, and it won't let you drive off a cliff even if the signal glitches.

It keeps your smart home safe, auditable, and smart, even when the internet is shaky or the devices act up. It moves the "brain" of your home out of the cloud and right into your house, where it can actually see and control your physical world.

Technical Summary

1. Problem Statement

Current smart home automation faces a critical mismatch between the capabilities of modern AI agent toolkits and the realities of persistent, physical edge environments.

• Session vs. Persistence: Dominant multi-agent frameworks (e.g., AutoGen, LangGraph, Anthropic's Agent Teams) are optimized for transient, session-scoped delegation. They rely on shared context windows that vanish when a session ends, making them unsuitable for smart homes where coordination must be long-running, event-driven, and persistent.
• Fragility in Physical Environments: Real-world deployments suffer from device failures, network partitions, and battery deaths. Existing systems often lack explicit mechanisms for recovery, conflict resolution, or authorization boundaries, leading to safety and privacy risks.
• Context Loss: A monolithic agent with implicit context or ephemeral subagents cannot reliably handle heterogeneous devices across different privilege domains without a shared, durable state mechanism.

2. Methodology: HearthNet Architecture

HearthNet is an edge-based multi-agent orchestration system designed to run on commodity hardware. It externalizes context, separates planning from actuation, and enforces strict authorization boundaries.

2.1 Hierarchical Role Architecture

The system employs four distinct roles running as independent processes on separate devices, coordinated via MQTT and a Git-backed shared state:

1. Root Agent ("Rupert"): The top-level orchestrator (running on a Mac Mini). It receives high-level user commands (via Telegram), decomposes them into domain-specific tasks, arbitrates conflicts, and is the only component authorized to issue actuation leases.
2. Manager Agents: Specialized agents governing specific device domains.
   • Jeeves (Home Assistant Manager): Runs on an x64 NUC, managing standard IoT devices (lights, climate, media).
   • Darcy (Mobile Manager): Runs on an Android phone, controlling proprietary apps via UI automation and accessing on-device sensors (camera, gyroscope) unavailable to hub agents.
3. Librarian Agent ("Dewey"): A non-participating observer running on a separate NUC. It maintains a Git repository as the canonical source of truth. It mirrors MQTT traffic, records events with structured metadata, and provides versioned state for recovery and auditability.
4. Device Adapters: Thin, deterministic endpoints controlled by managers. Devices themselves are not agents, preventing an "agent-per-device" explosion.

2.2 Core Mechanisms

• Git-Backed Freshness & State: All agents load the current system state (device shadows, policy snapshots, repository HEAD) from the Git repository before reasoning. This replaces session-local memory with a durable, versioned history.
• Actuation Leases: To prevent stale or unauthorized commands, the Root Agent issues short-lived, machine-checkable leases for every state-changing operation. A lease binds:
  • Grantee identity.
  • Target device/scope.
  • Permitted parameters.
  • The base_commit (state version) under which approval was granted.
  • Expiration time and justification.
  • Adapters reject any command lacking a valid, non-expired lease or referencing an outdated commit.
• Execution Protocol:
  1. Ground: Agents load current state from Git.
  2. Propose: Managers translate tasks into concrete actions (no execution yet).
  3. Verify & Grant: Root checks freshness against the current HEAD, validates against policy, and issues a lease.
  4. Execute & Record: Manager executes via the adapter; Librarian records the outcome in Git.

3. Key Contributions

1. Git-Backed Freshness Mechanism: A base-commit verification system allowing agents to detect and reject stale or conflicting commands before they reach physical devices.
2. Policy-Bound Actuation Leases: A security model where the root agent issues short-lived authorizations binding commands to specific identities, scopes, and state versions, turning authorization into a checkable boundary.
3. Librarian-as-Observer Pattern: A dedicated agent that mirrors coordination traffic into a versioned repository, enabling crash recovery, full auditability, and attribution without participating in the control loop.
4. Deployed Edge Prototype: A working system running on three classes of hardware (ARM64 Mac Mini, x64 NUCs, Android Phone) demonstrating real-device control without simulation.

4. Results & Evidence

The system was evaluated through three live scenarios on a testbed with ~10 devices and 4 agents:

• Scene 1: Intent-Driven Coordination: Successfully interpreted an ambiguous command ("I'm working from home") and decomposed it into multi-device actions (lighting, media, TV) across different managers.
  • Latency: Median end-to-end time of 8 seconds (dominated by LLM inference, not orchestration overhead).
• Scene 2: Conflict Resolution: Detected a conflict between a scheduled "evening wind-down" and an active user intent. The system used the Git timeline to prioritize the explicit user request over the schedule, preventing the conflict.
  • Success Rate: 5/5 conflicts correctly detected and resolved.
• Scene 3: Safety & Freshness: Simulated a crash recovery where a manager attempted to replay a stale command with an expired lease. The Root Agent rejected the command based on the base_commit mismatch.
  • Success Rate: 5/5 stale/expired commands rejected; 0 false rejections.

Performance Metrics:

• Lease Validation Overhead: <0.01 ms (p95), proving the security checks are negligible compared to LLM inference.
• Auditability: 100% of events (153/153) persisted to Git with full attribution.

5. Significance

HearthNet addresses a fundamental gap in AI agent deployment: the transition from cloud-based, session-limited agents to persistent, safety-critical edge systems.

• Safety First: By decoupling planning from actuation and enforcing strict lease/freshness checks, it prevents the "hallucinated" or stale commands that could cause physical harm in smart homes.
• Resilience: The Git-backed state model ensures that agents can restart after a crash and resume operations correctly without needing to reconstruct conversational history.
• Auditability: The system provides a complete, diffable history of all decisions and actions, which is crucial for debugging and compliance in smart home environments.
• Feasibility: It demonstrates that complex, multi-agent orchestration for heterogeneous physical devices is achievable today on commodity edge hardware, provided the architecture prioritizes explicit state management over implicit context.

The paper concludes that while LLM inference latency remains the primary bottleneck, the orchestration layer itself is highly efficient and robust, paving the way for more reliable, autonomous smart home ecosystems.