🤖 AI

HetGPS: Scalable Graph Multi-Agent Reinforcement Learning with Physics-Anchored Adaptive Safety for EV Charging

HetGPS is a scalable graph-based multi-agent reinforcement learning framework that ensures safe electric vehicle charging across large fleets by decoupling intervention magnitude from direction, using a learned graph model to adaptively schedule safety authority and a physics model to guide corrective actions, thereby eliminating voltage violations while maintaining high charging success rates with a constant model size regardless of fleet scale.

Original authors: Xiangwei Wang, Nanduni Nimalsiri, Yu Xia, Peng Wang, Saman Halgamuge

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiangwei Wang, Nanduni Nimalsiri, Yu Xia, Peng Wang, Saman Halgamuge

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive digital playground where thousands of tiny, independent robots are trying to do their jobs without tripping over each other or breaking the floorboards. This is the world of Multi-Agent Reinforcement Learning (MARL). Think of it like teaching a swarm of bees to build a hive: each bee makes its own decisions based on what it sees, but they all need to work together to succeed. Now, add a layer of Graph Neural Networks. If the robots are the bees, the graph is the invisible web of connections between them, showing who is next to whom and how their actions ripple through the group. Finally, imagine a Safety Filter. This is like a strict referee who watches the bees. If a bee is about to fly into a wall, the referee steps in to steer it away. The big challenge in this field is figuring out how to let the bees be creative and efficient without letting them crash the party, especially when the swarm grows from a dozen to thousands. If the referee is too strict, the bees stop working; if they are too loose, the hive collapses.

Enter HetGPS, a new framework designed to solve this exact problem for electric vehicles (EVs). The researchers at the University of Melbourne, CSIRO, and Shanghai Jiao Tong University faced a tricky scenario: coordinating thousands of home EV chargers. If everyone plugs in at once, the local power grid can get overwhelmed, causing voltage spikes that might damage equipment or cause blackouts. The team built a system that acts like a smart, adaptive referee. Instead of using a giant, clumsy computer to watch every single car (which gets too slow and expensive as the number of cars grows), they used a "graph" approach where each car learns from its neighbors' general situation.

Here is the clever twist: HetGPS splits the referee's job into two parts. First, a learned graph model acts like a "risk detector." It looks at the current situation and decides how much it is allowed to intervene. It asks, "Is this a calm day, or is the grid about to explode?" If it's calm, it lets the cars charge freely. If it's risky, it tightens the leash. Second, a physics model acts as the "steering wheel." Once the risk detector says, "We need to intervene!" the physics model takes over to figure out exactly which way to push the cars to fix the voltage. It uses the laws of electricity (physics) to ensure the correction actually works, rather than guessing.

The results from their simulations are impressive. They tested this system on five different network sizes, ranging from 200 to 3,218 electric vehicles. Without this safety filter, the system caused voltage violations (dangerous spikes) in about 3.9% to 7.7% of the time steps. With HetGPS, they slashed that number down to between 0.52% and 3.44%, all while ensuring that 99% to 100% of the cars still got fully charged and were ready to leave on time. Perhaps most exciting is the system's ability to scale. The "brain" of the system has a fixed size of about 383,700 learned parameters, no matter if there are 200 cars or 3,218. In contrast, a traditional "centralized" brain that tries to control every car individually would need to be about 170 times larger for the biggest fleet.

The team also showed that a policy trained on a medium-sized network (1,236 cars) could be dropped into a much larger network (3,218 cars) or a completely different type of grid without any extra training, a feat known as "zero-shot transfer." It worked almost as well as if it had been trained specifically for that new size. While the paper notes these are simulations and real-world deployment would need further validation, the results suggest that separating the "how much to stop" decision from the "which way to turn" decision is a powerful way to keep large groups of AI agents safe, efficient, and scalable.

Technical Summary: HetGPS

Problem Statement

Coordinating residential Electric Vehicle (EV) charging presents a dual challenge: scalability and network safety. While uncoordinated charging ignores the electrical coupling between agents that leads to voltage violations, centralized coordination becomes computationally intractable as fleet sizes and network topologies grow. Existing Graph-based Multi-Agent Reinforcement Learning (MARL) approaches offer scalability through parameter sharing but often fail to guarantee that every realized joint action satisfies physical constraints under dynamic demand and generation. Furthermore, standard safety interventions (e.g., fixed limits or training-time penalties) either lack the flexibility to handle high-risk scenarios or unnecessarily distort useful actions in benign conditions. The core problem is how to deploy a scalable, learned policy that protects shared network constraints (specifically bus voltages) without overriding task-oriented decisions or requiring a model size that scales with the number of agents.

Methodology: HetGPS Framework

The authors propose HetGPS, a hybrid graph-control framework that decouples the magnitude of safety intervention from its direction. The system operates as a safety filter layer between a learned MARL policy and the physical grid actuation.

1. Hybrid Safety Filter Factorization

The core innovation is the separation of two roles:

Learned Graph Risk (Authority Scheduling): A graph-based model predicts the risk of voltage violations based on the current state and the proposed actions. This model does not determine how to correct the action but rather schedules a state-dependent intervention authority ( $\Delta_t$ ). This defines the maximum permissible change to the proposed EV actions.
Physics-Anchored Direction: A linearized power-flow model (LinDistFlow) determines the direction of the correction. It calculates the physical sensitivity of bus voltages to EV power injections. The filter projects the learned policy's proposal onto a feasible set defined by these physical sensitivities and the learned authority budget.

2. Scalable Graph MARL Architecture

Heterogeneous Graph: The system models the distribution network as a heterogeneous graph with bus nodes (exposing voltage and aggregate power) and EV agent nodes (holding private states like State of Charge and departure time).
Parameter Sharing: A shared Soft Actor-Critic (SAC) policy and a shared graph encoder are used across all agents. The graph encoder aggregates topology-aware electrical context from the operator side without pooling private EV states into a centralized critic.
Action-Conditioned Residual Model: During training, a residual head is supervised by power-flow labels to predict voltage outcomes. At deployment, this model acts as the risk estimator to trigger the safety filter.

3. Execution Pipeline

Proposal: The shared graph policy proposes joint EV actions ( $a^{RL}_t$ ).
Risk Assessment: The residual model predicts voltage consequences. If the predicted risk exceeds a threshold, an authority budget $\Delta_t$ is calculated.
Projection: If triggered, a convex quadratic program (solved via sequential projection sweeps) adjusts the actions. The correction direction is strictly determined by the physics model ( $J_t$ ), while the magnitude is bounded by $\Delta_t$ .
Service Map: The final action is passed through a deterministic service map ( $G_{svc}$ ) enforcing availability, SoC limits, and departure rules.

Key Contributions

Hybrid Safety Filter Factorization: The paper introduces a novel factorization where a learned graph risk model schedules state-dependent intervention authority, while a physics model determines the corrective direction. This avoids the need for a single model to infer both magnitude and direction.
Scalable Parameter-Shared Architecture: HetGPS instantiates this principle using a parameter-shared graph MARL policy and an action-conditioned voltage-residual model. Crucially, the number of learned parameters remains fixed regardless of fleet size (e.g., 383,702 parameters for both 200 and 3,218 EVs), whereas a matched centralized SAC actor scales linearly (reaching ~65 million parameters for 3,218 EVs).
Zero-Shot Topology Transfer: The framework demonstrates the ability to transfer policies trained on smaller networks (e.g., 1,236 EVs) to larger, unseen topologies (e.g., 3,218 EVs) and different feeder families (IEEE 33) with zero-shot or few-shot adaptation, maintaining high safety and service levels.

Experimental Results

The framework was evaluated on five nested CRE21 distribution networks (200 to 3,218 EVs) and three IEEE feeders over 100 evaluation days.

Safety Performance: Adaptive Authority reduced bus-step voltage violations from 3.93–7.74% (without filtering) to 0.52–3.44%, while maintaining 99.06–100% departure success rates.
Reward-Safety Trade-off: Compared to a fixed-authority physics projection, the adaptive scheduler improved the mean reward on all five CRE21 networks and lowered the mean safety score on four.
Scalability: At the largest scale (3,218 EVs), the HetGPS deployed model is approximately 170× smaller than a fully centralized SAC actor.
Transferability: A policy trained on an 8-transformer system (1,236 EVs) transferred zero-shot to 16- and 32-transformer systems, achieving violation rates of 0.57–0.75% and at least 99.99% departure success.

Significance and Claims

The paper claims that learned graph risk can effectively allocate intervention authority at scale, while feeder physics anchors the corrective action. This approach resolves the tension between protecting shared constraints and preserving useful learned coordination.

The authors emphasize that their evidence is empirical. They explicitly state that due to the use of balanced power flow approximations and the absence of a certified model-error margin, they do not claim pointwise feasibility for every stochastic realization. Instead, they present a practical framework where the learned model compresses variable-size network states and assigns intervention authority, while the physics model ensures the direction of correction remains physically grounded. The authors suggest this factorization extends to other graph-coupled systems with tractable first-order action-to-constraint sensitivities, such as coordinated inverter/storage control and district heating, though they note that cross-domain validation and certified margins remain future work.