💻 computer science

Replicating the behaviour of electric vehicle drivers using an agent-based reinforcement learning model

This paper proposes a multi-stage reinforcement learning framework to simulate the adaptive and bounded-rational charging behaviors of private electric vehicle drivers across a national road network, successfully validating the model against real-world data to identify critical charging deserts and inform policy on rapid charging hub expansion.

Original authors: Zixin Feng, Qunshan Zhao, Alison Heppenstall

Published 2026-02-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zixin Feng, Qunshan Zhao, Alison Heppenstall

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a giant, digital simulation of the entire United Kingdom, filled with thousands of virtual electric vehicle (EV) drivers. This is the core of the research paper by Feng, Zhao, and Heppenstall. Their goal was to build a computer model that doesn't just follow rigid rules, but actually learns how real people drive and charge their cars, just like a human does.

Here is a breakdown of their work using simple analogies:

1. The Problem: The "Robot" vs. The "Human"

Previous computer models for EVs were like robots following a strict instruction manual. They would say, "If battery is low, charge now." They couldn't adapt. If a driver got used to their car, they might feel comfortable driving with a lower battery, but the old models didn't understand that. They also mostly focused on optimizing fleets of taxis or buses, not regular people making their own independent choices.

The researchers wanted to build a model that acts more like a human learner. They wanted to see how drivers adapt over time, make mistakes, learn from them, and develop "habits" based on their own experiences.

2. The Solution: The "Video Game" Approach

To solve this, the team used a technique called Reinforcement Learning (RL). Think of this like training a character in a video game:

The Player: The virtual EV driver.
The Goal: Get from Point A to Point B without running out of battery (game over).
The Rewards/Penalties:
- Good: Reaching a destination, keeping a comfortable battery level, and not spending too much money or time.
- Bad: Running out of power (stranded), paying high fees, or getting stuck in a long line at a charger.

Instead of programming the drivers with specific rules, the researchers let the virtual drivers play the "game" thousands of times. Through trial and error, they learned the best strategies to survive.

3. The "Training Camp" Method

The researchers didn't just run the simulation once. They used a multi-stage training process:

Grouping: They sorted thousands of real-world trip data (from a national survey) into different "clumps" or clusters. Some drivers take short trips to work; others take long trips for leisure; some start with full batteries, others with half-full ones.
Training: They picked a few "representative" drivers from each group to train in the RL model. These drivers learned how to navigate the UK road network and decide when to charge.
Simulation: Once the "trainees" learned their lessons, the researchers used those learned behaviors to simulate the entire population of drivers.
Reality Check: They compared the virtual drivers' behavior against real-world data from actual charging stations (like ChargePoint). They looked for the specific "training episode" where the virtual drivers acted most like real humans.

Key Discovery: They found that the "perfect" strategy wasn't the answer. Real humans aren't perfect; they have "bounded rationality" (they make good-enough decisions based on limited info). The model that matched reality best was one where drivers were still learning and adapting, not one where they had already become perfect robots.

4. Finding the "Charging Deserts"

Once the model was validated, the researchers used it to look at the map of Great Britain. They were looking for "Charging Deserts."

Imagine a map where:

Red Zones: Drivers are constantly running low on battery (low "State of Charge").
Blue Zones: There are very few charging stations nearby.

Where Red and Blue overlap, you have a Charging Desert. These are places where drivers are likely to get stranded because they are low on power and there are no stations nearby to help them.

What they found:

These deserts aren't just in the middle of nowhere. They are often on the edges of big cities (like Outer London) and along major highways (like the M4 and M1 corridors).
Even though some roads have low battery levels, if there are plenty of chargers nearby, it's not a "desert." The real danger spots are where low battery meets a lack of infrastructure.

5. Why This Matters (According to the Paper)

The paper concludes that to fix these "deserts," we shouldn't just put chargers everywhere randomly. We need to focus on rapid charging hubs along motorways and on the boundaries of cities. This supports recent government shifts in policy to build fast chargers in these specific high-risk areas to help people on long-distance trips.

In a nutshell: The researchers built a "smart" computer simulation that learns like a human driver. By testing it against real data, they found the specific spots in the UK where electric drivers are most likely to run out of gas (battery) and get stuck, helping planners know exactly where to build the next charging stations.

Technical Summary: Replicating the Behaviour of Electric Vehicle Drivers Using an Agent-Based Reinforcement Learning Model

Problem Statement
The rapid expansion of electric vehicle (EV) adoption, particularly among the mass market lacking access to private home charging, has created a critical need for accurate public charging demand estimation. Previous approaches face significant limitations: data-driven methods often rely on biased proxies (e.g., conventional vehicle GPS or commercial fleets) due to privacy concerns regarding private EV data, while traditional simulation-based Agent-Based Models (ABMs) typically rely on static behavioural rules. These static rules fail to capture the adaptive nature of human drivers, who learn from experience and adjust their charging habits over time. Furthermore, existing Reinforcement Learning (RL) applications in transportation have predominantly focused on optimizing fleet operations or assuming perfect rationality and information sharing, which does not reflect the reality of private EV drivers operating under bounded rationality with limited communication. There is a lack of robust, validated frameworks that simulate the adaptive, heterogeneous decision-making of private drivers at a national scale.

Methodology
The authors propose a multi-stage, agent-based Reinforcement Learning (RL) framework using a Deep Q-Network (DQN) to simulate private EV driver behaviour across Great Britain (GB).

Data and Environment: The study utilizes the National Travel Survey (NTS) for trip schedules (origin, destination, purpose, distance) and Ordnance Survey data for road networks. EV attributes (battery size, consumption) are sourced from the EV Database. The simulation environment integrates 59,125 public chargers (as of April 2024) with real-time availability status scraped from ChargePoint.
Agent Clustering: Drivers are clustered based on trip distance, initial State of Charge (SOC), Trip Co-occurrence Density (TCD), and Charger Density (CD). Each cluster is further subdivided by trip purpose (work vs. leisure).
Multi-Stage Training-Simulation Framework:
1. Training: Representative agents from each cluster are trained using DQN. The agents observe a state vector $S_t$ comprising SOC, distance to destination, time of day, remaining trips, and the percentage of available stations within range.
2. Action Space: Agents choose between moving without charging or rerouting to a reachable station to charge a specific percentage ( $n\%$ ).
3. Reward Function: The reward ( $R_t$ ) is a composite of four factors: SOC threshold impact (psychological comfort), charging urgency (timing and opportunity), cost impact (financial and time penalties), and trip completion status (penalty for running out of charge).
4. Simulation Loop: Trained policies guide a larger population of agents in a shared environment where charger availability is dynamic. Agents interact by updating charger status (available/in-use) and potentially queuing, though queue lengths are not explicitly visible to agents, forcing them to estimate wait times based on experience.
5. Validation Strategy: Unlike standard RL studies that seek the episode with optimal performance, this model validates against 21 days of real-world charging session data. The authors identify the specific training episode where the simulated spatial and temporal charging patterns exhibit the highest correlation with observed reality. This "best-matching" episode represents a population-level blend of driver experience levels rather than a uniformly optimal outcome.

Key Results

Model Validity: The model successfully identified a training stage (specifically the 703rd episode) that closely aligned with real-world spatial and temporal charging patterns, achieving spatial correlations of 0.74–0.75 and temporal correlations of 0.65–0.70. This performance exceeded previous static ABM approaches.
Emergent Behaviours: The simulation captured intuitive driver behaviours: leisure drivers demonstrated more flexible charging patterns compared to commuters, and long-distance inter-city trips generated higher en-route charging demand. Agents learned to balance range anxiety with the costs of detours and waiting.
Charging Deserts: By analyzing the mean SOC distribution along road networks and cross-referencing with charger availability, the study identified "charging deserts"—locations with both low SOC and limited infrastructure. These high-risk areas were found primarily in Outer London, the eastern M4 corridor, Greater Manchester, and south Newcastle.
Infrastructure Insights: The analysis revealed that while some motorway corridors (e.g., M4, M1) frequently show low SOC, they possess sufficient charger density to mitigate actual depletion risks. Conversely, the identified "deserts" highlight gaps where infrastructure expansion is most critical.

Significance and Contributions
The paper claims to make three primary contributions:

Methodological Innovation: It introduces a novel framework that uses RL not to derive optimal strategies for rational agents, but to replicate the adaptive learning process of private drivers operating under bounded rationality. It bridges the gap between static simulation and data-driven limitations by validating against empirical data.
Scalability and Validation: The study demonstrates a scalable approach for simulating heterogeneous driver populations across a national-scale road network, overcoming computational challenges often associated with large-scale RL. The validation against real-world session data provides a level of credibility often missing in simulation studies.
Policy Relevance: The findings support recent policy shifts toward expanding rapid charging hubs along motorway corridors and city boundaries. By identifying specific high-risk locations where low SOC coincides with infrastructure scarcity, the model offers a validated tool for informing equitable and effective public charger deployment planning.

The authors acknowledge limitations, including the reliance on travel survey data that assumes existing travel patterns persist after EV adoption, the exclusion of home/workplace charger access in initial SOC calculations, and the current computational limit of simulating 1,000 agents simultaneously. However, the framework provides a robust foundation for future research into adaptive EV driver behaviour and infrastructure planning.