💻 computer science

Adaptive RL-Based Traffic Signal Control with Dynamic Graph Attention and Temporal Awareness under Disruptions

This paper proposes a robust Spatio-Temporal Graph Reinforcement Learning framework that integrates Graph Attention Networks and Transformer-based temporal encoders to enable multi-agent traffic signal control systems to effectively distinguish between transient noise and persistent disruptions, thereby achieving superior throughput and faster recovery from structural failures compared to state-of-the-art baselines.

Original authors: Wissem Yousfi, Rim Slama, Pierre-Antoine Laharotte

Published 2026-10-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Wissem Yousfi, Rim Slama, Pierre-Antoine Laharotte

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Traffic lights are the silent conductors of our cities, orchestrating the flow of millions of vehicles every day. For decades, engineers have relied on fixed schedules or simple sensors to decide when a light should turn green or red. These systems work well when traffic is predictable, but they often struggle when the unexpected happens. A sudden accident, a road closure for construction, or even a heavy rainstorm can throw a carefully timed system into chaos, causing queues to spill back and gridlock to spread. In recent years, researchers have turned to artificial intelligence, specifically a type of learning called reinforcement learning, to teach traffic lights how to adapt in real time. By letting computer agents learn through trial and error in simulated cities, these systems can often manage normal traffic better than traditional methods. However, a critical gap remains: most of these intelligent systems are trained in perfect, idealized worlds. When the simulation is disrupted by a major failure, the artificial intelligence often panics or freezes, unable to recover as quickly as a human operator might hope.

This is the challenge that a team of researchers from the University Gustave Eiffel in France set out to solve. They asked a fundamental question: how can we build a traffic control system that not only handles busy hours but also survives and recovers from structural disasters? Their answer is a new kind of smart traffic controller designed to be resilient. Instead of just reacting to the traffic it sees right now, this system looks at how traffic has been changing over time and pays close attention to which neighboring intersections are still useful to talk to. The researchers built a sophisticated digital twin of a city grid to test their idea, subjecting it to a series of severe stress tests that included random road blockages, the total closure of major highways, and even simulated rainy weather that slows down every car on the road.

The core of their innovation lies in how the traffic lights "think" about their neighbors. In a normal city, every intersection is connected to the ones around it, and they share information to coordinate the flow. But when a road is blocked, the old connections become useless. A traffic light might keep listening to a neighbor that is now cut off from the main flow, wasting time on information that no longer matters. The new system uses a dynamic attention mechanism, which acts like a filter that constantly re-evaluates who is worth listening to. If a road leading to a neighbor is blocked, the system instantly stops relying on that neighbor's data and shifts its focus to other intersections that are still carrying traffic. This allows the network to reorganize itself on the fly, finding new paths for cars to flow through without waiting for a central command to tell it what to do.

To make this even more robust, the system also learns to distinguish between a temporary glitch and a permanent problem. Traffic naturally fluctuates; a bus might arrive late, or a group of cars might bunch up after a green light. A simple system might mistake this for a major jam and overreact. The researchers' design includes a memory component that watches the traffic over a short history. It can tell the difference between a sudden, fleeting spike in cars and a sustained blockage that won't go away. This prevents the lights from making frantic, unnecessary changes during minor hiccups while ensuring they react decisively when a real disaster strikes. By combining this ability to ignore noise with its skill at ignoring broken neighbors, the system creates a control strategy that is both calm and agile.

The researchers tested their creation against the best existing methods, including other advanced artificial intelligence controllers and traditional fixed-time lights. They ran thousands of simulations on a 4x4 grid of intersections, first under normal, high-demand conditions and then under various forms of chaos. In the normal scenarios, their system performed slightly better than its competitors, keeping vehicles moving faster and reducing the time cars spent waiting in line. But the real test came when they introduced the disruptions. When they randomly blocked eight different roads at once, or when they completely shut down a major east-west corridor, the other systems struggled significantly. Their waiting times skyrocketed, and queues grew long and unmanageable. The new system, however, held its ground. It maintained a higher average speed and kept the total waiting time lower than the other methods, even as the network was being torn apart by the simulated accidents.

Perhaps most importantly, the study measured how quickly the system could recover once the disruption ended. When the blocked roads were cleared, the new controller returned to a smooth, stable flow much faster than the others. In one scenario involving a closure of the city center, the system recovered its average speed nearly two minutes faster than the next best competitor. This speed of recovery is crucial for real-world cities, where the time spent in gridlock after an incident can be just as damaging as the incident itself. The researchers also tested the system under simulated rainy conditions, where cars drive slower and keep larger distances from one another. While the weather model was simplified, the system adapted well, showing that it could handle changes in how vehicles behave, not just changes in the road layout.

The findings suggest that the key to resilient traffic control is not just making the system smarter, but making it more aware of its own context. By teaching the traffic lights to dynamically decide who to listen to and to understand the history of the traffic flow, the researchers have created a system that behaves more like a seasoned human dispatcher than a rigid computer program. It does not break when the rules of the road change; it simply rewrites its own strategy to fit the new reality. While these results come from computer simulations and the researchers note that real-world testing is the next necessary step, the work offers a promising path forward for building cities that can withstand the inevitable disruptions of modern life. The goal is no longer just to move traffic efficiently when everything is perfect, but to keep the city moving when everything goes wrong.

Technical Summary: Adaptive RL-Based Traffic Signal Control with Dynamic Graph Attention and Temporal Awareness under Disruptions

Problem Statement
While Multi-Agent Reinforcement Learning (MARL) has demonstrated potential in optimizing Traffic Signal Control (TSC) under nominal conditions, existing approaches often lack resilience when faced with non-stationary, abnormal traffic conditions. Models trained on stationary demand and fixed network capacities tend to develop brittle policies that degrade sharply during disruptions such as sensor failures, road blockages, or extreme weather. The core challenge lies in three simultaneous requirements: distinguishing between transient noise and persistent capacity reductions, determining the optimal timing for reaction to avoid unstable phase switching, and dynamically identifying which neighboring agents provide relevant information as the network topology changes due to disruptions. Current graph-based methods often rely on static neighbor rankings or instantaneous spatial snapshots that fail to separate transient fluctuations from structural failures.

Methodology
The authors propose a Dynamic Spatio-Temporal Graph Reinforcement Learning framework designed to maintain structural stability and adaptability. The architecture integrates four key components within a Partially Observable Markov Decision Process (POMDP) setting:

Observation Embedding: Local observations (active phase, transition eligibility, lane density, queue saturation) are projected into a latent space using a Multi-Layer Perceptron (MLP).
Dynamic Spatial Collaboration (GATv2): Unlike static graph attention mechanisms, the framework employs GATv2 (Brody et al., 2022). This allows agents to compute context-dependent attention weights based on the joint state of the agent and its neighbors. Consequently, the functional neighborhood adapts online; agents can down-weight neighbors whose upstream supply is throttled by a disruption and up-weight those absorbing rerouted demand, without altering the physical graph.
Temporal Interaction Encoding (Transformer): To distinguish between stochastic arrival bursts and persistent capacity losses, a Transformer-based encoder processes a finite history of spatial representations. This mechanism allows the policy to identify "persistence signatures," enabling agents to ignore short-lived fluctuations and commit to adaptive behaviors only when deviations persist.
Control Algorithm: The framework utilizes Proximal Policy Optimization (PPO) under a Centralized Training with Decentralized Execution (CTDE) paradigm. The actor receives a concatenated state of the current local observation and the spatio-temporal representation, while the critic utilizes global information during training.

Experimental Setup
The evaluation was conducted on two networks: a controlled synthetic $4 \times 4$ grid and a real-world-derived Hangzhou $4 \times 4$ benchmark.

Stochastic Training: A probabilistic traffic generation engine was introduced to create unique, non-stationary demand patterns for each episode, preventing overfitting to specific arrival sequences.
Perturbation Scenarios: A rigorous stress-test benchmark was developed, including:
- Random Road Blockages: High-frequency, stochastic edge restrictions.
- Realistic Structural Failures: West-East and South-North corridor failures, and a Central Square closure.
- Adverse Weather: A simplified model reducing vehicle speed and increasing minimum gaps to simulate wet road conditions.
Baselines: The proposed method was compared against Fixed-Time (FT), Independent DQN (IDQN), Independent PPO (IPPO), Multi-Agent PPO (MAPPO), and MacLight (a state-of-the-art MARL approach specifically evaluated for disruptions).

Key Results

Nominal Performance: On both the synthetic and Hangzhou networks, the proposed method achieved the best mean performance across Total Waiting Time, Queue Length, and Average Speed. On the Hangzhou network, it reduced mean waiting time by approximately 26.6% compared to MacLight.
Robustness (During Disruption): Under active perturbations, the proposed controller consistently maintained lower absolute waiting times and queue lengths than MacLight. In terms of Performance Degradation Rate (PDR), the proposed method showed superior relative robustness in scenarios involving severe topological changes (e.g., Random Blockage and Center Closure), limiting queue length increases more effectively than the baseline.
Resilience (Recovery): The framework demonstrated faster recovery times ( $T_{rec}$ ) and higher Area Under the Curve (AUC) ratios in most structural failure scenarios. Notably, in the Center Square Closure scenario, the proposed method recovered queue length 95 seconds faster than MacLight.
Weather Sensitivity: Under the simplified rainy weather scenario, both controllers showed reduced average speeds due to the imposed parameter changes (lower speed, larger gaps). The authors note that the negative PDR values in this specific scenario reflect the simplified weather model rather than inherent robustness to rain.

Significance and Claims
The paper claims that the proposed framework addresses the gap between simulation-based success and real-world viability by explicitly designing for structural stability. The primary significance lies in the dual mechanism of dynamic neighbor weighting and temporal persistence detection, which allows agents to adapt their coordination patterns and reaction timing online without retraining or external incident detectors.

The authors emphasize that their approach achieves superior elasticity under stress, recovering stable traffic flow significantly faster than state-of-the-art baselines following severe structural disruptions. However, the paper maintains a modest tone regarding its claims:

It acknowledges that improvements in nominal conditions, while statistically best in mean, were accompanied by substantial variability.
It clarifies that the weather results are sensitivity analyses based on a simplified model, not evidence of robustness to complex physical rainfall effects.
It concludes that the findings are scenario-dependent empirical results rather than proof that the architecture inherently guarantees resilience in all possible conditions.

Future work is identified as extending evaluations to larger, more heterogeneous networks and calibrating weather-related vehicle behaviors using observed traffic data.

More like this