Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control

1. Problem Statement

Traditional Traffic Signal Control (TSC) systems often prioritize vehicle-centric metrics (e.g., minimizing vehicle queue length or delay), which can overlook the needs of high-occupancy public transportation and pedestrians. In multimodal corridor networks, optimizing solely for private vehicles leads to inefficiencies in moving large numbers of people. Furthermore, existing Deep Reinforcement Learning (DRL) approaches for multi-intersection coordination often rely on standard graph representations (pairwise edges) that fail to capture complex, higher-order spatio-temporal dependencies across a network.

The paper addresses the challenge of designing a human-centric, multimodal TSC framework that:

• Prioritizes the movement of passengers (including those in buses, trams, and private cars) rather than just vehicles.
• Effectively models complex spatial and temporal interactions between multiple intersections in a corridor.
• Operates in a scalable, decentralized manner suitable for real-time deployment.

2. Methodology: STDSH-MARL

The authors propose STDSH-MARL (Spatio-Temporal Dual-Stage Hypergraph-based Multi-Agent Reinforcement Learning). The framework follows a Centralized Training and Decentralized Execution (CTDE) paradigm.

A. Core Architecture

• Agent Definition: Each intersection in the corridor is an independent agent.
• Training: Agents share a global critic that utilizes a Spatio-Temporal Hypergraph to evaluate joint actions.
• Execution: Agents act based solely on local observations using a shared policy network (decentralized).

B. Spatio-Temporal Dual-Stage Hypergraph (DSH)

Unlike standard graphs where edges connect only two nodes, the authors use a hypergraph where a single "hyperedge" can connect multiple nodes.

• Nodes ($V$): Represent intersections.
• Hyperedges ($E$):
  • Spatial Hyperedges ($E_s$): Connect all intersections at a specific time step to model spatial interdependencies.
  • Temporal Hyperedges ($E_t$): Connect a single intersection across multiple time steps to model temporal dynamics.
• Dual-Stage Hypergraph Attention (DSHA): A novel attention mechanism with two stages:
  1. Intra-hyperedge Attention: Weights the importance of nodes within a specific hyperedge (e.g., which intersections are most critical at time $t$).
  2. Inter-hyperedge Attention: Weights the importance of different hyperedges for a specific node (e.g., determining whether spatial or temporal context is more relevant for a specific intersection).
• Output: This mechanism produces a global hypergraph-level embedding fed into the centralized critic.

C. State and Action Space

• State Representation: Lane-level features including vehicle counts, passenger counts (mode-specific), queue lengths, and speeds for buses, trams, and private vehicles.
• Action Space: A hybrid discrete action space that jointly selects:
  1. Signal Phase: One of four configurations (e.g., Through/Left-turn NS, Protected Right-turn NS, etc.).
  2. Green Duration: An integer value from 8 to 45 seconds.
  • Total Action Space: $4 \text{ phases} \times 38 \text{ durations} = 152$ discrete actions.

D. Reward Mechanism

The reward function is human-centric, defined as the negative aggregate number of passengers experiencing delay across the network.

• It accounts for delays in private cars, buses, and trams.
• It balances local intersection delays ($w_1$) and global network delays ($w_2$).
• This shifts the optimization goal from "minimizing vehicle delay" to "minimizing passenger delay."

E. Learning Algorithm

The framework utilizes Proximal Policy Optimization (PPO) with parameter sharing among agents. The critic network is trained to minimize the value regression loss using the hypergraph embedding, while the actor network maximizes the clipped PPO objective.

3. Key Contributions

1. Human-Centric Objective: Introduction of a reward function based on the number of delayed passengers rather than vehicles, explicitly prioritizing high-occupancy modes (buses/trams).
2. Novel Network Representation: Development of a Spatio-Temporal Dual-Stage Hypergraph that captures higher-order dependencies beyond pairwise interactions, utilizing both spatial and temporal hyperedges.
3. Adaptive Hybrid Action Space: A unified action space that simultaneously determines the signal phase and the green duration, allowing for more flexible and adaptive timing decisions.
4. Scalable CTDE Framework: A multi-agent architecture that scales to large corridor networks by using decentralized execution while leveraging global hypergraph information during training.

4. Experimental Results

The model was tested in PTV VISSIM on a synthetic Australian corridor network with 6 intersections under five distinct traffic scenarios (off-peak, peak transition, rush hour, morning school peak, afternoon school peak).

Benchmarks: Compared against Fixed-Time (Webster), MADQN, MADDQN, MAA2C, MAPPO, and CMRM.

Key Findings:

• Passenger Delay (ANP): STDSH-MARL consistently achieved the lowest Average Number of Passengers (ANP) experiencing delay across all scenarios. For example, in Scenario 1, it reduced ANP by 18.65% compared to the standard MAPPO baseline.
• Public Transport Priority: The model significantly reduced waiting times for buses and trams compared to vehicle-centric baselines (MADQN/MADDQN), which often performed poorly for public transport despite good vehicle queue metrics.
• Robustness: The model maintained stable performance and low delay intensity even during high-demand rush hours and school peak periods, outperforming baselines in handling dynamic demand variations.
• Ablation Studies:
  • Removing the Temporal Hyperedges (THE) caused the most significant performance drop, identifying it as the most critical component.
  • The full DSHA mechanism and Hypergraph (HG) structure were essential for optimal performance.
  • The proposed method outperformed the standard PPO (MAPPO) by ~5-18% in passenger delay metrics, validating the efficacy of the hypergraph attention mechanism.

5. Significance and Future Work

Significance:
This paper bridges the gap between advanced deep learning techniques and equitable traffic management. By shifting the optimization metric from "vehicles" to "people," it offers a practical solution for cities aiming to prioritize public transport and reduce overall human congestion. The use of hypergraphs provides a more mathematically robust way to model the complex, non-linear interactions in traffic networks compared to traditional graph neural networks.

Future Work:

• Scalability: Testing on city-scale networks with hundreds of intersections to evaluate training time and memory usage.
• Robustness: Evaluating performance under realistic disruptions (accidents, sensor noise, sudden demand spikes).
• Explainability: Extending the framework to provide explainable AI (XAI) outputs to build trust with traffic operators for real-world deployment.