Adaptive traffic signal control optimization using a novel road partition and multi-channel state representation method

1. Problem Statement

Urban traffic congestion at intersections leads to reduced travel efficiency, increased carbon emissions, and higher energy consumption. While Deep Reinforcement Learning (DRL) offers a promising solution for adaptive signal control, existing methods face two critical limitations:

1. Rigid Road Partitioning: Traditional methods often use fixed-length cells to discretize road segments. This creates a mismatch between the training environment (where sensor ranges are flexible) and real-world applications (where sensors have specific, varying detection ranges). Consequently, models trained on one intersection often fail to generalize to others with different sensor configurations.
2. Insufficient State Representation: Many approaches rely on aggregated metrics (e.g., total queue length) or single-channel data, failing to capture the fine-grained spatial distribution and dynamic behaviors of vehicles (e.g., position-dependent speed and occupancy).

2. Methodology

The authors propose a novel framework that integrates Variable Cell Length (VCL) partitioning with a Multi-Channel State Representation, tested using Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO).

A. Novel Road Partition Method (Variable Cell Length)

Instead of fixed-length cells, the paper introduces a mathematical formula to dynamically adjust cell lengths based on the sensor's detection range ($d$) and a fixed number of cells ($n$).

• Formula: The length of the $x$-th cell, $f(x)$, is defined as the sum of a logarithmic and a linear function:
  $$f(x) = a \ln(x + 1) + bx$$
• Logic: Cells closer to the stop line are shorter (finer granularity) because vehicles near the intersection have a higher impact on signal timing. Cells further away are progressively longer.
• Adaptability: The coefficients $a$ and $b$ are calculated by solving a system of equations based on the first cell length ($l_1$) and the total detection range ($d$). This ensures that regardless of whether a sensor detects 300m or 500m, the road is always divided into the same number of cells ($n$), enabling cross-range transferability.

B. Multi-Channel State Representation

The traffic state is abstracted into a 3D tensor (Lanes $\times$ Cells $\times$ Channels), analogous to image processing (Height $\times$ Width $\times$ Channels).

• Three Channels: For each cell, the state vector includes:
  1. Number of vehicles (Count)
  2. Average speed
  3. Space occupancy (Ratio of vehicle length sum to cell length)
• Normalization: Data is normalized using historical maximum values to ensure model convergence across varying traffic flows.

C. Action and Reward Space

• Action Space: 8 discrete signal phases (e.g., East-West straight, North-South left turn, etc.). Selected phases are executed with a fixed green time (15 seconds in the study).
• Reward Function: A weighted sum of normalized key metrics:
  $$r(t) = -0.7 \times \text{waiting\_penalty} + 0.2 \times \text{speed\_reward} - 0.1 \times \text{fuel\_penalty}$$
  • Waiting time and fuel consumption are penalized (negative weights).
  • Speed is rewarded (positive weight).
  • Metrics are capped at typical maximums (200s wait, 20 m/s speed, 100mL fuel) to prevent outliers from dominating.

D. Algorithms

Two algorithms were implemented and compared:

1. DQN: Uses a convolutional neural network (CNN) to extract features from the state tensor, followed by fully connected layers to estimate Q-values.
2. PPO: Uses a similar CNN architecture but optimizes a parameterized policy directly using a clipped surrogate objective function to ensure stable updates.

3. Key Contributions

1. Adaptive Partitioning Formula: The introduction of a logarithmic-linear function that allows a trained AI model to be transferred directly to intersections with different sensor detection ranges (e.g., 300m to 500m) without retraining, provided the number of cells remains constant.
2. Multi-Channel State Encoding: Moving beyond simple queue lengths to a 3-channel representation (count, speed, occupancy) that provides a comprehensive view of traffic dynamics for the neural network.
3. Cross-Range Transferability: Demonstrated that a model trained on a specific detection range can effectively control an intersection with a different range, solving a major generalization gap in current DRL traffic control research.
4. Trajectory Generation Algorithm: A novel method to generate stochastic vehicle trajectories from sources to sinks to simulate realistic, varied traffic flows for training.

4. Experimental Results

The study utilized the SUMO microscopic traffic simulator coupled with TensorFlow and Python.

• Performance Comparison:
  • VCL-PPO (Variable Cell Length + PPO) achieved the best performance in reducing cumulative queue length and waiting time across various traffic volumes (500–3000 veh/h).
  • VCL-DQN performed the worst among the proposed methods, likely due to PPO's superior stability in handling the sequential nature of signal control and noisy reward signals.
  • Both VCL methods outperformed Fixed Cell Length (FCL) models and models without cell discretization.
• Transferability Evaluation:
  • A model trained on a 300m detection range was directly applied to a 500m detection range scenario.
  • The transferred model maintained superior performance compared to fixed-time control and performed comparably to a model specifically trained on the 500m range, validating the effectiveness of the variable cell partitioning.
• Convergence: Under stochastic traffic flow conditions, the PPO agent showed significant oscillations and required more episodes (~1000) to converge compared to fixed flow scenarios, highlighting the complexity of real-world traffic.

5. Significance

This research bridges the gap between theoretical DRL traffic control and practical deployment. By decoupling the learned policy from fixed physical scales through variable cell partitioning, the study offers a scalable and transferable solution for smart city infrastructure. It suggests that traffic signal control systems can be deployed across diverse intersections with varying sensor capabilities without the need for expensive, intersection-specific retraining, significantly lowering the barrier to entry for AI-driven traffic management.