⚡ electrical engineering

Spatiotemporal Decision Transformer for Traffic Coordination

This paper introduces MADT, a novel Multi-Agent Decision Transformer that leverages graph attention and temporal modeling to reformulate traffic signal control as a sequence modeling problem, achieving state-of-the-art performance and improved multi-intersection coordination through offline learning from historical data.

Original authors: Haoran Su, Yandong Sun, Hanxiao Deng

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haoran Su, Yandong Sun, Hanxiao Deng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a city's traffic system as a massive, chaotic orchestra. In a traditional setup, every musician (traffic light) plays their own sheet music, ignoring the rest of the band. Sometimes, the violins (one intersection) play a loud crescendo while the drums (the next intersection) are silent, causing a jarring mess.

The paper introduces a new conductor called MADT (Multi-Agent Decision Transformer). Instead of teaching each musician to play alone, MADT teaches the whole orchestra how to read a single, shared story and play in perfect harmony.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Lone Wolf" Traffic Lights

Current traffic lights are like soloists. They look at the cars right in front of them and decide when to turn green. They don't really know what the light down the street is doing.

The Result: You get a "stop-and-go" wave. You hit a green light, drive 100 meters, and hit a red light because the next intersection didn't know you were coming. This wastes fuel, time, and patience.

2. The Solution: The "Storyteller" Approach

The authors realized that traffic isn't just about reacting to the now; it's about understanding a story that unfolds over time. They treated traffic control like a movie script.

The Script (Sequence Modeling): Instead of just looking at the current frame, the AI looks at the last 20 seconds of traffic history (the previous scenes) to predict what should happen next.
The Goal (Return-to-Go): Imagine you are directing a movie and you tell the actors, "I want this scene to be 10% faster than usual." The AI uses this "target speed" as a guide. It doesn't just try to do its best; it tries to hit a specific performance goal you set for it.

3. The Secret Sauce: The "Neighborhood Watch" (Graph Attention)

This is the paper's biggest innovation. In a city, not all intersections are equal. The light on 5th Avenue needs to talk to the light on 6th Avenue, but it doesn't need to worry about the light in a different city.

The Metaphor: Think of the road network as a neighborhood. MADT gives every traffic light a "neighborhood watch" system.
How it works: The AI uses a special tool called Graph Attention. It learns to pay extra attention to its immediate neighbors (the lights connected by the same road) and ignore distant ones.
The Analogy: If you are at a party, you don't shout your conversation to the whole room. You lean in and talk to the people standing right next to you. MADT does the same: it coordinates tightly with its direct neighbors to create a "Green Wave" (a continuous stream of green lights) while ignoring irrelevant distant lights.

4. How It Learns: The "History Book" Method

Most AI learns by trial and error, crashing into walls millions of times in a simulator. That takes forever and is dangerous.

MADT's Trick: It learns offline. It reads a massive "history book" of traffic data collected from the past. It studies how traffic moved under different conditions and learns the patterns without ever having to drive a car or wait in a real line.
The Result: It learns from a "good but not perfect" teacher (a standard traffic algorithm) and figures out how to be even better, essentially "studying harder" than the teacher did.

5. The Results: A Smoother Ride

The authors tested this on computer simulations of city grids and real-world maps of Atlanta and Boston.

The Win: MADT reduced the average time people spent driving by about 5-6% compared to the best existing methods.
The Analogy: If you drive 100 miles, that's like saving the time it takes to stop for a coffee.
The "Green Wave": It successfully coordinated lights so that cars could drive through multiple intersections without stopping, much like a surfer riding a perfect wave.
The "Spillback" Fix: It learned to stop cars from piling up and blocking the next intersection (a phenomenon called spillback), keeping the whole network flowing.

Summary

The paper claims that by treating traffic lights as a team of neighbors reading a shared story, rather than as isolated individuals, we can make cities move faster. MADT is a new type of AI that:

Reads the past (sequence modeling) to predict the future.
Talks to neighbors (graph attention) to coordinate timing.
Follows a target (return-to-go) to aim for specific efficiency goals.

The result is a traffic system that is less chaotic, moves more cars per hour, and gets people to their destinations faster, all by learning from historical data rather than crashing into problems in real-time.

Technical Summary: Spatiotemporal Decision Transformer for Multi-Agent Traffic Coordination (MADT)

1. Problem Statement

Traffic signal control is a critical challenge in urban transportation, requiring the coordination of multiple intersections to optimize network-wide flow. While Deep Reinforcement Learning (DRL) has shown promise, existing methods face three fundamental limitations when scaling to city-wide networks:

Coordination Complexity: Naive multi-agent formulations struggle with the exponential growth of the joint action space and lack effective mechanisms for inter-intersection information sharing (e.g., "green waves" and spillback prevention).
Sample Efficiency: Online RL methods require millions of simulator interactions, limiting practical deployment and introducing exploration risks.
Structural Mismatch: Existing sequence modeling approaches (like the Decision Transformer) were designed for single-agent settings and do not naturally extend to multi-agent coordination where spatial topology (road networks) dictates interaction patterns.

The paper posits that reformulating multi-agent traffic signal control as a conditional sequence modeling problem can address these issues by leveraging offline historical data, explicit spatial modeling, and goal-directed conditioning.

2. Methodology: MADT Architecture

The authors introduce MADT (Multi-Agent Decision Transformer), which extends the Decision Transformer paradigm to multi-agent settings by integrating three core components:

A. Problem Formulation

The task is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP).

Observations: Include queue lengths, average waiting times, current signal phases, time since last phase change, and neighbor queue information.
Actions: Selection of traffic signal phases (e.g., 4-phase or 8-phase configurations).
Reward: A shared reward based on the negative average waiting time across the network, capturing both local efficiency and global throughput.

B. Architectural Components

MADT processes an interleaved sequence of return-to-go targets, observations, and actions using a unified architecture:

Observation Encoding: Traffic observations for each agent are encoded via MLPs and augmented with learnable agent embeddings and positional encodings.
Spatial Graph Attention: Unlike prior work that treats agents symmetrically or uses arbitrary ordering, MADT employs Graph Attention Networks (GAT). This mechanism models the road network topology explicitly, allowing each intersection to attend to its spatial neighbors with learned weights. This ensures permutation equivariance, meaning coordination patterns emerge from road connectivity rather than arbitrary agent indexing.
Temporal Transformer Encoding: A causal Transformer encoder processes the sequence of states, actions, and returns, capturing long-horizon temporal dependencies (e.g., traffic progression over multiple signal cycles).
Return-to-Go Conditioning: The model is conditioned on a target "return-to-go" ( $\hat{R}_t$ ), representing the desired cumulative future reward. This allows the model to be steered toward specific performance targets (e.g., minimizing travel time) at test time without retraining.
Parallel Action Generation: Unlike autoregressive multi-agent transformers that generate actions in a fixed order, MADT generates joint actions for all intersections in parallel after spatial message passing, ensuring computational efficiency and structural consistency.

C. Training and Inference

Offline Learning: The model is trained via supervised learning on offline trajectories collected from a traffic simulator (CityFlow) using a MaxPressure policy. No online environment interaction is required during training.
Inference: At test time, a target return is specified. The model autoregressively predicts actions conditioned on the current state and the target return, updating the return-to-go based on observed rewards.

3. Key Contributions

The paper outlines four primary contributions:

Graph-Structured Spatial Attention: The introduction of explicit graph attention layers to model road network topology, enabling intersections to coordinate based on spatial proximity rather than arbitrary indexing.
Unified Spatiotemporal Architecture: A novel integration of graph attention networks within the Decision Transformer framework, jointly capturing spatial dependencies (who coordinates with whom) and temporal dynamics (when to change phases).
Return-to-Go Conditioning for Network Optimization: A mechanism to specify desired network-level performance targets at test time, allowing flexible trade-offs between efficiency and other objectives.
Comprehensive Evaluation: Extensive validation on synthetic grid networks and real-world scenarios (Atlanta, GA and Boston, MA), demonstrating state-of-the-art performance and robust ablation studies.

4. Experimental Results

Experiments were conducted on four environments: Grid 3×3, Grid 4×4, Atlanta (16 intersections), and Boston (15 intersections). MADT was compared against traditional methods (FixedTime, MaxPressure), online RL baselines (FRAP, MPLight, MAPPO, CoLight), and sequence modeling baselines (MAT, IndependentDT).

Performance Metrics

Average Travel Time (ATT): MADT achieved the lowest ATT in all environments, reducing travel time by 5.3% to 5.9% compared to the strongest baseline (CoLight).
- Example: On the Atlanta network, ATT decreased from 478.5s (CoLight) to 452.1s (MADT).
Throughput: MADT increased network throughput by 5.5% to 6.2% compared to CoLight.
Coordination Index: MADT achieved a coordination index of 0.78 (vs. 0.52 for CoLight), a 50% relative improvement, indicating superior phase synchronization between adjacent intersections.
Offline Learning Efficiency: MADT significantly outperformed the MaxPressure policy used to collect the training data (e.g., 14.8% improvement on Grid 3×3), demonstrating the ability to extract better policies from suboptimal demonstrations.

Ablation Studies

Graph Attention: Removing graph attention increased ATT by 8.4% on average, confirming its critical role in spatial coordination.
Return Conditioning: Removing return-to-go conditioning increased ATT by 5.2%, validating the benefit of goal-directed control.
Combined Effect: Removing both components resulted in a 13.4% performance drop, suggesting a super-additive relationship where spatial coordination and goal conditioning are complementary.

Robustness and Scalability

Congestion Sensitivity: Improvements were most pronounced under high demand (8.1% improvement at 400 veh/h/lane), indicating the method is most valuable when coordination is critical.
Inference Latency: Inference time was <50ms, well within real-time control requirements (<1 second).
Data Quality: The model degraded gracefully when trained on lower-quality (random) data, maintaining reasonable performance.

5. Significance and Claims

The paper claims that MADT represents a significant advancement in multi-agent traffic control by successfully bridging the gap between sequence modeling and spatial coordination.

Paradigm Shift: It demonstrates that multi-agent coordination can be effectively cast as a supervised sequence modeling problem, inheriting the training stability of supervised learning while avoiding the sample inefficiency and exploration risks of online RL.
Structural Integrity: By respecting the road network topology through graph attention, MADT overcomes the limitations of methods that treat agents as independent or symmetrically ordered, proving that spatial structure is essential for effective traffic coordination.
Practical Viability: The ability to learn from offline historical data (without simulator interaction) and achieve real-time inference makes the approach viable for deployment in existing traffic management systems.
Emergent Behaviors: The analysis reveals that MADT learns interpretable traffic engineering strategies—such as "green wave" formation, spillback prevention, and demand-responsive timing—without explicit programming, suggesting that the model discovers meaningful coordination strategies directly from data.

The authors conclude that while challenges remain regarding scalability to hundreds of intersections and distribution shifts in real-world deployment, MADT provides a robust foundation for future research at the intersection of sequence modeling and multi-agent reinforcement learning.