Arterial Network Traffic State Prediction with Connected Vehicle Data: An Abnormality-Aware Spatiotemporal Network

This paper proposes a novel framework for predicting arterial traffic states using real-world connected vehicle data, featuring a two-stage traffic state extraction method and an Abnormality-aware Spatiotemporal Graph Convolution Network (AASTGCN) that effectively handles both normal and abnormal traffic conditions by separately modeling them with a dual-expert architecture and a gated-fusion mechanism.

The Gist

Imagine a massive, bustling city as a giant, living organism. The arterial roads (the main streets) are its veins, carrying the blood (traffic) that keeps the city alive. For a long time, city planners tried to monitor these veins using fixed sensors like loop detectors buried in the pavement. But this is like trying to understand the health of a whole body by only checking the temperature of a few specific spots on the skin. It's expensive, limited, and often misses the bigger picture.

Enter Connected Vehicles (CVs). These are modern cars equipped with GPS and communication tech. They act like millions of tiny, mobile doctors floating through the city, constantly reporting their own health (speed, location, stops).

This paper introduces a new "super-doctor" system called AASTGCN that uses data from these cars to predict traffic jams and delays across an entire city-scale network. Here is how it works, broken down simply:

1. The Problem: The "Normal" vs. The "Chaos"

Most traffic prediction systems are like students who only study for a standard math test. They are great at predicting what happens on a normal Tuesday morning (the "normal" pattern). But when a crash happens, a hurricane evacuation starts, or a parade blocks the road (the "abnormal" chaos), these systems get confused. They try to force the chaotic data into their "normal" patterns, leading to bad predictions exactly when you need them most.

2. The Solution: The "Dual-Expert" Team

The authors built a system with two specialized experts working together, like a medical team with a General Practitioner and a Trauma Surgeon.

• The General Practitioner (Normal Expert): This expert studies the "routine." It looks at historical data to see that "every Monday at 8 AM, traffic is heavy." It uses this long-term memory to predict the future when things are running smoothly.
• The Trauma Surgeon (Abnormal Expert): This expert is on high alert for emergencies. It is trained specifically to recognize when something is wrong—when traffic suddenly stops or slows down in a way that doesn't fit the usual pattern.

The Magic Trick: Instead of mixing these two experts together, the system keeps them separate. It first checks: "Is this a normal Tuesday, or is there a crash?"

• If it's normal, it listens mostly to the General Practitioner and the long-term history.
• If it's abnormal, it switches to the Trauma Surgeon, who ignores the "usual" patterns and focuses entirely on the real-time chaos to make a quick, accurate guess.

3. The "Gatekeeper" Mechanism

How does the system decide which expert to listen to? Imagine a smart gatekeeper standing at a crossroads.

• When the weather is calm, the gatekeeper opens the door wide to the "History Library," letting the system use past patterns to predict the future.
• When a storm hits (an accident), the gatekeeper slams the library door shut and opens the "Live News Feed" wide, forcing the system to focus only on what is happening right now.

This "gate" automatically adjusts itself, ensuring the system uses the right tool for the job.

4. The Scale: A City-Scale Experiment

To prove this works, the researchers didn't just test it on a tiny neighborhood. They tested it on a giant network of 1,050 road links (covering nearly 400 miles) in Central Florida.

• They used real data from over 400,000 daily car journeys.
• They had to deal with a "low penetration" problem: Only about 2-3% of the cars on the road were connected. It's like trying to predict the weather with only a few weather balloons.
• The Result: Their system (AASTGCN) was significantly better than all other existing methods. It predicted delays and queue lengths (how long the line of cars is) with much higher accuracy, especially during those rare, chaotic moments when other systems failed.

Why This Matters

Think of traffic management like conducting an orchestra.

• Old methods were like a conductor who only knew the sheet music for a calm symphony. When a drummer started playing jazz during a classical piece, the conductor got lost, and the music fell apart.
• This new system is a conductor who can instantly switch styles. It knows the symphony by heart, but the moment the jazz drum starts, it knows exactly how to adapt the orchestra to keep the music (traffic) flowing smoothly.

In short: This paper gives cities a smarter way to "see" traffic. By separating the "boring routine" from the "chaotic emergencies" and using a special "gate" to switch between them, it can predict traffic jams and delays more accurately than ever before, even when things go wrong.

Technical Summary

Here is a detailed technical summary of the paper "Arterial Network Traffic State Prediction with Connected Vehicle Data: An Abnormality-Aware Spatiotemporal Network."

1. Problem Statement

Urban arterial networks are critical for transportation systems, yet reliable real-time estimation and prediction of traffic states (specifically travel delay and queue length) remain challenging.

• Data Limitations: Traditional fixed sensors (loop detectors, cameras) are expensive and limited to small networks or specific road types. While Connected Vehicle (CV) data offers high-resolution, city-scale coverage, existing studies relying on CV data are often limited to simulated high-penetration scenarios (10–50%) or small network segments. There is a lack of methods validated on real-world, city-scale arterial networks with low CV penetration rates (typically 2–3%).
• Modeling Limitations:
  1. Periodicity: Most deep learning models focus on short-term dynamics, failing to effectively capture long-term temporal periodicities (e.g., recurring weekly patterns).
  2. Abnormal Events: Existing models typically train on mixed data (normal and abnormal traffic). Since abnormal events (crashes, non-recurrent congestion) are rare, models become biased toward normal patterns, resulting in poor robustness and accuracy when predicting critical abnormal conditions.

2. Methodology

The authors propose a comprehensive framework consisting of two main components: a Two-Stage Traffic State Extraction Method and an Abnormality-Aware Spatiotemporal Graph Convolution Network (AASTGCN).

A. Two-Stage Traffic State Extraction

To handle low-penetration real-world CV data, the authors developed a pipeline to convert raw vehicle trajectories into network-level traffic measures:

1. Data Processing: Raw CV trajectories are matched to arterial links using a three-step spatial filtering process (geofence, heading check, trajectory validation) and projected onto link-based distances.
2. Vehicle-Level Estimation: Individual vehicle trajectories are segmented into states (free-flow, transition, stop) to calculate metrics like control delay, stop delay, and queue length.
3. Link-Level Aggregation: Vehicle-level metrics are aggregated over 15-minute time windows to generate link-level averages. This window size balances the need for sufficient sample sizes (given ~2–3% penetration) with temporal granularity.

B. Abnormality-Aware Spatiotemporal Graph Convolution Network (AASTGCN)

The core prediction model is designed to handle both normal and abnormal traffic dynamics separately.

• Abnormal Traffic Identification: Before prediction, a data-driven, median-based rule identifies abnormal conditions in real-time. It compares current observations against historical distributions (same time/day of week) using Median Absolute Deviation (MAD). If the current value exceeds a threshold ($\tilde{x}_t = m_t + k\sigma_t$), it is flagged as abnormal.
• Dual-Expert Architecture: The model employs two separate "expert" networks:
  • Normal Expert: Trained on normal traffic data.
  • Abnormal Expert: Trained exclusively on identified abnormal data.
  • Masking: Inputs are masked based on the abnormality flag, routing data to the appropriate expert.
• Spatiotemporal WaveNet Structure: Each expert utilizes a Spatiotemporal Graph Convolution Network with:
  • Embedding Layer: Encodes heterogeneous inputs (dynamic traffic, temporal attributes, static road features).
  • Two Spatiotemporal Branches:
    1. Real-time Branch: Captures short-term dynamics using the past 4 time steps (1 hour).
    2. Historical Branch: Captures long-term periodicity using historical averages from the same time/day of the week.
  • Gated Fusion Mechanism: A learnable gate adaptively balances the contributions of real-time and historical branches.
    • Normal Traffic: The gate favors historical periodic patterns.
    • Abnormal Traffic: The gate shifts weight toward real-time dynamics to capture abrupt deviations.
  • Graph Convolution: Uses both a fixed adjacency matrix (topology-based) and a learnable adaptive adjacency matrix to model spatial dependencies between links.
• Loss Function: A weighted combination of Smooth L1 losses for both experts, with a higher weight assigned to the abnormal expert to ensure the model learns rare events effectively.

3. Key Contributions

1. Real-World Validation at Scale: The framework is validated on a massive, real-world dataset covering 1,050 arterial links (386.4 miles) in Orlando, Florida, using real CV data with a low penetration rate (~2.2–3.6%).
2. Dual-Expert Architecture: The novel separation of normal and abnormal traffic into dedicated experts significantly improves prediction robustness for rare, critical events without compromising normal traffic accuracy.
3. Adaptive Gated Fusion: The mechanism dynamically adjusts the reliance on historical periodicity vs. real-time dynamics based on traffic conditions, effectively handling both stable and disrupted traffic states.
4. Robust State Extraction: A two-stage method that successfully aggregates low-penetration CV data into reliable link-level delay and queue length estimates.

4. Experimental Results

• Dataset: 45 days of CV data (May and October 2024) from Orange and Seminole counties.
• Baselines: Compared against state-of-the-art models including LSTM, GCN, DCRNN, STGCN, Graph WaveNet, ASTGCN, Conv-LSTM, iTransformer, SATEformer, and STABC.
• Performance:
  • Overall: AASTGCN achieved the lowest Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) for both delay and queue length.
  • Improvements: Compared to the best baseline, AASTGCN reduced MAE by 20.1% (delay) and 15.6% (queue) overall.
  • Abnormal Conditions: The improvement was most significant for abnormal traffic, where the model reduced MAE by 6.6% (delay) and 12.2% (queue) compared to baselines that failed to distinguish abnormal patterns.
• Ablation Studies:
  • Removing the Abnormal Expert led to significant performance drops in abnormal scenarios.
  • Removing Historical Features hurt normal traffic prediction but had less impact on abnormal traffic.
  • Removing the Gated Fusion layer resulted in higher errors across all conditions, confirming the need for adaptive weighting.
  • The model showed robustness to the abnormal detection threshold ($k$), with conservative thresholds ($k \in [1, 2]$) yielding the best results.

5. Significance

This research bridges the gap between theoretical CV-based traffic prediction and real-world application. By demonstrating that high-resolution traffic state estimation and prediction are feasible on city-scale arterial networks with low CV penetration, the study provides a viable solution for urban traffic management. The Abnormality-Aware approach is particularly significant for traffic operations, as it ensures that critical incidents (crashes, evacuations) are detected and predicted accurately, enabling faster response times and better traffic control strategies. The proposed framework can be directly implemented in real-time traffic management systems to improve urban mobility and safety.