🤖 machine learning

STAGformer: A Spatio-temporal Agent Graph Transformer for Micro Mobility Demand Forecasting

This paper introduces STAGformer, a Spatio-Temporal Agent Graph Transformer that utilizes a novel two-step agent attention mechanism to achieve efficient global modeling with linear complexity, significantly outperforming state-of-the-art baselines in forecasting bike-sharing demand across complex urban networks.

Original authors: Ye Zihao

Published 2026-07-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ye Zihao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a city's bike-sharing system as a giant, living organism with thousands of "organs" (the bike stations). Every day, these organs breathe in and out: people take bikes from some stations (exhale) and drop them off at others (inhale). The challenge for city planners is predicting exactly where and when these "breaths" will happen so they can move bikes around before the system gets clogged or empty.

The paper introduces STAGformer, a new computer brain designed to solve this prediction puzzle. Here is how it works, broken down into simple concepts:

The Problem: Too Much Noise, Too Much Data

Predicting bike demand is hard because it depends on two things happening at once:

Time: Bikes are used differently at 8:00 AM (rush hour) than at 2:00 PM.
Space: A station in a busy office district behaves differently than one in a quiet park.

Older computer models tried to look at every single station and every single minute simultaneously. Imagine trying to understand a conversation by having every person in a stadium of 10,000 people talk to every other person at the same time. It's too loud, too slow, and the computer gets overwhelmed (this is the "quadratic complexity" problem the paper mentions).

The Solution: The "Agent" System

STAGformer solves this by introducing a clever shortcut called Agent Attention.

Think of the city's bike stations as a massive classroom of students.

The Old Way: To figure out the class mood, you ask every single student to talk to every other student. It takes forever.
The STAGformer Way: You pick a small group of "Class Agents" (say, 32 students).
1. Aggregation: First, all the other students whisper their news to these 32 Agents. The Agents gather the "global" picture of what's happening in the whole room.
2. Broadcasting: Then, the Agents shout the summary back to the whole class.

This allows the computer to understand the big picture (global trends) without needing every single station to talk to every other station directly. It's like having a news anchor summarize the day's events instead of reading every single newspaper in the city. This makes the process linear (fast and efficient) rather than quadratic (slow and heavy).

The Four Pillars of STAGformer

The model isn't just about the Agents; it uses four specific tools to get the job done:

The Context Reader (Encoder): Before making a prediction, it reads the "mood" of the city. It looks at the weather, the time of day, and what's nearby (like parks, schools, or restaurants). It knows that a sunny day near a park means more bikes will be rented.
The Neighborhood Watch (Graph Propagation): It looks at immediate neighbors. If Station A is next to Station B, and Station A is empty, it knows Station B might get crowded soon. It learns from the physical map of the city.
The Pattern Spotter (Temporal Convolution): It looks at the recent past for each specific station. It notices, "Oh, this station always gets busy 15 minutes after the subway opens." It catches short-term, local habits.
The Global Agent (Agent Attention): This is the star. It connects the dots between distant places. It realizes that when people leave the business district in the morning, they are likely heading to the residential suburbs, even if those suburbs are miles away.

The Results: Faster and Smarter

The authors tested this system on real data from New York City and Chicago.

Accuracy: STAGformer was better at predicting bike demand than all the other top models they compared it against. It made fewer mistakes in guessing how many bikes would be rented or returned.
Speed: Because of the "Agent" shortcut, it didn't get bogged down by the sheer size of the city. It could handle the massive number of stations without slowing down.

Why It Matters

The paper concludes that by combining local neighborhood checks (Graph), short-term habit spotting (Time Convolution), and global trend summarizing (Agent Attention), STAGformer creates a system that is both highly accurate and fast enough to run on real, large-scale city networks. It helps city planners keep bikes where people need them, ensuring the system runs smoothly without wasting time or money.

Technical Summary: STAGformer for Micro-Mobility Demand Forecasting

1. Problem Statement

Accurate station-level demand forecasting is critical for the efficient operation of bike-sharing systems, particularly for vehicle rebalancing and operational optimization. However, this task faces two primary challenges in large-scale urban networks:

Complex Spatio-Temporal Dependencies: Demand is influenced by periodic fluctuations, road network structures, Points of Interest (POIs), and real-time weather, creating intricate local and global interactions.
Computational Scalability: Existing deep learning approaches, particularly those utilizing standard self-attention mechanisms (Transformers), suffer from quadratic computational complexity $O((NT)^2)$ with respect to the number of stations ( $N$ ) and time steps ( $T$ ). This renders them impractical for city-scale systems. Furthermore, many models fail to effectively integrate multi-source external factors (e.g., weather, POIs), limiting prediction accuracy.

2. Methodology: STAGformer

The paper proposes STAGformer (Spatio-Temporal Agent Graph Transformer), a model designed to achieve efficient global modeling with linear computational complexity while maintaining high predictive accuracy. The architecture integrates four core modules:

A. Spatio-Temporal Feature Encoder

This module fuses dynamic node features (historical rental/return counts) with global external factors (weather, time, POIs). It employs linear projections to map inputs to a hidden dimension, concatenates them, and injects learnable temporal and spatial positional encodings to preserve order information.

B. Graph Propagation Module

To explicitly model local spatial dependencies, this module aggregates information from spatial neighbors. It utilizes an external adjacency matrix (based on road network distances) or an adaptively learned graph structure. Through $K$ -step propagation, it captures higher-order spatial influences, fusing them via learnable weights.

C. Temporal Convolution Module

This lightweight module captures local temporal patterns for each station. It applies 1D convolutions (potentially depthwise separable) along the time dimension, enhanced by residual connections, batch normalization, and GELU activation, to extract short-term variation patterns before global modeling.

D. Agent Attention Module (Core Innovation)

To address the quadratic complexity of standard self-attention, STAGformer introduces a two-step agent attention mechanism with separate spatial and temporal branches:

Agent Tokens: A small set of learnable "agent" tokens ( $n_s$ for spatial, $n_t$ for temporal, where $n \ll N, T$ ) are introduced.
Aggregation: Agent tokens act as queries to aggregate global information from all nodes/time steps into a compressed context.
Broadcasting: The original nodes attend to these agent tokens to receive the aggregated global context.
Residual Diversity: A Depthwise Convolution (DWC) branch is added to the spatial attention output to recover feature diversity potentially lost during the compression step.

This mechanism reduces the computational complexity from $O((NT)^2)$ to $O(NT)$, enabling the modeling of long-range spatio-temporal dependencies without the prohibitive cost of standard attention.

3. Key Contributions

The paper outlines three main contributions:

Novel Encoding Scheme: A design that fuses dynamic node features with global external factors, enhancing both global and local pattern extraction through graph propagation and temporal convolution.
STAGformer Architecture: The introduction of the agent attention mechanism to bike-sharing demand forecasting, achieving linear computational complexity while preserving global modeling capabilities.
Empirical Validation: Extensive experiments demonstrating that STAGformer outperforms state-of-the-art baselines in accuracy (RMSE and MAE) and offers superior computational efficiency.

4. Experimental Results

The model was evaluated on two real-world datasets: NYC Citi-Bike and Chicago Divvy-Bike.

Performance: STAGformer consistently outperformed nine baseline methods, including Linear Regression, GRU, Transformer, GAT, and specialized models like BikeMAN and STAEformer. It achieved the lowest RMSE and MAE across multiple prediction horizons (e.g., on NYC data, RMSE of 0.4874 vs. 0.5106 for the next best model).
Ablation Studies: Removing any core module degraded performance.
- Removing Agent Attention caused the most significant drop (e.g., +37.9% RMSE on NYC), confirming its necessity for global dependency modeling.
- Removing External Features led to substantial errors, highlighting the importance of weather and POI data.
- Removing Graph Propagation and Temporal Convolution also resulted in measurable performance declines, validating the need for explicit local spatial and temporal modeling.
Efficiency: The model demonstrated linear scaling with respect to $N$ and $T$ , making it feasible for large-scale deployment where standard Transformers are computationally infeasible.
Case Studies: Visualizations of NYC demand patterns confirmed the model's ability to capture tidal flows (commuter patterns between residential and business districts), spatial imbalances, and long-range correlations between distant stations.

5. Significance and Claims

The paper claims that STAGformer provides a scalable and accurate solution for micro-mobility demand forecasting in large-scale urban systems. Its significance lies in successfully bridging the gap between the expressive power of Softmax attention and the linear efficiency required for city-scale networks. By combining graph-based local modeling with agent-based global modeling, it offers a robust framework that does not sacrifice predictive accuracy for computational speed.

The authors acknowledge limitations, noting the current reliance on fixed graph structures and the need for empirical tuning of agent token counts. Future work is suggested to explore dynamic graph learning, adaptive agent selection, and application to other micro-mobility systems (e.g., e-scooters).