🤖 machine learning

MoE Enhanced Federated Learning for Spatiotemporal Prediction

The paper proposes MoE-FedTP, a personalized federated learning framework that leverages lightweight Mixture-of-Experts networks and dynamic gating to effectively address spatiotemporal heterogeneity and privacy concerns in cross-city traffic prediction for data-scarce cities.

Original authors: Zhehao Dai, Xiao Han, Zhaolin Deng, Zijian Zhang, Xiangyu Zhao, Guojiang Shen, Xiangjie Kong

Published 2026-06-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhehao Dai, Xiao Han, Zhaolin Deng, Zijian Zhang, Xiangyu Zhao, Guojiang Shen, Xiangjie Kong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict traffic jams in a small town that doesn't have many traffic cameras or sensors. You know the town has unique roads and driving habits, but you don't have enough data to build a smart prediction system from scratch.

Now, imagine you have access to the massive, detailed traffic data from four huge, busy cities (like New York, Los Angeles, etc.). You want to use what those big cities know to help your small town, but there's a catch: the big cities don't want to share their raw data (like exact GPS locations of every car) because of privacy rules. Also, the traffic patterns in a small town are very different from a massive metropolis, so a model trained on the big city might not work well in the small one.

This paper introduces a solution called MoE-FedTP. Think of it as a "Smart Traffic Consultant" that works without ever seeing the private data. Here is how it works, broken down into simple concepts:

1. The Problem: "One Size Does Not Fit All"

In the past, researchers tried to train one giant model on all cities and then just tweak it for the small town. But this is like trying to teach a person who drives a compact car how to drive a semi-truck just by giving them the same manual. The differences (heterogeneity) between cities are too big. The model gets confused, and the predictions are bad.

2. The Solution: The "Expert Panel" (Mixture of Experts)

Instead of one giant brain, the authors created a system with a Panel of Experts.

The Experts: Imagine each big city trains its own specialized "traffic expert." These experts learn the specific rhythm of their own city (e.g., how traffic flows in Chicago vs. how it flows in Chengdu).
The Privacy Trick: These experts don't send their private data to the small town. Instead, they send only their "knowledge weights" (the math inside their brains) to a central server. This is like sending a recipe card instead of the actual food. This keeps the data private.

3. The "Gating Mechanism": The Smart Manager

When the small town needs a prediction, it doesn't ask all the experts to shout out answers at once. That would be chaotic.

Instead, there is a Smart Manager (called a "Gating Network").
The Manager looks at the small town's current traffic situation.
It asks: "Which of our big-city experts knows the most about this specific situation?"
If the small town has a situation that looks a bit like Los Angeles, the Manager picks the "LA Expert." If it looks like Shenzhen, it picks the "Shenzhen Expert."
It might even pick two experts if the situation is complex, blending their advice together.

4. Why It's "Lightweight" and "Federated"

Federated: The small town never sees the raw data from the big cities. They only swap the "recipes" (model parameters). Privacy is safe.
Lightweight: The small town doesn't have to run all the experts at once. It only activates the top 2 or 3 experts that the Manager thinks are relevant. This saves computing power, which is great for cities with fewer resources.

The Results

The authors tested this on real traffic data from four major cities. They simulated a scenario where one city had very little data (like a new city just starting to collect data).

The Outcome: Their "Expert Panel" system (MoE-FedTP) predicted traffic much better than other methods. It was more accurate than trying to force a single model to work for everyone, and it was better than just copying a model without the "Manager" to pick the right experts.
The Proof: In their tests, the system reduced prediction errors significantly (by about 5% to 6% in some cases), proving that mixing and matching the right "experts" helps small, data-poor cities learn from big, data-rich ones without breaking privacy.

In short: This paper builds a privacy-safe system where small cities can borrow the "brainpower" of big cities by hiring the right "experts" for the job, rather than trying to copy the whole city's data.

Technical Summary: MoE-FedTP for Spatiotemporal Prediction

Problem Statement
Traffic prediction is critical for intelligent transportation systems, yet many cities suffer from data scarcity due to limited sensor deployment and uneven urban development. While cross-city knowledge transfer offers a solution, existing approaches face two primary hurdles:

Privacy Concerns: Centralized methods require sharing raw data, posing significant privacy risks.
Spatiotemporal Heterogeneity: Cities exhibit distinct traffic evolution patterns, functional area distributions, and travel behaviors. Existing Federated Learning (FL) and Personalized FL (PFL) methods often rely on unified prediction strategies that struggle to accommodate these inherent differences, leading to model drift and suboptimal performance in data-scarce target cities.

Methodology: MoE-FedTP
The authors propose MoE-FedTP, a personalized federated cross-city spatiotemporal prediction framework based on lightweight Mixture-of-Experts (MoE) networks. The framework operates in two stages:

Source City Learning (Local Training):
- Each source city acts as an independent client training a local spatiotemporal neural network.
- The network architecture integrates Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Attention modules to model spatial dependencies and temporal dynamics.
- Features are extracted via Gated Recurrent Units (GRU) for temporal sequences, pooled, and fused with spatial embeddings from GCN/GAT layers.
- A Multi-Layer Perceptron (MLP) generates the final prediction, optimized using Mean Squared Error (MSE).
Target City Adaptation (Federated Personalization):
- Expert Construction: Critical parameters from the pre-trained prediction heads of multiple source cities are shared and retained as "expert networks." Each expert encodes the unique spatiotemporal patterns of its respective source city.
- Gating Mechanism: A gating network dynamically computes importance weights for each expert based on the target city's input features.
- Top-k Selection: To ensure computational efficiency, the system activates only the top- $k$ experts with the highest gating scores.
- Prediction Fusion: The final prediction is a weighted sum of the outputs from the selected experts. The target city minimizes its own prediction loss using its limited local data, preserving privacy while leveraging external knowledge.

Key Contributions

Framework Design: Introduction of MoE-FedTP, a personalized federated framework utilizing a lightweight MoE structure to specifically address cross-city spatiotemporal heterogeneity.
Parameter Sharing Mechanism: A novel cross-city parameter sharing strategy that summarizes spatiotemporal patterns from source cities into expert networks and dynamically transfers them to target cities, enhancing performance in data-scarce scenarios.
Empirical Validation: Extensive experiments demonstrating that the proposed method outperforms state-of-the-art baselines in both accuracy and generalization.

Experimental Results
The method was evaluated on four real-world datasets: PEMS-BAY, METR-LA, DiDi-Chengdu, and DiDi-Shenzhen. The experimental setup simulated a data-scarce target city using only three days of data, while three other datasets served as data-rich source cities for pre-training.

Performance: MoE-FedTP consistently outperformed baselines including STGCN (spatiotemporal GNN), ST-GFSL (few-shot learning), and pFedCTP (adaptive interpolation PFL).
- On METR-LA, the model achieved a 6.23% improvement in MAE and 5.20% in RMSE compared to baselines.
- On DiDi-Shenzhen, an average performance gain of 4.68% was observed.
Ablation Studies:
- Removing the MoE module resulted in significant performance degradation (e.g., a 7.4% MAE increase on DiDi-Chengdu), confirming the necessity of the expert network structure.
- Top-k Sensitivity: The optimal number of active experts ( $k$ ) varied by dataset. Cities with complex, diverse patterns (PEMS-BAY, METR-LA) performed best with $k=2$ , while cities with distinctive patterns (DiDi-Chengdu, DiDi-Shenzhen) achieved optimal results with $k=1$ . This suggests that activating too many experts can introduce noise for specific city types.

Significance and Claims
The paper claims that MoE-FedTP provides an efficient, privacy-preserving solution for cross-city traffic prediction. By combining the privacy benefits of federated learning with the fine-grained modeling capabilities of MoE, the framework successfully mitigates the challenges of spatiotemporal heterogeneity. It enables data-scarce cities to leverage knowledge from data-rich counterparts without compromising data security or ignoring local traffic dynamics. The authors position this work as a promising direction for federated transfer learning in broader urban computing and spatiotemporal forecasting tasks.

Limitations and Future Work
The authors acknowledge that the current partial parameter sharing mechanism may be insufficient for cities with extreme heterogeneity or when the number of source cities is limited. Additionally, the fixed top- $k$ selection strategy may not optimally adapt to dynamic traffic scenarios. Future work aims to explore adaptive expert selection strategies (e.g., reinforcement learning or uncertainty-aware gating) and integrate multi-modal urban data to further improve robustness and scalability.