🤖 machine learning

CoMemNet: Contrastive Sampling with Memory Replay Network for Continual Traffic Prediction

CoMemNet is a dual-branch continual learning framework that combines a momentum-updated target branch with a Dynamic Contrastive Sampler and a lightweight temporal memory buffer to effectively capture evolving traffic patterns and mitigate catastrophic forgetting, achieving state-of-the-art performance on large-scale real-world datasets.

Original authors: Mei Wu, Wenchao Weng, Wenxin Su, Wenjie Tang, Wei Zhou

Published 2026-05-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mei Wu, Wenchao Weng, Wenxin Su, Wenjie Tang, Wei Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict traffic jams in a city that is constantly growing. Every year, new roads are built, new intersections open up, and the way people drive changes. This is the challenge the paper CoMemNet tackles.

Here is the story of how they solved it, using simple analogies:

The Problem: The "Moving Target" City

Most traffic prediction tools are like old maps. They are drawn based on the city as it was a few years ago.

The Issue: If you build a new highway or a new neighborhood, your old map is useless.
The Old Way: To fix an old map, you usually have to throw it away and redraw the entire city from scratch every time a new road appears. This takes forever and costs a fortune (computational power).
The "Forgetting" Problem: If you try to just add the new road to the old map without re-drawing everything, the AI gets confused. It learns the new road but forgets how the old roads worked. This is called "catastrophic forgetting."

The Solution: CoMemNet (The Smart, Adaptable Traffic Coach)

The authors created a system called CoMemNet that learns like a smart coach who never forgets, even as the team grows and changes. It has three main tricks up its sleeve:

1. The "Dual-Brain" System (Online vs. Target Branch)

Imagine a coach with two brains working together:

The "Fast Learner" Brain (Online Branch): This brain is hyper-focused on today's traffic. It learns quickly to handle the new roads and current rush hour patterns. It's agile and fast.
The "Wise Mentor" Brain (Target Branch): This brain moves slowly. It remembers the traffic patterns from last year and the year before. It doesn't change its mind easily.
How they work together: The Fast Learner tries new things, but the Wise Mentor gently pulls it back to remember the basics. This ensures the system learns the new stuff without forgetting the old stuff.

2. The "Spotlight" Selector (DC Sampler)

You can't watch every single car in a massive city at once; your computer would explode.

The Trick: Instead of watching every road, CoMemNet uses a "Spotlight." It uses a special math tool (Wasserstein Distance) to ask: "Which roads are changing the most?"
The Result: If a road has been the same for 10 years, the Spotlight ignores it. If a new road just opened or an old road's traffic pattern suddenly flipped, the Spotlight shines brightly on it. The system only spends its energy studying the roads that actually need attention. This saves massive amounts of time and memory.

3. The "Pocket Notebook" (TMRB-N)

Usually, to remember old traffic, you'd need to store a giant library of every car's history. That's too heavy.

The Trick: CoMemNet uses a tiny, lightweight "Pocket Notebook." It doesn't store every detail. Instead, it only writes down the "gist" of the most important changes.
The Result: When the system needs to remember the past, it flips through this small notebook. It's so efficient that the system never runs out of memory, even as the city grows for years.

The Results: Why It Matters

The authors tested this on real traffic data from California (including two new datasets they created).

Accuracy: It predicted traffic better than any other method currently available (State-of-the-Art).
Efficiency: While other methods had to retrain on thousands of roads every year, CoMemNet only focused on a tiny fraction (sometimes less than 30%) of the roads, yet still got better results.
Longevity: It successfully handled a city growing over 7 years without "forgetting" how to predict traffic from the early years.

In a Nutshell

CoMemNet is like a traffic prediction system that doesn't need to relearn the whole city every time a new street is built. It has a fast brain for new things, a slow brain for old memories, a spotlight to focus only on what's changing, and a tiny notebook to keep things organized. This allows it to predict traffic accurately in a city that is constantly expanding, without crashing the computer.

Technical Summary: CoMemNet

Problem Statement

Traffic prediction is critical for intelligent transportation systems, yet existing methods face significant limitations when applied to real-world, evolving traffic networks. Most current approaches rely on static underlying graph structures and short-term training windows (typically one month), failing to account for the continuous expansion and evolution of road networks where new nodes and edges are added over time. This dynamic evolution leads to three primary challenges:

Feature Extraction under Dynamic Topology: Traditional methods relying on explicit graph structures (e.g., adjacency matrices) are inflexible to incremental topological changes and prone to information disruption during structural reconstruction.
Catastrophic Forgetting under Spatio-Temporal Drift: As traffic flow distributions shift over time, models trained on new data often forget historical patterns. Direct application of memory replay can introduce noise due to invalidated spatio-temporal dependencies.
Memory Explosion in Incremental Graphs: Storing multi-hop neighbor information for all historical nodes in dynamic, dense traffic graphs leads to exponential memory growth, making global storage and recomputation infeasible.

Methodology

The authors propose CoMemNet (Contrastive Sampling with Memory Replay Network), a dual-branch continual learning framework designed for long-term traffic prediction on expanding networks. The framework operates without requiring explicit graph structure inputs, relying instead on embedding-based learning.

1. Dual-Branch Architecture

CoMemNet employs a momentum-updated dual-branch structure to balance rapid adaptation with knowledge retention:

Online Branch: Optimized via gradient descent for the current time period ( $\tau$ ), this branch facilitates rapid convergence on current traffic patterns.
Target Branch: Updated via an Exponential Moving Average (EMA) of the Online Branch parameters, this branch maintains long-term stability and preserves historical knowledge, acting as a low-pass filter against rapid fluctuations.

2. Dynamic Contrastive Sampler (DC Sampler)

To address the challenge of selecting relevant data for training without global graph storage, the DC Sampler dynamically selects a subset of nodes ( $V^s_\tau$ ) for each time period.

Mechanism: It calculates the Wasserstein Distance between the feature distributions of the current period and the previous period (processed through the Target Branch).
Selection Strategy: It selects the top- $M$ nodes with the largest distribution shifts (indicating significant dynamic network feature changes) and combines them with newly added nodes. This implements a "hard-example-first" strategy, focusing training on nodes most sensitive to topological and traffic pattern changes.

3. Node-Adaptive Temporal Memory Replay Buffer (TMRB-N)

To prevent memory explosion while consolidating historical knowledge, CoMemNet utilizes a lightweight memory buffer.

Function: Instead of storing entire graph neighborhoods, TMRB-N stores low-dimensional temporal embeddings of key nodes.
Update Mechanism: It employs a gating mechanism (similar to GRU/LSTM cells) to update temporal features. It calculates feature differences between the current and previous periods, selects the top- $K$ nodes with the largest differences, and updates their hidden states using a weighted average of current and historical features. This ensures the model adapts to new distributions while retaining essential historical priors.

4. Training Objective

The model is trained using a Mean Absolute Error (MAE) loss function computed exclusively on the node subset selected by the DC Sampler. Evaluation, however, is performed on the entire node set to ensure comprehensive performance assessment.

Key Contributions

CoMemNet Framework: A simple, embedding-based backbone that eliminates the need for explicit graph structure inputs, enabling dynamic adaptation to incremental node and edge additions.
Dual-Branch Momentum Contrast: A mechanism that synergizes a fast-adapting Online branch with a stable Target branch to mitigate catastrophic forgetting under spatio-temporal drift.
Efficient Memory Management: The introduction of the DC Sampler and TMRB-N, which selectively update key nodes and store low-dimensional representations, effectively avoiding global graph storage and memory explosion.
New Datasets: The authors curated and released two new large-scale, open-source traffic datasets (PEMSD4(L) and PEMS8(M)) derived from CalTrans data, covering different regions of California with varying network scales.

Experimental Results

The authors evaluated CoMemNet on three large-scale real-world datasets: PEMSD3(S), PEMSD4(L), and PEMSD8(M).

Performance: CoMemNet achieved State-of-the-Art (SOTA) performance across all three datasets, outperforming baseline continual learning methods (e.g., TrafficStream, PECPM, TFMoE) and static/expansible models in terms of MAE, RMSE, and MAPE.
Efficiency: The model demonstrated superior computational efficiency. By training only on a small subset of nodes (e.g., 15–30% of the total nodes in later years) selected by the DC Sampler, CoMemNet significantly reduced training time and resource consumption compared to retraining models on all data or training on all nodes incrementally.
Ablation Studies: Experiments confirmed the necessity of both the DC Sampler (for selecting informative nodes) and the TMRB-N (for memory replay and temporal feature updates). Removing either component led to significant performance degradation.
Sensitivity Analysis: The model showed robust performance with specific hyperparameters (e.g., $\rho=0.05$ for node selection ratio and $K=12$ for key node selection), indicating stability across different dataset scales.

Significance and Claims

The paper claims that CoMemNet provides an efficient and effective solution for the emerging challenge of long-term continual learning in expanding traffic networks. By decoupling the model from fixed graph structures and utilizing a contrastive sampling strategy, it successfully addresses the trade-off between adapting to new topological changes and retaining historical knowledge. The authors assert that their approach validates the feasibility of deploying scalable, long-term traffic prediction systems that can evolve alongside urban infrastructure without incurring prohibitive computational or memory costs. The release of new open-source datasets further contributes to the research community's ability to benchmark continual learning in spatio-temporal domains.