Scaling Real-Time Traffic Analytics on Edge-Cloud Fabrics for City-Scale Camera Networks

This paper presents a scalable, AI-driven Intelligent Transportation System that orchestrates edge-cloud resources to process thousands of city-scale CCTV streams in real time, utilizing DNNs for detection, Spatio-Temporal GNNs for forecasting, and continuous federated learning to maintain accuracy under strict latency and bandwidth constraints.

The Gist

Imagine a city like Bengaluru as a giant, living organism. Its veins are the roads, and its blood cells are the millions of vehicles zipping around every day. Right now, this organism is suffering from a severe case of "traffic congestion," causing stress, pollution, and lost time for everyone.

The paper you shared describes a smart, AI-powered nervous system designed to fix this. Instead of trying to watch every single car with a human eye (which is impossible), the researchers built a system that uses thousands of existing security cameras to "see" the traffic, understand it, and predict where the jams will happen next.

Here is how their system works, broken down into simple concepts and analogies:

1. The Problem: Too Much Data, Too Little Time

Bengaluru has over 5,000 security cameras. If you tried to send all that video footage to a giant central computer (the "Cloud") to analyze, two things would happen:

• The Internet would clog: It's like trying to drink a firehose through a straw. The bandwidth isn't there.
• It would be too slow: By the time the central computer figures out there's a jam, the jam has already happened.

The Solution: Don't send the video; send the insights.
The researchers put "mini-brains" (small, powerful computers called Jetsons) right next to the cameras. These mini-brains watch the video, count the cars, and only send a tiny, lightweight summary (e.g., "5 cars, 2 buses, moving slowly") to the central cloud.

2. The Edge: The "Local Managers"

Think of the Jetson devices as local managers in a busy office.

• What they do: They watch the live feed, identify vehicles (cars, trucks, auto-rickshaws), and track them.
• The Challenge: Some managers are super-fast (high-end computers), and some are slower (older models). If you give a heavy workload to a slow manager, they crash. If you give too little to a fast one, you're wasting money.
• The Magic Trick: The system uses a Smart Scheduler. It's like a traffic cop who constantly checks which manager has the most energy left. It assigns video streams to the right manager so that everyone works at a steady pace without burning out. This allows the system to handle hundreds of cameras simultaneously without slowing down.

3. The Cloud: The "City Oracle"

Once the local managers send their summaries, the Cloud (a powerful central server) takes over.

• The Graph: Imagine the city roads as a giant spiderweb. The Cloud connects all the data points from the cameras to build a dynamic traffic map.
• The Crystal Ball (GNN): The system uses a special type of AI called a Spatio-Temporal Graph Neural Network (ST-GNN). Think of this as a "Crystal Ball" that doesn't just look at where cars are now, but predicts where they will be in 5 or 10 minutes.
• Why it matters: If the Crystal Ball sees a jam forming near a school, the city can change traffic lights before the jam happens, or tell drivers to take a different route.

4. The Self-Learning Loop: The "Teacher"

One of the hardest parts of traffic in India is that it's chaotic. You have cows, motorcycles, three-wheeled rickshaws, and massive trucks all mixed together. Standard AI models often get confused by these unique vehicles.

• The Problem: If the AI sees a new type of vehicle it doesn't know, it ignores it.
• The Solution: The system uses a Foundation Model (a super-smart AI like SAM3) as a "Teacher."
  • When the local cameras see something weird, the "Teacher" helps label it automatically.
  • The local computers then learn from this new label.
  • Federated Learning: Instead of sending all the video to the cloud to retrain the AI (which is slow and expensive), the computers learn locally and just send the "lessons learned" back to the cloud. The cloud combines these lessons and sends a smarter version of the AI back to all the cameras.
  • Analogy: It's like a school where every student (camera) studies a different topic, writes a short summary of what they learned, and the teacher (cloud) combines all summaries to update the textbook for the whole class.

5. The Dashboard: The "Mission Control"

Finally, all this data is visualized on a Dashboard (like a video game map).

• City officials can see a live map of Bengaluru.
• Roads turn Red (heavy traffic), Yellow (moderate), or Green (free flow).
• They can see not just the current traffic, but the predicted traffic, allowing them to make proactive decisions.

The Big Achievement

The researchers tested this on a neighborhood with 100 cameras and proved it works. They showed that:

1. It scales: They can handle 1,000 cameras (and eventually 5,000) without the system breaking.
2. It's fast: The predictions happen in real-time.
3. It's smart: It adapts to new vehicles and changing conditions automatically.

In summary: This paper describes a way to turn a chaotic, congested city into a well-oiled machine by giving every camera a local brain, connecting them with a smart scheduler, and using a central "Oracle" to predict the future, all while teaching the system to learn from its own mistakes without needing human help. It's a blueprint for making cities breathe easier.

Technical Summary

Here is a detailed technical summary of the paper "Scaling Real-Time Traffic Analytics on Edge–Cloud Fabrics for City-Scale Camera Networks."

1. Problem Statement

The paper addresses the critical challenge of implementing Real-Time Intelligent Transportation Systems (ITS) in rapidly urbanizing, heterogeneous environments like Bengaluru, India. Key problems include:

• Heterogeneous Traffic: Unlike structured Western traffic, Indian roads feature mixed vehicle types (two-wheelers, three-wheelers, buses, trucks) moving in non-lane-based patterns, making traditional simulation tools (e.g., SUMO) ineffective.
• Infrastructure Limitations: Cities rely on thousands of general-purpose safety cameras (CCTV) rather than specialized traffic sensors. These cameras generate massive HD video streams (5000+ in Bengaluru alone).
• Scalability & Latency: Centralized cloud architectures struggle with the bandwidth, compute, and latency requirements of processing 100s–1000s of video streams simultaneously. Transmitting raw video to the cloud is unsustainable, and centralized inference introduces unacceptable delays for real-time traffic management.
• Model Drift: Standard object detection models trained on datasets like COCO perform poorly on local, evolving vehicle classes (e.g., specific auto-rickshaw models), requiring continuous adaptation without expensive manual re-annotation.

2. Methodology: The Edge-Cloud Fabric

The authors propose a scalable, AI-driven Intelligent Transportation System (AIITS) that utilizes an edge-first architecture coupled with cloud-based aggregation. The system is designed to transform raw video into a dynamic traffic graph.

A. Edge Layer (Ingestion & Inference)

• Hardware: The system uses Raspberry Pis (colocated with cameras) to serve RTSP video streams and Nvidia Jetson Orin accelerators (Orin AGX 32GB/64GB) for high-throughput inference.
• Pipeline:
  1. Decoding: Uses hardware-accelerated GStreamer pipelines (nvdec) to decode streams directly into GPU memory.
  2. Batching: nvstreammux batches frames from multiple streams to maximize GPU utilization.
  3. Detection & Tracking: Employs YOLOv8 (specifically YOLO26s) trained on a local dataset (UVH-26) for vehicle detection, coupled with BoT-SORT for multi-object tracking to assign persistent IDs.
  4. Output: Instead of sending video, the edge devices emit lightweight flow summaries (vehicle counts, classes, bounding boxes) aggregated over short intervals (5–30s).

B. Intelligent Routing & Scheduling

• A capacity-aware scheduler dynamically assigns video streams to heterogeneous Jetson devices.
• It treats stream assignment as a bin-packing problem, utilizing empirical FPS thresholds for each device.
• Two heuristics are evaluated:
  • Best Fit: Packs streams onto the smallest capable device first to minimize active hardware and power consumption.
  • Worst Fit: Distributes load to devices with the most remaining capacity to improve thermal balance and load distribution.

C. Cloud Layer (Aggregation & Forecasting)

• Temporal Graph Construction: The cloud ingests lightweight flow summaries to build a Spatio-Temporal Graph.
• Forecasting: Uses Spatio-Temporal Graph Neural Networks (ST-GNN), specifically the TrendGCN architecture, to perform:
  • Nowcasting: Real-time traffic state estimation.
  • Short-horizon Forecasting: Predicting congestion 1–5 minutes ahead.
• Graph Abstraction: Since camera coverage is sparse, the system constructs a "coarsened" graph where unobserved road segments are collapsed into "super-edges" connecting observed junctions.

D. Continuous Adaptation (Federated Learning)

• To handle evolving vehicle classes, the system implements Continuous Federated Learning (FL).
• SAM3 Integration: Edge devices use the SAM3 foundation model to automatically annotate sampled frames with text prompts (e.g., "a sedan," "a 3-wheeler").
• Local Retraining: Devices locally fine-tune the YOLO detector on this new data and upload updated weights to a central server for aggregation (FedAvg), which is then redistributed. This occurs concurrently with inference, avoiding data drift.

3. Key Contributions

1. Scalable Edge-Cloud Architecture: A unified framework demonstrating multi-dimensional scaling:
   • Scale-out: Across 50+ heterogeneous edge devices (RPis and Jetsons).
   • Scale-up: GNN pipelines sustaining inference for up to 1000 streams.
   • Cross-fabric: Seamless coordination between edge inference and cloud forecasting.
2. Capacity-Aware Scheduler: A novel scheduling policy that balances load and power consumption across heterogeneous hardware, ensuring real-time performance as stream counts increase.
3. Foundation Model-Assisted FL: An innovative pipeline integrating SAM3 for automatic labeling within a Federated Learning loop, enabling continuous model adaptation on the edge without raw data transmission.
4. End-to-End Validation: A testbed in Bengaluru validating the system with 100+ real-world camera feeds, proving stability and low latency.

4. Experimental Results

The system was evaluated on a testbed emulating a city neighborhood with 100+ streams and planned for 1000 streams.

• Edge Performance:
  • Throughput: Sustained ingestion of >2000 FPS on Jetson Orin clusters.
  • Stability: Raspberry Pis maintained stable FPS (25 ± 1) with low CPU usage (<25%) using stream-copy mechanisms.
  • Scheduling: Both "Best Fit" and "Worst Fit" strategies maintained real-time throughput. Best Fit minimized active devices (power efficiency), while Worst Fit improved load balancing.
• Forecasting Accuracy:
  • The ST-GNN (TrendGCN) achieved stable training convergence.
  • RMSE: Increased modestly from ~20 (1-min horizon) to ~23 (4-min horizon), demonstrating robust short-term forecasting.
  • Latency: Forecasting latency remained manageable (<2s per request) even with 4 concurrent clients scaling to 1000 streams.
• Federated Learning:
  • SAM3 Labeling: Achieved annotation latencies of 4.0s–6.3s per image on Jetson Orins.
  • Data Distribution: Successfully handled non-IID data distributions across devices, proving the necessity of FL for heterogeneous traffic environments.

5. Significance and Impact

• Urban Mobility: Provides a deployable solution for "Smart Cities" in developing nations where infrastructure is limited and traffic is chaotic. It transforms existing CCTV networks into active traffic management tools.
• Sustainability: By processing data at the edge, the system drastically reduces bandwidth requirements and cloud compute costs, making city-scale analytics economically viable.
• Resilience: The continuous FL loop ensures the system adapts to new vehicle types and changing traffic patterns without requiring manual re-annotation or system downtime.
• Future Roadmap: The authors plan to scale the live demonstration to 1000 streams and eventually 5000+ feeds (covering the entire Bengaluru network), integrating dynamic model selection and network topology optimization.

In conclusion, this paper presents a robust, end-to-end architecture that solves the "data deluge" problem in urban traffic analytics by combining edge computing, graph neural networks, and foundation-model-assisted federated learning.