🤖 AI

Hybrid Congestion Classification Framework Using Flow-Guided Attention and Empirical Mode Decomposition

This paper proposes FLO-EMD, a hybrid framework that integrates motion-guided attention with Empirical Mode Decomposition to jointly capture spatial context and non-stationary temporal dynamics, achieving 97.5% accuracy in classifying traffic congestion levels across diverse surveillance environments.

Original authors: Eugene Kofi Okrah Denteh, Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Eugene Kofi Okrah Denteh, Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how bad traffic is just by looking at a video feed from a highway camera. You have two main ways to do this, but both have a flaw:

The "Static Photo" Approach: You look at the picture and count cars. The problem? A picture of a crowded parking lot looks the same as a picture of a traffic jam, even though one is stopped and the other is moving slowly. You might get fooled by the static scenery (like the road paint or guardrails) instead of the actual cars.
The "Sound Wave" Approach: You listen to the traffic like a sound engineer, analyzing the rhythm of cars passing a specific point. The problem? You can't see where the jam is happening on the screen. You know there's a problem, but you don't know if it's in the left lane or the right lane.

This paper introduces a new system called FLO-EMD that acts like a super-smart traffic detective who combines the best of both worlds. Here is how it works, using simple analogies:

1. The "Motion Detective" (Optical Flow)

Instead of just looking at the colors of the cars (RGB), the system first looks at how things move. It creates an invisible "wind map" (called optical flow) that shows the direction and speed of every pixel in the video.

The Analogy: Imagine watching a river. A normal camera sees the rocks and the water color. The "Motion Detective" ignores the rocks and only looks at the swirling currents. This helps the system ignore static things like road signs or shadows that don't move, focusing only on the cars that actually matter.

2. The "Focus Filter" (Flow-Guided Attention)

Once the system knows where the movement is, it uses that information to tell its "eyes" where to look.

The Analogy: Think of a spotlight on a stage. Usually, a spotlight might shine on the most colorful part of the set (like a bright red curtain). But this system uses the "Motion Detective" to move the spotlight. If the cars are moving in the left lane, the spotlight moves there. If the right lane is empty, the spotlight dims. This stops the system from getting distracted by static background noise.

3. The "Rhythm Analyzer" (Empirical Mode Decomposition)

Traffic isn't steady; it stops, starts, and wobbles. Standard computers often try to fit traffic into a rigid, predictable box (like a metronome). This system uses a technique called Empirical Mode Decomposition (EMD) to break the traffic rhythm down into its natural, messy parts.

The Analogy: Imagine a musician playing a complex jazz solo. A standard recorder might try to force that solo into a simple 4/4 beat. EMD is like a genius music producer who listens to the solo and breaks it down into its unique layers: the fast drum beats, the slow bass line, and the sudden pauses. This allows the system to understand the true nature of the traffic flow, even if it's chaotic or changing suddenly.

4. The "Final Verdict"

The system takes the "Focus Filter" (what it sees) and the "Rhythm Analyzer" (how the traffic moves over time) and combines them.

The Result: It can tell the difference between Light (cars cruising freely), Medium (cars slowing down but moving), and Heavy (cars stopped or crawling) with 97.5% accuracy.

Why is this a big deal?

It works in the rain and fog: Because it focuses on movement patterns rather than just clear pictures, it doesn't get confused as easily when the weather is bad.
It uses existing cameras: You don't need to install new sensors on the road. It can use the CCTV cameras cities already have.
It's honest about what it sees: The system can show you a "heat map" of exactly which part of the road it is looking at, proving it isn't just guessing based on the background.

The Bottom Line

The authors tested this on over 1,000 video clips from highways in the US, covering sunny days, rainy nights, and snowy conditions. It beat all the other existing methods (like standard AI video models) because it didn't just "look" at the traffic; it "felt" the motion and understood the rhythm of the jam.

What it is NOT:
The paper does not claim this system can predict future traffic jams before they happen, nor does it claim to fix the traffic. It is strictly a tool for classifying (labeling) what is happening right now in the video feed. It also notes that while it works great on highways, it hasn't been fully tested on complex city intersections with cars turning in every direction yet.

Technical Summary: FLO-EMD Hybrid Congestion Classification Framework

Problem Statement
Accurate traffic congestion classification requires models that simultaneously capture roadway scene context and non-stationary traffic motion. Existing approaches typically treat these requirements in isolation. Vision-based methods often rely on appearance cues and standard temporal pooling, which can bias predictions toward static infrastructure rather than dynamic traffic states. Conversely, signal-based approaches (e.g., using loop detectors) characterize temporal dynamics well but lack the spatial context necessary for scene-level localization. Furthermore, vision-based models frequently suffer from attention mechanisms that drift toward high-contrast static elements (e.g., lane markings, shadows) rather than motion-relevant regions, and they often fail to explicitly decompose non-stationary motion patterns like stop-and-go oscillations.

Methodology: The FLO-EMD Framework
The paper proposes FLO-EMD, a hybrid framework that couples motion-guided attention with empirical, data-driven temporal decomposition. The architecture processes traffic video clips through parallel RGB and dense optical flow backbones, integrating them via a dual-stream design.

Optical Flow Extraction and Motion Characterization:
- Dense optical flow is estimated between consecutive frames using the Farneback algorithm.
- Instead of feeding raw flow fields directly into a deep network for temporal modeling, the system aggregates flow statistics into compact, frame-level motion descriptors. These include the spatial mean and standard deviation of flow magnitude ( $\mu_M, \sigma_M$ ) and flow direction ( $\mu_D, \sigma_D$ ).
- These descriptors form time-series "motion traces" representing scene-level congestion dynamics.
Dual-Stream Backbone and Flow-Guided Attention:
- The framework employs two parallel convolutional encoders: one for RGB frames and one for optical flow fields. Both produce spatially and channel-wise aligned feature maps.
- A Flow-Guided Attention Mechanism (an enhanced CBAM block) uses the optical flow features to refine the RGB features.
  - Channel Attention: Computed from pooled descriptors of both RGB and flow features to emphasize motion-relevant feature channels.
  - Spatial Attention: Incorporates a resized flow-magnitude map alongside RGB statistics to reweight spatial regions, ensuring the model focuses on traffic-active areas rather than static infrastructure.
Empirical Mode Decomposition (EMD) for Temporal Analysis:
- The compact motion traces (the four statistical descriptors over time) are decomposed using Empirical Mode Decomposition (EMD).
- EMD adaptively decomposes the non-stationary time series into Intrinsic Mode Functions (IMFs) without fixed frequency assumptions.
- The paper retains $N=4$ IMFs per descriptor. Each IMF is summarized by its temporal mean and standard deviation, creating a fixed-dimensional embedding (32 scalar features per clip) that captures multi-scale temporal dynamics.
Feature Fusion and Classification:
- The learned spatiotemporal representations (from the RGB and flow streams processed by BiLSTMs and attention-based pooling) are fused with the EMD-derived motion embedding.
- A final Multi-Layer Perceptron (MLP) classifies the fused vector into three congestion states: Light, Medium, and Heavy.
- Training utilizes cross-entropy loss with label smoothing to improve generalization.

Key Contributions

Application of EMD to Optical Flow: The study introduces the application of Empirical Mode Decomposition to optical-flow time series. This allows for the adaptive decomposition of non-stationary traffic motion into intrinsic mode functions, characterizing multi-scale temporal dynamics critical for distinguishing congestion levels, offering a robust alternative to fixed-window temporal analysis.
Hybrid Signal-Vision Framework: The paper presents a unified architecture that seamlessly combines classical signal processing (EMD) with modern deep learning (dual encoders and attention). This integration bridges spatial video analysis and temporal pattern recognition, addressing the limitations of purely vision-based or purely sensor-based approaches.

Experimental Results
The framework was evaluated on a dataset of 1,050 five-second clips from four surveillance networks (Seattle I-5, AI-City Challenge 2021, Tennessee TDOT, and NYC 511NY), covering diverse conditions including clear, rainy, snowy, foggy, and nighttime scenarios.

Performance: FLO-EMD achieved an overall test accuracy of 97.5% with a weighted F1 score of 0.9742, outperforming established baselines including Transformer-based models (TimeSformer, Video Swin Transformer), CNN-based models (X3D, TANet), and hybrid pipelines (TSM, YOLO+LSTM).
Robustness: The model demonstrated consistent performance across environmental conditions, with the lowest accuracy observed in rainy conditions (93.50%) and the highest in clear daytime (98.50%).
Ablation Studies:
- Removing flow-guided attention resulted in a significant performance drop and increased overfitting, confirming the necessity of motion-conditioned feature refinement.
- Removing the EMD branch reduced accuracy, validating the value of adaptive temporal decomposition.
- Sensitivity analysis indicated that retaining 4 IMFs provided the optimal balance between capturing distinct temporal behaviors and avoiding overfitting to short-lived fluctuations.
Interpretability: Attention visualizations showed that the flow-guided mechanism successfully focused on traffic-active regions (e.g., vehicle queues) rather than static background elements, although some attention was occasionally assigned to road shoulders due to flow artifacts.

Significance and Claims
The paper claims that FLO-EMD provides a unified solution that links motion evidence to spatial feature selection while preserving data-adaptive temporal characterization. By leveraging existing CCTV infrastructure, the method enables transportation agencies to expand monitoring coverage without installing additional roadside sensors. The authors assert that the explicit motion-driven nature of the framework supports reliable operation under adverse weather and low-light conditions, making it well-suited for routine corridor monitoring and real-time operational decision support within intelligent transportation systems. The study concludes that combining motion-conditioned spatial analysis with adaptive temporal decomposition is an effective direction for reliable congestion classification, though it acknowledges limitations regarding generalization to complex urban intersections and the potential for attention to be influenced by boundary artifacts.