Driver Activity Detection and Classification Using Deep Learning- Based YOLO Models for Intelligent Transportation Systems

Technical Summary: Driver Activity Detection and Classification Using Deep Learning-Based YOLO Models

Problem Statement
Driver distraction and unsafe behaviors remain primary contributors to road traffic accidents globally, resulting in significant economic losses, injuries, and fatalities. While Intelligent Transportation Systems (ITS) aim to mitigate these risks through real-time monitoring, existing solutions often face limitations. Prior research frequently relies on isolated behavioral detection (e.g., only phone usage or fatigue), lacks comprehensive multi-feature analysis (ignoring full-body posture or object interactions), and is often evaluated in offline settings rather than real-time video environments. Furthermore, many existing datasets are sensor-based or lack the specific bounding-box annotations required for modern object detection architectures, creating a gap in robust, real-time driver activity recognition systems suitable for diverse driving conditions.

Methodology
The study proposes a deep learning framework utilizing You Only Look Once (YOLO) object detection architectures to detect and classify driver activities in real-world scenarios.

• Dataset Construction: A custom dataset was developed from real-world driving conditions in Debre Markos and Motta towns, Ethiopia. Using a Samsung Galaxy A52 smartphone and a digital camera, 10 video sequences were recorded and converted into 5,422 manually annotated frames. The dataset includes eight distinct activity classes: safe driving, drinking, eating, phoning, operating vehicle components, makeup/hair adjustment, object management, and talking to passengers.
• Preprocessing and Annotation: Images were resized to 1280×720 (JPG format) and annotated using the LabelImg tool. Annotations were exported in YOLO format, consisting of normalized bounding box coordinates (center x, y, width, height) and class IDs. The dataset was split into 80% training, 10% validation, and 10% testing sets.
• Model Architecture: Four YOLO-based models were evaluated: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv8n. These models were selected for their balance of speed and accuracy. The study employed transfer learning techniques, training the models on the custom dataset using hyperparameters including a batch size of 10, 5 epochs, an image size of 640, and a learning rate of 0.001.
• Evaluation Metrics: Performance was assessed using standard object detection metrics: Precision, Recall, F1-score, and Mean Average Precision (mAP) at an Intersection over Union (IoU) threshold of 0.5. Training and validation loss curves were also analyzed to monitor convergence.

Key Contributions
The paper outlines five primary contributions:

1. Integrated Detection Model: The design of a model that incorporates both driver actions and object-handling behaviors, moving beyond isolated feature detection.
2. Comparative Framework: A systematic evaluation comparing state-of-the-art YOLOv5 variants against YOLOv8 for driver activity recognition.
3. Refined Activity Classes: The introduction of a compact, well-defined set of eight activity classes to enhance the accuracy and interpretability of detected behaviors.
4. Real-Time Monitoring System: The development of a fixed-camera-based system capable of observing driver activities and passenger visibility in real time.
5. Performance Validation: Demonstration of improved detection accuracy and classification reliability compared to existing related studies.

Experimental Results
Experiments were conducted using Google Colab with GPU acceleration (NVIDIA Tesla K80). The results indicated a clear correlation between model complexity and detection performance:

• YOLOv5s: Achieved a mAP of 90.0%.
• YOLOv5m: Achieved a mAP of 92.5%.
• YOLOv5l: Achieved a mAP of 95.0%.
• YOLOv8n: Achieved the highest performance with a mAP of 96.3%.

While YOLOv8n demonstrated superior accuracy, it required the longest training time (3.012 hours) and resulted in the largest model weight (102.4 MB). Conversely, YOLOv5s offered the fastest training but with reduced accuracy. The training and validation loss curves confirmed stable convergence across all models.

Significance and Claims
The paper claims that advanced YOLO architectures, particularly YOLOv8, provide accurate and efficient solutions for real-time driver activity recognition. The study asserts that its proposed framework significantly contributes to the enhancement of Intelligent Transportation Systems and road safety by enabling the continuous, automated monitoring of risky driving behaviors. The authors conclude that the proposed method outperforms existing approaches in terms of detection accuracy and reliability, validating the use of deep learning-based object detection for in-cabin monitoring.

Future Directions
The authors modestly suggest that future work should focus on optimizing YOLO architectures and exploring additional hyperparameters. They recommend expanding the dataset to include more diverse behaviors and increasing the number of activity classes to improve generalization. Additionally, future research should address limitations regarding nighttime detection and real-world deployment in complex traffic environments to further enhance system robustness.