Classification of Driver Behaviour Using External Observation Techniques for Autonomous Vehicles

This study presents a novel computer vision framework that utilizes external observation techniques, including YOLO-based object detection and lane monitoring, to classify distracted and impaired driver behaviors in non-connected vehicles without relying on inter-vehicular communication.

The Gist

Imagine you are driving down a highway, and you spot a car ahead of you swerving wildly, drifting in and out of its lane like a drunk sailor on a rocking boat. You know something is wrong with that driver, but you can't see inside the car to check if they are texting, sleeping, or have had too much to drink.

This research paper proposes a clever solution: teaching the car behind to "read" the road like a detective.

Here is the breakdown of the study in simple terms, using some everyday analogies.

The Big Problem: The "Black Box" of Driving

Currently, self-driving cars are great at following rules, but they struggle when they are surrounded by regular human drivers. If a human driver is distracted or impaired, they don't have a "Wi-Fi signal" to tell the self-driving car behind them, "Hey, I'm not paying attention!"

Without this communication, the self-driving car is flying blind. It doesn't know why the car in front is swerving; it just sees the swerving and has to react instantly, which can be dangerous.

The Solution: The "Digital Detective"

The researchers at Atlantic Technological University built a system that acts like a super-observant passenger sitting in the back seat of your car. Instead of looking at the driver's face (which it can't see from the outside), it watches the car's movements to guess what the driver is doing.

Think of it like a dance instructor watching a student.

• If the student is stepping smoothly and staying in rhythm, they are fine.
• If the student is stumbling, stepping on their own feet, or drifting off the dance floor, the instructor knows something is wrong.

How the System Works (The Three Steps)

1. The Eyes (Object Detection)
The system uses a camera (like a dashcam) and a smart AI brain called YOLO (You Only Look Once).

• Analogy: Imagine a security guard at a stadium who instantly spots every person in the crowd and puts a digital "box" around them. The system does this for cars, identifying them instantly so it knows, "Okay, that's a car, and I'm going to watch that one."

2. The Ruler (Lane Detection)
The system draws invisible lines on the road to see where the lane boundaries are.

• Analogy: It's like a tightrope walker looking at the rope. The system calculates exactly where the "rope" (the lane) is. If the car stays on the rope, it's safe. If the car starts walking off the edge, the system notices immediately.

3. The Pattern Recognizer (Behavior Analysis)
This is the magic part. The system measures two specific things:

• The Drift (Distracted Driving): If a car slowly drifts away from the center of the lane, like a boat losing its anchor, the system thinks, "This driver is probably looking at their phone or daydreaming."
• The Wiggle (Impaired Driving): If a car jerks left, then right, then left again (oscillating), like a snake slithering, the system thinks, "This driver is likely impaired or very tired."

The "Traffic Light" System

When the system spots these bad patterns, it doesn't just sit there. It triggers an alarm, similar to a traffic light changing colors:

• Green: You are driving straight. All good.
• Yellow: You are wobbling a bit. The system says, "Hey, watch out!"
• Red: You are swerving wildly or drifting far off course. The system screams, "Distracted Driver Ahead!" or "Impaired Driver Ahead!"

The Results: Did It Work?

The researchers tested this with video footage of cars driving on highways and local roads. They even had people pretend to be distracted or impaired drivers to see if the system would catch them.

• Success: It was very good at spotting the "wiggles" and the "drifts." It successfully flagged cars that were acting erratically.
• The Hiccups: Like any new technology, it had trouble when the road was messy. If the lane lines were faded (like a worn-out chalk line) or if it was raining heavily, the system got a bit confused. It also sometimes mistook a bumpy road for a drunk driver.

Why This Matters

This research is a bridge. We are currently in a "mixed traffic" era where self-driving cars and human drivers share the road. Until every car can talk to every other car (V2V communication), we need a way for self-driving cars to "see" the danger before it hits them.

This system gives autonomous vehicles super-vision. It allows them to anticipate danger not by guessing, but by watching the "dance moves" of the cars around them, making our roads safer for everyone, whether they are driving a robot or a human.

Technical Summary

Here is a detailed technical summary of the paper "Classification of Driver Behaviour Using External Observation Techniques for Autonomous Vehicles" by Ian Nell and Shane Gilroy.

1. Problem Statement

Road traffic accidents remain a critical global issue, with human error (specifically distraction and impairment) being a leading cause. While Advanced Driver Assistance Systems (ADAS) and Artificial Intelligence have improved safety, a significant gap exists in mixed-traffic environments where autonomous vehicles (AVs) coexist with non-connected human-driven vehicles (HDVs).

• The Core Challenge: Most current driver monitoring systems rely on internal sensors (e.g., cabin cameras, steering wheel trackers) or Vehicle-to-Vehicle (V2V) communication. However, many HDVs lack these connectivity features or internal monitoring capabilities.
• The Need: There is an urgent requirement for external observation systems that can infer a driver's cognitive or physiological state (distracted or impaired) solely by analyzing observable driving patterns (e.g., lateral movement, lane deviation) without requiring direct communication with the target vehicle.

2. Methodology

The authors propose a vision-based framework using a monocular camera mounted on an "ego vehicle" to detect unsafe behaviors in surrounding vehicles. The system operates through the following pipeline:

A. Hardware and Data Collection

• Sensor: A Garmin 55 dashcam (1440p resolution, 122° FOV, 30 FPS) mounted on the ego vehicle's windshield.
• Data Sources: Diverse video datasets including highways, local roads, and back streets under varying lighting and weather conditions.
• Simulations: Controlled scenarios were created where a target vehicle simulated distracted driving (excessive lane deviation) and impaired driving (oscillatory steering).

B. Computer Vision Pipeline

1. Object Detection:
   • Utilizes YOLOv8n (You Only Look Once) for real-time detection of vehicles and road users.
   • Selected for its lightweight architecture, high inference speed (~2–4 ms/frame), and training on the Microsoft COCO dataset.
2. Lane Detection:
   • Preprocessing: Grayscale conversion $\rightarrow$ Gaussian blurring $\rightarrow$ Adaptive thresholding $\rightarrow$ Canny edge detection.
   • Lane Fitting: Detected edges are processed using RANSAC (Random Sample Consensus) regression to fit polynomial curves, estimating lane boundaries and the lane center while minimizing noise and outliers.
3. Behavioral Analysis & Logic:
   • Lateral Displacement: Calculates the deviation of the detected vehicle's centroid from the estimated lane center.
   • Oscillatory Analysis: Tracks the frequency of sign changes in lateral movement direction.
   • Region of Interest (ROI): Focuses analysis on a specific area of the frame to filter irrelevant data.

C. Classification Logic

The system triggers alarms based on two distinct behavioral patterns:

• Distracted Driving Alarm: Triggered when:
  • Average lateral movement > 0.3 pixels.
  • Vehicle deviation from lane center > 40 pixels.
• Impaired Driving Alarm: Triggered when:
  • Three or more sign changes (oscillations) in lateral movement are detected within a specific timeframe, indicating erratic steering.

3. Key Contributions

• External Observation Framework: Introduces a novel approach to classifying driver behavior without relying on V2V/V2X communication or internal vehicle sensors, making it applicable to non-connected vehicles.
• Hybrid Algorithm: Successfully integrates YOLOv8n for object tracking with custom RANSAC-based lane estimation to create a robust external monitoring system.
• Real-Time Classification: Develops specific logic to distinguish between distracted (steady deviation) and impaired (oscillatory) driving based on lateral displacement metrics.
• Scalability: The use of a monocular camera and lightweight models suggests the system is cost-effective and deployable on standard vehicles, not just high-end autonomous platforms.

4. Results and Performance

The system was evaluated using controlled simulations and real-world video data.

• Distracted Driving Detection:
  • Successfully identified vehicles deviating significantly from the lane center.
  • Example: In Frame 36, a vehicle was flagged with a lateral offset of 56 pixels and an average lateral movement of 1.14 pixels, triggering the "Distracted Driver Ahead" alarm.
• Impaired Driving Detection:
  • Reliably detected oscillatory patterns.
  • Example: In Frame 483, the system detected 3 sign changes in lateral movement, triggering the "Impaired Driver Ahead" alarm. The system visualized the severity progression (Blue $\rightarrow$ Green $\rightarrow$ Yellow $\rightarrow$ Red).
• Data Logging: All metrics (Frame ID, Object Class, Lateral Movement, Lane Offset, Alarms) were logged to CSV files, confirming high alignment between visual indicators and recorded data.
• Limitations Observed:
  • Environmental Factors: Accuracy decreased in adverse weather (rain, fog) and on roads with faded or absent lane markings.
  • False Positives: Uneven road surfaces (e.g., potholes) were occasionally misclassified as impaired driving behaviors.
  • Performance: Real-time processing degraded slightly with multiple moving objects, suggesting a need for hardware acceleration.

5. Significance and Future Work

• Safety in Transition: This research addresses the critical "transitional phase" of autonomous transportation, where AVs must safely navigate alongside human drivers who lack advanced connectivity.
• Proactive Hazard Mitigation: By detecting unsafe behaviors externally, the system allows autonomous or semi-autonomous vehicles to take proactive measures (e.g., increasing following distance, preparing for emergency braking) before a collision occurs.
• Future Improvements:
  • Sensor Fusion: Integrating LiDAR, RADAR, and infrared sensors to overcome weather and lighting limitations.
  • Dataset Expansion: Training on more diverse scenarios (extreme weather, complex urban geometries).
  • False Positive Reduction: Developing algorithms to distinguish between road-induced vehicle movements and actual driver impairment.

Conclusion: The paper demonstrates that external observation using computer vision is a viable, scalable solution for enhancing road safety in mixed-traffic environments, offering a critical layer of protection where direct vehicle communication is unavailable.