A 360-degree Multi-camera System for Blue Emergency Light Detection Using Color Attention RT-DETR and the ABLDataset

This paper presents a 360-degree multi-camera system for detecting blue emergency lights that utilizes a curated ABLDataset and an enhanced RT-DETR model with a color attention block to achieve high accuracy, azimuthal localization, and approach angle estimation for integration into Advanced Driver Assistance Systems.

The Gist

Imagine you are driving down a busy street. You are focused on the road ahead, checking your mirrors, and listening to the radio. Suddenly, an ambulance or fire truck is rushing toward you, but you don't hear its siren yet, or maybe it's too loud outside to hear it clearly. In the old days, you'd have to rely on luck or a sudden flash of blue light to realize, "Oh no, I need to move!"

This paper is about building a super-powered "sixth sense" for your car that acts like a vigilant co-pilot, constantly scanning 360 degrees around the vehicle to spot emergency vehicles before they become a danger.

Here is the breakdown of how they did it, using some everyday analogies:

1. The Eyes: The "Owl's Gaze" (360° Cameras)

Most cars have a few cameras, but this system uses four fish-eye cameras mounted on the front, back, and sides.

• The Analogy: Think of a normal camera as a human eye looking straight ahead. A fish-eye lens is like an owl's head that can swivel 180 degrees without moving its body. By placing four of these "owls" on the car, the system creates a perfect, unbroken circle of vision. It sees everything behind you, beside you, and in front of you simultaneously, just like a security guard with a spinning head.

2. The Brain: The "Color-Sniffing Detective" (AI Model)

The car needs a brain to figure out what it's seeing. The researchers tested many different "brains" (AI models like YOLO and Faster R-CNN), but they found that one called RT-DETR was the best at the job.

• The Upgrade: They realized that standard AI sometimes gets confused by white lights or reflections. So, they gave this AI a special pair of glasses called a "Color Attention Block."
• The Analogy: Imagine you are looking for a specific red balloon in a sea of white clouds. A normal person might get distracted by the white clouds. But if you put on red-tinted glasses, the white clouds fade away, and the red balloon pops out instantly. That's what this "Color Attention" does: it tells the AI, "Ignore everything that isn't blue or bluish-white. Focus only on the emergency lights."

3. The Training: The "YouTube Detective" (The Dataset)

To teach this AI, you need a massive library of pictures. But there was a problem: no one had a library of just flashing blue emergency lights.

• The Solution: The team became "detectives" on YouTube. They hunted down videos made by "emergency vehicle spotters" (people who love filming ambulances and police cars). They downloaded thousands of clips and manually drew boxes around every single blue light they could find.
• The Result: They created a new, custom textbook called the ABLDataset. It's like a specialized training manual that teaches the AI to recognize blue lights in rain, snow, day, and night, regardless of what country the car is in.

4. The Map: The "Compass" (Calibration)

Knowing that there is an emergency vehicle is good, but knowing where it is is better. The system doesn't just say "Emergency vehicle detected!"; it says "Emergency vehicle detected 45 degrees to your left."

• How it works: They used a mathematical process called calibration to map the distorted fish-eye images onto a flat map of the road.
• The Analogy: Imagine looking at a reflection in a funhouse mirror. It's distorted and curved. The system takes that curved image and "unfolds" it mathematically to tell the driver exactly where the danger is coming from, like a compass pointing directly at the threat.

5. The Results: The "Night Owl" vs. The "Daytime Eagle"

They tested the system in the real world, driving cars with fake emergency lights.

• Daytime: The system is like an eagle, spotting lights clearly up to 30 meters (about 100 feet) away.
• Nighttime: It gets even better! Because the blue lights contrast sharply against the dark, the system can spot them from up to 70 meters (over 200 feet) away.
• The Magic: Even if the siren is off, or the vehicle is far away, this system spots the blue light and alerts the driver instantly.

Why Does This Matter?

Think of this system as a safety net.

• For the Driver: It gives you extra time to react. Instead of swerving at the last second, you get a gentle nudge on your dashboard screen saying, "Ambulance coming from the left," giving you time to move over calmly.
• For the Emergency Team: It helps them get through traffic faster and safer, potentially saving lives.
• For the Future: This is a building block for self-driving cars. If a car can "see" and "understand" emergency lights, it can automatically pull over, making our roads safer for everyone.

In a nutshell: The researchers built a 360-degree "owl-eye" vision system, taught it to ignore distractions using "red-tinted glasses" (color attention), and trained it on a massive library of YouTube videos. The result is a car that never misses a flashing blue light, no matter where it is or what time of day it is.

Technical Summary

Here is a detailed technical summary of the paper "A 360° Multi-camera System for Blue Emergency Light Detection Using Color Attention RT-DETR and the ABLDataset."

1. Problem Statement

The automatic detection of emergency vehicles (EVs) is critical for Advanced Driver Assistance Systems (ADAS) to improve road safety and reduce response times. However, existing solutions face several challenges:

• Visual Variability: Emergency vehicles vary significantly in design, shape, and emblem placement across different regions, making whole-vehicle detection difficult to generalize.
• Siren Limitations: Audio-based detection fails when sirens are off (common at night or in quiet zones) or when the vehicle is not yet visible.
• Sensor Constraints: Standard cameras have limited fields of view (FoV), potentially missing EVs approaching from blind spots. Fisheye cameras offer 360° coverage but introduce severe radial distortion that complicates object detection.
• Data Scarcity: There is a lack of specialized datasets containing annotated images of active blue emergency lights under diverse real-world conditions (day/night, various weather, and geographic locations).

2. Methodology

The authors propose a holistic system comprising a new dataset, a modified deep learning architecture, and a multi-camera hardware setup.

A. The ABLDataset (Active Blue Light Dataset)

To address the data gap, the authors created the ABLDataset, a curated resource specifically for detecting active blue emergency lights.

• Source: Extracted from YouTube videos of emergency vehicle spotters, covering European countries (Spain, UK, Portugal, Italy, Hungary, Germany).
• Scale: ~3,000 images with 10,437 annotations.
• Annotation Strategy: Instead of labeling entire vehicles, the dataset focuses on "Active Blue Light" as a single class. Annotations cover individual bulbs or contiguous bulb arrays.
• Augmentation: To simulate the fisheye environment, the authors applied fisheye distortion augmentation. They calibrated real fisheye lenses to obtain intrinsic parameters and distortion coefficients, then applied random variations (<10%) to the dataset images and transformed bounding box coordinates accordingly.

B. Model Architecture: Color Attention RT-DETR (CA RT-DETR)

The authors evaluated several state-of-the-art models (YOLO v5/v8/v10, RetinaNet, Faster R-CNN, RT-DETR) and selected RT-DETR as the baseline due to its superior balance of precision and recall. They enhanced it with a Color Attention Module:

• Motivation: Blue lights often appear as intense white with blue tints or pure blue, which generic detectors may miss.
• Mechanism:
  1. Convert input images to HSV color space.
  2. Generate two masks: one for pure blue regions and one for white regions with blue components.
  3. Apply a residual connection: $output = x + (x \times mask)$. This amplifies pixels corresponding to emergency lights while preserving the original image features for the transformer backbone.
• Goal: To force the model to focus on chromatic features specific to emergency signals, improving robustness against varying lighting conditions.

C. 360° Multi-Camera System & Calibration

• Hardware: A system of four fisheye cameras (180° FoV each) mounted on the front, side mirrors, and rear of a vehicle to provide full 360° coverage.
• Calibration:
  • Intrinsic: Calibrated individually using chessboard patterns and a specialized fisheye distortion model (4 radial coefficients).
  • Extrinsic: Determined the relative position and orientation of cameras using overlapping fields of view and SIFT feature matching, unified via a bundle adjustment framework.
• Azimuthal Estimation: The system converts pixel coordinates of detections into a 3D vector relative to the vehicle's center, calculating the azimuthal angle (direction) of the approaching EV to alert the driver.

3. Key Contributions

1. ABLDataset: The first specialized dataset for active blue emergency light detection, addressing the lack of annotated data for this specific task.
2. CA RT-DETR: A novel integration of a Color Attention block into the RT-DETR architecture, significantly boosting detection performance for chromatic-specific objects.
3. 360° System Integration: A complete pipeline combining fisheye multi-camera hardware, geometric calibration for directional estimation, and deep learning inference.
4. Contextualized Metrics: Introduction of Bulb Array and Vehicle-level metrics (Precision/Recall/F1) that account for the fact that detecting any light on a vehicle counts as a successful vehicle detection.

4. Results

A. Model Performance (Dataset Evaluation)

• Baseline Comparison: RT-DETR outperformed YOLO variants and two-stage detectors (Faster R-CNN) in recall, a critical factor for safety.
• CA RT-DETR Improvements:
  • Precision: Increased from 85.0% (RT-DETR) to 88.8%.
  • Recall: Increased from 82.8% to 86.4%.
  • mAP@0.5: Improved from 89.1% to 91.6%.
  • Vehicle-Level Detection: At a confidence threshold of 0.6, the model achieved 94.9% Precision and 94.1% Recall for detecting entire emergency vehicles.
  • Error Reduction: Compared to the baseline, CA RT-DETR reduced False Negatives by 35.2% and False Positives by 23.1%.

B. Field Test Results

Tests were conducted with a stationary camera vehicle and a moving EV (or vice versa) under day and night conditions.

• Detection Range:
  • Daytime: Reliable detection up to 30 meters (100% DR), dropping to ~23% at 50m.
  • Nighttime: Significantly better performance due to contrast; reliable detection up to 50 meters (99.2% DR) and partial detection up to 70 meters.
• Azimuthal Accuracy: The system estimated the approach angle with an average error of < 2.5° across most distances.
• Dynamic Scenarios: In dynamic tests (simulating overtaking and turns), the system maintained a Detection Rate (DR) of >91% during the day and >99% at night.

5. Significance and Conclusion

This research presents a robust, real-time solution for emergency vehicle detection that overcomes the limitations of whole-vehicle detection and audio-only systems.

• Safety Impact: By detecting the light rather than the vehicle, the system achieves higher recall and earlier detection, crucial for preventing collisions.
• Practicality: The system is designed for edge computing (low inference time ~4.1ms) and integrates seamlessly with existing automotive 360° camera architectures.
• Community Contribution: The release of the ABLDataset allows other researchers to benchmark and improve emergency vehicle detection algorithms.
• Future Outlook: The authors position this system as a foundational component for multimodal ADAS, capable of fusing visual data with acoustic sensors for even greater reliability in autonomous and semi-autonomous driving scenarios.