A Multi-modal Detection System for Infrastructure-based Freight Signal Priority

This paper presents the design, deployment, and evaluation of an infrastructure-based multi-modal detection system integrating LiDAR and camera sensors with hybrid perception algorithms to reliably monitor freight vehicle movements for effective Freight Signal Priority applications.

The Gist

Imagine a busy intersection as a chaotic dance floor where cars, trucks, and pedestrians are all trying to move in rhythm. Usually, the traffic lights act like a strict DJ, switching the beat on a fixed timer, regardless of who is on the floor. But big trucks are like heavy dancers; they take a long time to start moving and a long time to stop. If they get stuck in stop-and-go traffic, they waste a lot of fuel and pollute the air.

Freight Signal Priority (FSP) is like giving these heavy dancers a special VIP pass. The goal is to let the traffic light stay green (or turn green sooner) for a truck that is approaching, so it can glide through without stopping.

However, there's a catch: The traffic light system needs to see the truck coming from far away to know when to change the light. If the road curves or if there are hills, the traffic light "eyes" might be blind to the truck until it's too late.

This paper describes a new, high-tech "super-eye" system built by researchers at UC Riverside to solve this problem. Here is how it works, broken down into simple concepts:

1. The "Binocular Vision" Setup

Most traffic cameras are like a person looking through a single telescope. If a building or a curve blocks the view, they can't see what's coming.

The researchers built a two-part system (like giving the intersection a pair of eyes):

• The Main Eye: A powerful sensor (LiDAR and camera) sits right at the intersection, looking down the road.
• The "Spotter" Eye: Because the road curves and hides the view, they installed a second sensor halfway down the block (the "midblock").
• The Walkie-Talkie: These two sensors talk to each other wirelessly. The "Spotter" sees the truck first, shouts a message to the "Main Eye," and together they tell the traffic light, "Hey, a big truck is coming! Get ready!"

2. The "Digital Snow Globe" (How the Sensors Work)

The system uses LiDAR, which is like a flashlight that shoots out millions of invisible laser beams every second. When these beams hit a truck, they bounce back, creating a 3D map of the world made of tiny dots (a "point cloud").

Think of the raw data as a messy pile of snowflakes. The system has to clean this up:

• Straightening the Picture: Since the sensor is mounted on a pole, it might be slightly tilted. The computer mathematically "tilts" the digital snow back to be perfectly level, like leveling a camera on a tripod.
• Cleaning the Clutter: The system ignores the "snow" that is actually just the road, trees, or buildings. It only keeps the "snow" that is moving (the vehicles).
• Grouping the Friends: It groups the moving dots together. If a bunch of dots are moving together, the computer says, "That's one vehicle."

3. The "Truck vs. Car" Detective

Once the system spots a vehicle, it has to decide: Is this a tiny sedan or a massive semi-truck?

The researchers taught the computer to look at the vehicle's "shape profile":

• Height: Trucks are tall. Sedans are short.
• The "Z-Stack": The system looks at how the dots are stacked vertically. A truck has a tall, boxy stack of dots; a car has a flatter, wider stack.
• The Result: If the system sees a tall, boxy shape, it triggers the VIP pass (FSP). If it sees a small shape, it ignores it.

4. The "GPS Map Match"

To know exactly which lane the truck is in, the system has to align its laser map with the real-world GPS map.

• Imagine trying to overlay a transparent drawing of a street onto a real Google Map. If you don't line them up perfectly, the drawing will be in the wrong place.
• The researchers drove a car with a high-precision GPS around the area, recording the path. They then used that path to "calibrate" their laser map, ensuring the digital truck is placed in the exact real-world lane.

5. The Results: Good, but Room to Grow

The team tested this system with 20 different scenarios (trucks, vans, cars).

• The Good News: When the system did spot a truck, it was almost always right (high precision). It rarely gave a false alarm to a regular car.
• The Challenge: Sometimes, it missed a truck that was coming (lower recall). This happened because the truck was partially hidden or the road curve made it hard to see.
• Speed: The system is fast enough to work in real-time, processing data in less than a blink of an eye (under 0.05 seconds).

The Big Picture

This paper is essentially a blueprint for building a smarter, more attentive traffic light system. Instead of guessing when a truck is coming, we are giving the traffic lights "super-vision" to see around curves and detect heavy vehicles early.

By doing this, trucks can keep moving smoothly, saving fuel, reducing pollution, and getting their cargo to its destination faster. While the system isn't perfect yet (it needs to get better at spotting trucks in tricky spots), it proves that using "eyes" on the road is a powerful way to make our highways greener and more efficient.

Technical Summary

1. Problem Statement

Freight Signal Priority (FSP) aims to improve the mobility of heavy-duty trucks at signalized intersections, reducing stop-and-go conditions that lead to high fuel consumption and greenhouse gas emissions. While connected vehicle (V2I/V2V) systems exist, they require onboard equipment, limiting widespread adoption.
The Core Challenge: Existing infrastructure-based sensing solutions struggle with limited visibility caused by roadway curvature and occlusions. Standard roadside sensors often cannot detect approaching freight vehicles early enough to generate timely priority requests. Furthermore, accurately classifying vehicle types (trucks vs. passenger cars) and estimating Time of Arrival (ToA) using only infrastructure sensors in complex terrain remains difficult.

2. Methodology

The authors propose a multi-modal, infrastructure-based detection system integrating LiDAR and cameras, deployed across two distinct subsystems to overcome terrain constraints.

A. System Architecture

• Dual-Subsystem Deployment:
  1. Intersection Subsystem: Located at the main intersection, equipped with a long-range solid-state LiDAR (Livox TELE-15) and a camera to monitor southbound traffic.
  2. Midblock Subsystem: Located further upstream on a curved section of the road to overcome occlusion. It uses a mid-range mechanical LiDAR (Robosense Ruby Plus) and a camera.
• Communication: The two subsystems are connected via a wireless bridge (Ubiquiti AirFiber 5XHD) using Precision Time Protocol (PTP) for time synchronization, allowing midblock detection data to be transmitted to the intersection for priority calculation.
• Hardware: Data is processed locally on edge computers using Power-over-Ethernet (PoE) switches.

B. Data Processing Pipeline

The system employs a hybrid sensing approach combining traditional computer vision techniques with geometric processing:

1. Preprocessing:
   • Cloud Rotation: Corrects roll and pitch angles of the LiDAR to align the Z-axis vertically, ensuring consistent height measurements.
   • Cropping & Downsampling: Removes background clutter via Region of Interest (ROI) cropping and reduces point cloud density using voxel-grid downsampling for real-time efficiency.
   • Background Subtraction: Constructs a dynamic background map to filter out static infrastructure and vegetation, isolating foreground moving objects.
2. Detection & Clustering:
   • Uses DBSCAN clustering on foreground points to group object instances.
   • Extracts features such as centroid, maximum height, and vertical point distribution.
3. Tracking & Classification:
   • Kalman Filter: A 6-dimensional constant-velocity Kalman Filter smooths trajectories and estimates velocity, handling short occlusions.
   • Direction Determination: Uses the dot product between the velocity vector and the line-of-sight vector to classify approaching vs. departing vehicles.
   • Truck Classification: Classifies vehicles based on absolute height, relative height, and Z-axis point distribution. Trucks are distinguished from passenger vehicles by their significantly larger vertical dimensions.
4. Geospatial Calibration:
   • Registers LiDAR point clouds to global geodetic (ENU) coordinates using a two-stage process:
     1. Static Registration: Aligns stationary LiDAR points with GPS data from a reference vehicle.
     2. Dynamic Trajectory Alignment: Refines the transformation by aligning moving vehicle trajectories from LiDAR and GPS logs.

3. Key Contributions

• Complete System Pipeline: Presents a full design, implementation, and evaluation pipeline for an infrastructure-based, multi-modal freight detection system.
• Dual-Subsystem Architecture: Proposes a novel two-site setup (intersection + midblock) specifically designed to address visibility blockage caused by roadway curvature, a common real-world challenge.
• Lane-Level Localization: Successfully registers LiDAR-derived coordinates to geodetic reference frames, enabling precise lane-level vehicle localization without onboard GPS.
• Practical Evaluation Framework: Develops a frame-level evaluation metric focused on FSP triggers (Truck vs. Non-Truck) rather than complex bounding box accuracy, aligning directly with operational needs.

4. Results and Evaluation

The system was evaluated using field data collected at the deployment site.

• Detection Accuracy:
  • Precision: 0.75 (High precision indicates the system rarely triggers false alarms for non-trucks).
  • Recall: 0.50 (Lower recall indicates the system misses some trucks, likely due to conservative thresholds or occlusion).
  • F1 Score: 0.60.
  • Analysis: The system is conservative; it prioritizes avoiding false positives (unnecessary signal changes) over catching every single truck.
• Real-Time Efficiency:
  • The system processes frames in < 0.05 seconds, well within the required 10 Hz (0.1s) real-time constraint, even with varying foreground point densities.
• GPS Calibration Accuracy:
  • Main Intersection: Mean static registration error of 0.21m; Mean trajectory alignment error of 0.35m.
  • Midblock: Higher errors (Mean static: 0.35m; Mean trajectory: 0.89m) due to sparser point clouds at longer ranges and faster vehicle speeds reducing trajectory detail.
  • Despite higher midblock errors, the system achieved lane-level localization accuracy, sufficient for FSP applications.

5. Significance and Future Work

• Significance: This work demonstrates that infrastructure-based sensing can effectively support FSP without requiring expensive onboard truck equipment. It provides a practical solution for complex road geometries where standard single-sensor setups fail. The system offers a viable path toward eco-friendly freight management by enabling early, reliable priority requests.
• Limitations & Future Work:
  • Deep Learning: Current reliance on conventional algorithms was due to a lack of labeled data for solid-state LiDAR in roadside settings. Future work will focus on building a multi-modal dataset (LiDAR + Camera) to train deep learning models (e.g., PointPillars) for better generalization.
  • Sensor Fusion: While cameras are present, their data is currently underutilized. Future iterations will explore camera-LiDAR fusion to improve classification accuracy.
  • Calibration Robustness: Improving calibration by collecting static reference points across wider spatial distributions (planar coverage) rather than collinear paths.

In conclusion, the paper validates a robust, real-time infrastructure sensing system capable of detecting and tracking freight vehicles in challenging environments, providing a foundational step toward scalable, eco-friendly Freight Signal Priority deployments.