Adaptive Entropy-Driven Sensor Selection in a Camera-LiDAR Particle Filter for Single-Vessel Tracking

This paper presents an adaptive entropy-driven sensor selection policy within a camera-LiDAR particle filter that dynamically switches between modalities to optimize tracking accuracy and continuity for single-vessel surveillance, validated through real-world maritime deployment.

The Gist

Imagine you are trying to keep a perfect eye on a small, fast-moving speedboat in a busy harbor. You have two different tools to help you: a high-tech camera and a laser scanner (LiDAR).

The problem is that neither tool is perfect on its own:

• The Camera is like a human eye. It's great at seeing details and working during the day, but it gets confused by fog, rain, darkness, or if the sun reflects off the water (glare).
• The Laser Scanner is like a bat using echolocation. It measures exact distances and works in the dark or fog, but it gets "blind" if the boat is too far away or if the signal bounces off too many things.

This paper describes a smart system that combines these two tools to track the boat perfectly, even when the weather is bad or the boat moves far away.

The "Smart Switch" (The Core Idea)

Most systems try to use both the camera and the laser all the time, which is like trying to listen to two radio stations at once while driving—it's noisy and wastes energy.

Instead, the researchers built a "Smart Switch" based on a concept called Entropy (which is a fancy word for "confusion" or "uncertainty").

Think of it like this:
Imagine you are playing a guessing game. You have a hunch about where the boat is.

1. The Question: "Which tool will help me guess the boat's location better right now?"
2. The Test: The system quickly simulates: "If I look with the camera, how much clearer will the picture be? If I look with the laser, how much clearer will the distance be?"
3. The Decision: It picks the tool that reduces the "confusion" the most.
   • If the boat is close and the sun is shining, the system says, "The laser is super clear right now, let's use that!"
   • If the boat is far away or it's raining, the system says, "The laser is blurry, but the camera can still see it. Let's switch to the camera!"

How It Works (The "Particle Filter")

To track the boat, the system uses a method called a Particle Filter. Imagine you are trying to guess where a friend is hiding in a large park.

• You don't just guess one spot. Instead, you imagine 1,000 tiny ghosts (particles) scattered around the park, each representing a possible location of the boat.
• When the boat moves, all 1,000 ghosts move with it.
• When the "Smart Switch" picks a sensor (camera or laser), it checks the data.
  • If the sensor says, "The boat is definitely near the dock," the ghosts near the dock get a high score (they become "heavy"). The ghosts far away get a low score (they become "light").
  • The system then "resamples," keeping mostly the heavy ghosts and throwing away the light ones.
• The final answer is the average location of the remaining heavy ghosts.

The Real-World Test

The team tested this on a real marina in Cyprus with a rigid inflatable boat (RIB) that had a GPS tracker (the "truth") to see how well they did. They compared four scenarios:

1. Laser Only: Great when the boat is close, but loses it when it gets far.
2. Camera Only: Keeps the boat in sight far away, but the position is a bit fuzzy.
3. Both Together: Good, but sometimes the "noisy" camera data messes up the "precise" laser data.
4. The Adaptive (Smart Switch) System: This was the winner. It used the laser when the boat was close (for precision) and automatically switched to the camera when the boat went far (for continuity).

Why This Matters

This isn't just about tracking one boat. It's about efficiency and safety.

• Saving Power: By not processing both video and laser data 100% of the time, the system saves battery and computer power.
• Doing More: Because the system only uses one sensor at a time, the "unused" sensor can be turned to look at something else, like tracking a second boat or scanning for debris.
• Reliability: It ensures that even if one sensor fails (like a camera getting blinded by fog), the system knows to switch to the other one immediately, so you never lose track of the target.

In short: The paper presents a "smart manager" for sensors that knows exactly which tool to use at the right moment, ensuring the boat is tracked accurately from start to finish without wasting energy.

Technical Summary

Here is a detailed technical summary of the paper "Adaptive Entropy-Driven Sensor Selection in a Camera-LiDAR Particle Filter for Single-Vessel Tracking."

1. Problem Statement

Robust tracking of surface vessels from fixed coastal platforms is hindered by the inherent limitations of individual sensing modalities in dynamic marine environments:

• Optical Cameras: Provide rich visual data but suffer from performance degradation due to illumination changes (night), weather (fog, rain), and visual clutter (waves, foam).
• LiDAR: Offers precise, light-invariant 3D geometry but suffers from performance drops at long ranges, in fog, and with intermittent returns.
• The Challenge: Existing multi-sensor fusion systems often process all sensor streams continuously, which is computationally expensive and does not dynamically adapt to the reliability of specific sensors in real-time. The goal is to achieve continuous, accurate tracking while optimizing resource usage by selecting the most informative sensor at any given moment.

2. Methodology

The authors propose a heterogeneous multi-sensor fusion framework centered around a Particle Filter (PF) with an Adaptive Entropy-Driven Sensing Policy.

A. System Architecture & Calibration

• Sensors: A shore-mounted Ouster OS2 3D LiDAR (130m range) and an elevated AXIS Q8752-E RGB camera.
• Ground Truth: A Rigid Inflatable Boat (RIB) equipped with a GNSS receiver.
• Calibration:
  • Spatial: Camera pixels are mapped to a 2D world frame (sea surface) using a planar homography ($H_{C \to W}$). LiDAR points are transformed via a 2D rigid/similarity transform.
  • Temporal: Sensor clocks are aligned to a common GNSS timeline using piecewise affine warping based on manually selected anchor events.

B. Detection Pipelines

• Camera: Uses MOG2 background subtraction restricted to a sea mask, followed by morphological operations. To reduce false positives from waves/glare, a secondary HOG-SVM classifier filters candidate bounding boxes.
• LiDAR: Uses reflectivity and range-change masks on the destaggered LiDAR scan. Clusters are converted into polar measurements (median range, circular mean bearing) to create robust point estimates.

C. The Particle Filter (PF)

The state vector is $x_k = [p_x, p_y, v_x, v_y]^\top$. The filter operates in three stages:

1. Prediction: Uses a Constant Velocity (CV) motion model with process noise.
2. Update: Computes particle weights based on likelihoods from selected sensors. The likelihood models account for target-origin vs. clutter-origin hypotheses (mixture models) to handle false detections.
3. Resampling: Systematic resampling is triggered when the Effective Sample Size (ESS) drops below a threshold to prevent weight degeneracy.

D. Adaptive Sensor Selection (Key Innovation)

Instead of fusing all available data continuously, the system selects the single most informative sensor at each time bin using an Information Gain (IG) criterion:

1. Entropy Calculation: Calculate the Shannon entropy ($H$) of the predicted particle weights (prior).
2. Hypothetical Update: Simulate an update for each candidate sensor (Camera and LiDAR) using a "virtual measurement" derived from the predicted state.
3. Selection: Compute the Information Gain ($I_s = H_{prior} - H_{post}^{(s)}$) for each sensor. The sensor yielding the maximum entropy reduction is selected for the actual update.
   • Logic: If LiDAR is close and clear, it reduces uncertainty more than the camera. If LiDAR is out of range or occluded, the camera provides higher information gain.

3. Key Contributions

1. Unified Adaptive Framework: A particle filter that dynamically switches between LiDAR-only, Camera-only, or fused updates based on real-time information gain, rather than static fusion rules.
2. Entropy-Driven Policy: Implementation of an information-theoretic approach to sensor management in a real-world maritime setting, moving beyond simulation-based studies.
3. Resource Efficiency: The adaptive policy avoids continuous multi-stream processing, allowing non-selected sensors to be retasked (e.g., for other targets or archival) or powered down, reducing computational and bandwidth loads.
4. Real-World Validation: Extensive testing on the CMMI Smart Marina Testbed (Cyprus) with ground-truth GNSS data, covering diverse zones (near-field, mid-field, far-field).

4. Experimental Results

The system was evaluated across three zones defined by distance from the marina:

• Zone 1 (Near-field, <325m):
  • LiDAR-only achieved the lowest RMSE (highest accuracy) due to strong geometric constraints.
  • Adaptive matched LiDAR-only performance, successfully selecting LiDAR when available.
  • Camera-only and All Sensors showed higher errors due to projection noise and visual clutter.
• Zone 2 (Mid-field, 325m–650m):
  • LiDAR-only suffered from high track loss (~80% lost bins) as the target approached the range limit.
  • Adaptive and Camera-only maintained continuity. The adaptive policy successfully switched to the camera when LiDAR became unreliable.
• Zone 3 (Far-field, >650m):
  • LiDAR-only failed completely (100% track loss).
  • Adaptive and Camera-only maintained tracking with similar RMSE, though higher than in near-field zones.

Conclusion on Performance: The Adaptive configuration achieved the best overall trade-off, approximating the performance of the "best fixed sensor" in each specific zone (LiDAR in Zone 1, Camera in Zones 2–3) while maintaining continuous tracking.

5. Significance

• Resilience: The system demonstrates that information-driven selection can maintain tracking continuity even when one modality fails or degrades, a critical requirement for maritime security and collision avoidance.
• Operational Efficiency: By selecting only the most informative sensor per time step, the system reduces the computational burden of processing redundant data streams. This enables "retasking" of sensors for secondary objectives (e.g., monitoring multiple vessels simultaneously).
• Scalability: The modular framework is designed to be extended to richer heterogeneous suites (thermal cameras, radar, acoustic arrays) and future work aims to transition from single-target to multi-target tracking with robust data association.

In summary, this paper validates that entropy-driven adaptive sensing is a practical and effective strategy for maritime surveillance, offering a robust balance between tracking accuracy, continuity, and resource management in real-world, variable conditions.