Benchmarking Vision-Based Object Tracking for USVs in Complex Maritime Environments

This study proposes and validates a vision-guided object-tracking framework for unmanned surface vehicles (USVs) in complex maritime environments by benchmarking seven deep learning-based trackers and control algorithms, ultimately identifying the Transformer-based SeqTrack and Linear Quadratic Regulator (LQR) controller as the most robust solution for stable tracking under adverse conditions.

The Gist

Imagine you are trying to teach a small, autonomous boat (a USV) to follow another boat across the ocean, like a puppy chasing a ball. But here's the catch: the ocean is messy. The waves make the camera shake, the sun creates blinding reflections, and sometimes, a sudden dust storm or a splash of water covers the lens, making everything blurry.

This paper is essentially a big "try-out" session to figure out which "eyes" (vision software) and which "brain" (control system) work best for this boat when the going gets tough.

Here is the breakdown of their adventure, explained simply:

1. The Problem: Why is this hard?

Usually, boats use expensive radar to see things. But radar is heavy, eats up a lot of battery, and can't see small objects well. The researchers wanted to use cameras instead because they are cheaper and lighter.

However, cameras have a weakness: they get confused easily.

• The Shake: The boat bobs up and down on waves, so the camera isn't steady.
• The Blur: Water splashes on the lens, or the sun glares, making the target look like a smudge.
• The Chase: The target boat moves, and the camera boat has to react instantly.

If the software gets confused, the boat might crash or lose the target.

2. The Solution: A Three-Part Team

The researchers built a system with three main parts, like a relay race team:

• The Eyes (Perception): This is the camera and sensors (like LiDAR, which is like a bat using sound to see distance). It looks at the world and says, "Hey, there's a boat over there!"
• The Brain (Guidance): This is the Tracker. It takes the blurry, shaky video and tries to lock onto the target boat. It asks, "Is that the boat? Where is it going? Is it hiding behind a wave?"
• The Muscles (Control): This is the Controller. Once the Brain says, "The boat is to the left!", the Controller tells the boat's engines (thrusters) to turn left or speed up.

3. The Great "Try-Out" (Benchmarking)

The researchers didn't just guess which software was best. They put seven different "Trackers" (the Brain) and three different "Controllers" (the Muscles) through a rigorous test.

The Trackers (The Eyes):
They tested advanced AI models. Some were old-school (like SiamFC), and some were brand new, using "Transformers" (a fancy type of AI that pays attention to the whole picture, not just parts).

• The Analogy: Imagine asking seven different people to find a friend in a crowded, foggy room.
  • Some people get distracted by shadows (bad in storms).
  • Some people are slow.
  • SeqTrack (the winner) was like a super-focused detective who could still spot the friend even when the fog rolled in.

The Controllers (The Muscles):
They tested three ways to steer the boat:

1. PID: Like a driver who overreacts. "Oh, we're too far left!" Turns hard right. "Oh, too far right!" Turns hard left. This causes the boat to wobble and shake.
2. SMC: A very strong driver who fights the wind, but the steering feels a bit jerky and noisy.
3. LQR (The Winner): Like a smooth, experienced captain. This system calculates the perfect amount of force needed to turn, avoiding jerky movements. It keeps the boat gliding smoothly even when the waves are rough.

4. The Results: Who Won?

They tested these systems in two ways:

1. In a Video Game (Simulation): They created a virtual ocean with wind, waves, and even dust storms to see how the software handled bad weather.
2. In Real Life: They took the boat out to the waters near Saadiyat Island, UAE, and actually chased a motorboat.

The Winners:

• Best Tracker: SeqTrack. It was the only one that didn't lose the target when the "dust storm" (simulated bad weather) hit. It handled the blurry, messy images better than the others.
• Best Controller: LQR. It kept the boat moving smoothly without shaking or wobbling, even when the sensors were confused by the noise.

5. The Big Takeaway

The paper concludes that if you want a robot boat to chase something in a messy, real-world ocean, you shouldn't use the old, simple methods.

Instead, you should pair the SeqTrack (the smart, focused AI eyes) with the LQR (the smooth, calm captain). Together, they form a team that can handle the chaos of the sea, keeping the boat on target even when the weather tries to blind it.

In short: They found the perfect "puppy and trainer" combo that won't get distracted by the ocean's tricks.

Technical Summary

1. Problem Statement

The paper addresses the critical challenge of real-time autonomous target tracking for Unmanned Surface Vehicles (USVs) in complex maritime environments. While USVs are increasingly used for search and rescue, security, and monitoring, existing tracking systems face significant hurdles:

• Dynamic Conditions: Maritime environments involve dynamic camera motion due to vessel movement and wave action, causing target scale variation, occlusion, and blur.
• Limitations of Current Methods: Traditional approaches often rely on object detectors (e.g., YOLO, SSD) combined with filtering techniques (e.g., Kalman Filters). These methods struggle with missed detections, lack robustness against camera motion, and fail to maintain continuous locks in adverse weather (e.g., dust storms, splashes).
• Control Integration Gap: There is a lack of comprehensive frameworks that integrate advanced deep learning trackers with low-level control systems to ensure stable navigation under sensor noise and environmental disturbances.

2. Methodology

The authors propose a vision-guided object-tracking framework consisting of three main modules: Perception, Guidance, and Control.

A. System Architecture & Hardware

• USV Platform: A catamaran-style USV equipped with a Hesai Panda 32 LiDAR, four 360-degree cameras, an IMU, and a DVL. It utilizes a dual-computing architecture (NVIDIA Xavier NX for sensor fusion, Jetson Orin for heavy vision/control tasks).
• Perception Module: Uses onboard cameras for target detection and LiDAR for distance estimation and obstacle avoidance.
• Guidance Module: Processes visual data to generate navigation commands. It calculates pixel error (target position relative to image center) and distance error (relative to LiDAR range) to guide the USV.
• Control Module: Translates guidance commands into thruster forces (Surge and Yaw) using three distinct control algorithms:
  1. PID (Proportional-Integral-Derivative): Standard feedback control.
  2. SMC (Sliding Mode Control): Robust nonlinear control for handling disturbances.
  3. LQR (Linear Quadratic Regulator): Optimal control minimizing a cost function balancing state deviation and control effort.

B. Vision-Based Tracking Benchmarking

The study benchmarks six state-of-the-art trackers utilizing different deep learning architectures:

1. SiamFC: Siamese network (Convolutional).
2. ATOM: Correlation filter-based with IoU prediction.
3. DiMP: Discriminative model with iterative optimization.
4. ToMP: Transformer-based predictor.
5. TaMOs: Transformer-based with Feature Pyramid Network.
6. SeqTrack: Transformer-based encoder-decoder treating tracking as a sequence generation task.

C. Datasets

• Real-World Dataset: 21 videos (12 training, 9 testing) captured off Saadiyat Island, UAE, featuring calm seas and moderate waves.
• Simulation Dataset: Generated using the MBZIRC simulator, including controlled environments and synthetic dust storms to test robustness under low visibility.

3. Key Contributions

1. Comprehensive Benchmarking: The first extensive evaluation of advanced deep learning trackers (Siamese, Transformer-based) specifically tailored for the dynamic, noisy conditions of maritime USV operations.
2. Integrated Framework: A novel framework that seamlessly couples high-level vision trackers with low-level control systems (PID, SMC, LQR), analyzing how tracker performance impacts control stability.
3. Validation in Dual Environments: Rigorous testing in both simulated environments (including dust storms) and real-world sea trials, providing a realistic assessment of system robustness.
4. Identification of Optimal Pairing: Determining the most effective combination of tracker and controller for adverse conditions.

4. Results

Tracker Performance

• Simulation: SeqTrack and ToMP achieved the highest Area Under Curve (AUC) scores (90.10% and 88.95%, respectively).
• Real-World: ToMP ranked highest in AUC (76.07%), followed closely by TaMOs and SeqTrack.
• Adverse Conditions: In dust storm simulations (low visibility), SeqTrack outperformed all other trackers, demonstrating superior robustness against image degradation and blur. Traditional trackers like SiamFC performed poorly in these conditions.

Control Performance

• PID: Showed fast initial response but suffered from significant overshoot and oscillations.
• SMC: Exhibited smoother behavior than PID but had slower convergence.
• LQR: Demonstrated the most robust and smooth control. It minimized oscillations, provided stable thrust commands, and maintained the USV's trajectory effectively even when sensor data was noisy or degraded.

Combined System Performance

• The combination of SeqTrack (for vision) and LQR (for control) yielded the best overall performance.
• In calm conditions, all tested trackers and controllers performed comparably.
• In harsh conditions (dust storms, waves), the SeqTrack + LQR pair maintained stable tracking, whereas other combinations (e.g., PID or SMC with other trackers) showed instability or loss of lock.

5. Significance and Conclusion

This research provides a critical step forward in autonomous maritime operations by demonstrating that Transformer-based trackers (specifically SeqTrack) combined with optimal control strategies (LQR) can overcome the limitations of traditional vision systems in dynamic marine environments.

• Practical Impact: The proposed framework enables USVs to perform reliable inspection, surveillance, and search-and-rescue missions in adverse weather where human operators or standard systems would fail.
• Trade-off Awareness: The authors acknowledge the computational cost of Transformer-based models and LQR optimization but argue that the gain in stability and accuracy in critical scenarios justifies the resource usage.
• Future Work: The authors plan to expand the dataset to include low-light conditions, high-intensity storms, and complex currents to further validate the system's limits.

In summary, the paper establishes a new benchmark for maritime robotics, proving that integrating advanced deep learning vision with optimal control theory is essential for robust autonomous navigation in the real world.