💻 computer science

Edge-Aware Thermal Infrared UAV Swarm Tracking

This paper proposes an edge-aware online tracking pipeline centered on an Adaptive Kinematic Kalman Filter (AKKF) to enhance the real-time, robust tracking of tiny UAVs in thermal infrared imagery by balancing computational efficiency with improved handling of dynamic motion and occlusions.

Original authors: Yu-Hsi Chen

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu-Hsi Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to film a swarm of tiny, glowing fireflies flying through a thick, foggy night. You want to keep your camera locked onto every single one of them, even when they zip around fast, hide behind trees, or when your camera shakes a little. This is exactly the challenge of tracking Unmanned Aerial Vehicles (UAVs) using Thermal Infrared (TIR) cameras, which see heat instead of color.

The problem is that most current tracking systems are like a super-smart but slow librarian who reads every book in the library before deciding where to put a new one. They are accurate, but they are too heavy to run on the small, portable computers (edge devices) that drones actually carry. On the other hand, the old, simple tracking method is like a guess-and-check game: it assumes the fireflies move at a steady, boring speed. But real drones are chaotic; they zoom, stop, and turn instantly. When they do, the simple guess fails, and the tracker loses them.

The Paper's Big Idea: The "Smart Damping" Filter
The authors, Yu-Hsi Chen from the University of Melbourne, propose a new way to track these drones called the Adaptive Kinematic Kalman Filter (AKKF). Think of this not as a new camera, but as a smarter "brain" for the tracking software.

Instead of just guessing where the drone will be, this new brain uses pre-programmed rules to "dampen" or slow down its predictions when things get weird.

The "Damping" Trick: Imagine a skateboarder. If they are huge and heavy (a big drone), they keep their momentum. But if they are tiny and light, or if they seem to be disappearing behind a bush, the new filter says, "Whoa, slow down!" It automatically reduces the predicted speed based on the target's size so the tracker doesn't fly off into the distance just because the drone stopped.
The "Braking" Trick: If a drone flies away from the camera, it gets smaller. The old system might think, "It's getting smaller, so it must be shrinking!" and eventually predict it vanishes into nothing. The new system puts on the brakes, realizing, "Ah, it's just getting far away," and stops the size from collapsing by applying a fixed rule based on the target's scale.
The "Noise" Filter: Sometimes, the thermal camera sees a weird blob that looks like a drone but isn't. The new system checks the shape. If the shape looks weird compared to what it expected, it treats that blob as "noisy" and ignores it, trusting its own prediction instead.

What They Ruled Out
The paper notes that while complex, heavy math models like the Extended Kalman Filter (EKF) offer more flexibility, they are rarely used in this specific job because the computational cost is simply too high for tracking multiple targets in real-time. Instead of choosing between the "too slow" complex models and the "too simple" constant-velocity guesses, the authors found a middle ground: a smarter version of the standard Kalman Filter that adapts to the situation using specific formulas without breaking the bank on computing power.

The Results: A Balancing Act
The team tested their new system on a benchmark called the Beyond Strong Baseline (BSB), which uses real-world data of tiny drones. They didn't just look at accuracy; they looked at how fast it ran on a laptop with a 6GB memory GPU, simulating a real drone's computer.

The Trade-off: They found that adding their smart "damping" and "braking" features made the system slightly slower, dropping the speed from 132.03 FPS (frames per second) to 94.92 FPS. However, this small drop in speed was worth it because the tracking became much more reliable.
The "Coasting" Strategy: When the camera briefly loses sight of a drone (maybe due to a glitch or a tree blocking the view), the system doesn't panic. It uses "predictive coasting," essentially saying, "I know where it was, and I know how it moves, so I'll guess where it is for one extra frame." This tiny trick helped keep the tracking lines unbroken.
The Paradox: Interestingly, the paper suggests that having a super-accurate detector (one that finds the drone perfectly) doesn't always mean the tracker works better. Sometimes, a slightly less accurate detector combined with this smart tracking brain works best.

How Sure Are They?
The authors are confident in their findings based on extensive experiments on the BSB benchmark. They measured the results using a metric called HOTA (a score for tracking accuracy). For example, their best setup using the RF-DETR Nano model at 1280 resolution achieved a HOTA score of 0.8437.

They also noted that while their method is a strong starting point for edge deployment, it is not a magic bullet. They suggest that for even better results in the future, we might need to shrink the computer models themselves (model pruning) rather than just making the math smarter.

In short, this paper suggests that by teaching the tracker to be a little more cautious and adaptive—like a driver who slows down when the road gets foggy—we can keep our eyes on the tiny, fast-moving drones even when the world gets messy and the computer power is limited.

Technical Summary: Edge-Aware Thermal Infrared UAV Swarm Tracking

Problem Statement

Thermal infrared (TIR) imaging is critical for Unmanned Aerial Vehicle (UAV) swarm operations in visually degraded environments, such as nighttime or adverse weather, where RGB cameras fail. However, real-time multi-object tracking (MOT) of tiny UAVs on resource-constrained edge devices faces significant challenges. These include limited appearance cues, frequent occlusions, rapid maneuvers, and thermal sensor jitter.

While existing methods, often driven by benchmarks like the Anti-UAV challenge, have achieved high accuracy, they frequently overlook the computational constraints required for real-time edge deployment. The standard Kalman Filter (KF), widely used for its efficiency, relies on constant-velocity assumptions that often fail under highly dynamic UAV motion and sensor noise. Conversely, more sophisticated nonlinear estimators improve robustness but introduce prohibitive computational costs. There is a lack of solutions that effectively balance tracking robustness with the efficiency needed for edge deployment.

Methodology

The authors propose an edge-aware online tracking pipeline centered on the Adaptive Kinematic Kalman Filter (AKKF). This approach augments the standard linear KF with state-dependent kinematic modeling to handle dynamic motion while preserving real-time efficiency. The pipeline integrates three core components:

1. Adaptive Kinematic Kalman Filter (AKKF)

The AKKF modifies the standard KF transition ( $F_k$ ) and measurement ( $R_k$ ) matrices to be state-dependent, addressing specific failure modes in TIR UAV tracking:

Area-Adaptive Motion Damping (AAMD): To prevent trajectory drift during prolonged occlusions, the filter dynamically dampens positional velocity based on the target's bounding box area. Inspired by aerodynamic models, this reduces velocity retention when the target is small or occluded, preventing the propagation of stale velocity estimates.
Perspective-Aware Velocity Braking (PAVB): To avoid scale collapse when a UAV recedes from the camera (resulting in negative scale velocities), the filter dynamically suppresses scale velocity based on the current target scale. This prevents the bounding box from shrinking to zero if detections are temporarily lost.
Dynamic Measurement Noise Estimation (DMNE): Recognizing that TIR clutter can produce geometrically inconsistent detections, the filter adjusts measurement uncertainty based on detection consistency. By analyzing the deviation between measured and predicted aspect ratios, the system increases the measurement covariance for unreliable detections, allowing the motion prediction to dominate the update step.

2. Transient False Positive Suppression (TFPS)

Standard tracking-by-detection pipelines often output tracklets before temporal confirmation, allowing one-frame false positives to contaminate results. This is particularly problematic in TIR imagery due to thermal clutter. The proposed pipeline restores temporal track confirmation, outputting only confirmed tracklets to suppress transient false positives.

3. Kinematics-Guided Predictive Coasting

To maintain trajectory continuity during temporary observation gaps caused by sensor jitter, motion blur, or occlusion, the pipeline employs a predictive coasting strategy. When no valid detection is associated with a frame, the measurement update is skipped, and the state is propagated directly using the kinematic model. Experiments indicate that a single frame of coasting yields optimal performance.

Key Contributions

Adaptive Kinematic Kalman Filter: A novel extension of the standard KF that incorporates state-dependent motion damping, velocity constraints, and measurement uncertainty estimation, enabling robust tracking without sacrificing real-time efficiency.
Edge-Aware Online Tracking Pipeline: An integrated system combining AKKF with TFPS and predictive coasting, specifically tailored for the challenges of TIR UAV tracking.
Evaluation on the BSB Benchmark: Extensive experiments on the Beyond Strong Baseline (BSB) benchmark, which provides a platform for jointly evaluating tracking performance (HOTA) and computational efficiency (FPS), offering a practical reference for edge-aware deployment.

Experimental Results

The system was evaluated using the BSB benchmark, which includes diverse TIR UAV sequences with varying resolutions and target sizes.

Performance vs. Efficiency: The pipeline was tested with various detectors (YOLOv12, YOLO26, RF-DETR) and configurations. While the RF-DETR Nano at 1280 resolution achieved the highest HOTA (0.8437), the study highlights "edge-aware candidates" (e.g., RF-DETR Nano at 640, YOLO26n at 640) that offer a favorable trade-off between accuracy and throughput.
Ablation Study: The full AKKF configuration improved all tracking metrics over the baseline while reducing frame rate from 132.03 FPS to 94.92 FPS, demonstrating that the added kinematic modeling incurs a manageable computational cost for significant robustness gains.
Detection-Tracking Paradox: The results reinforce the observation that higher detection accuracy does not always translate to better tracking performance, particularly for tiny objects where temporal modeling is crucial.
Resolution Sensitivity: For tiny UAV targets, increasing input resolution consistently improved performance, suggesting that further resolution reduction may severely degrade tracking quality.

Significance and Claims

The paper claims to provide a starting point for edge-aware UAV tracking by explicitly addressing the trade-off between accuracy and computational efficiency. The proposed pipeline demonstrates that it is possible to move beyond the rigid assumptions of the standard KF without resorting to computationally expensive nonlinear estimators.

The authors emphasize that their work offers insights for future real-time deployment, particularly in scenarios where appearance cues are limited. They modestly conclude that while their method improves trajectory continuity and robustness under challenging TIR conditions, future work should focus on computational optimization (e.g., model pruning) and validation on dedicated edge hardware to fully realize the potential of these systems in resource-constrained environments.