Technical Summary: Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking (EATrack)

1. Problem Statement

Unmanned Aerial Vehicle (UAV) tracking faces a critical trade-off between computational efficiency and tracking robustness. While Vision Transformers (ViTs) offer superior global modeling capabilities compared to Convolutional Neural Networks (CNNs) and Discriminative Correlation Filters (DCFs), their high computational cost renders them impractical for resource-constrained UAV onboard platforms.

Existing lightweight approaches attempt to resolve this by simplifying network architectures (e.g., pruning layers, dropping tokens, or adaptive computation). However, the paper argues that such structural simplifications often weaken the discriminative capacity of target representations, leading to performance degradation in complex scenarios involving rapid motion, occlusion, and appearance variations. Furthermore, prior knowledge distillation methods in tracking often operate at a global level, failing to explicitly leverage spatial priors or focus supervision on the target region, which can result in the student model losing focus on the target or suffering from degraded localization.

2. Methodology

The authors propose EATrack, an efficient and asymmetric UAV tracking framework centered on a teacher-guided dual-branch distillation strategy. The core philosophy is to transfer the rich feature representations and precise localization capabilities of a full-scale "Teacher" ViT to a lightweight "Student" ViT during training, without incurring additional computational costs during inference.

2.1. Architecture

• Teacher Model: A full-scale Vision Transformer (12 layers) that serves as the source of knowledge.
• Student Model: A lightweight, asymmetric ViT (8 layers) designed for real-time inference. It processes template and search frames independently in the first stage (Stage 1) and fuses them in the second stage (Stage 2).
• Inference Pipeline: The student operates independently. To handle temporal variations, a Target Store accumulates reliable target features. Periodically, representative embeddings are sampled from this store to update the template, enabling temporal adaptation with minimal overhead.

2.2. Dual-Branch Distillation Strategy

The training process employs two complementary distillation branches to guide the student model:

1. Spatially Focused Feature-Level Distillation:

   • Goal: To compensate for weakened feature representations caused by pruning.
   • Mechanism: A spatial weight mask, derived from the ground-truth bounding box, is applied to both teacher and student features. This forces the distillation loss (Mean Squared Error) to concentrate on the target region while suppressing background noise.
   • Effect: Enhances the student's ability to learn strong, fine-grained target representations and improves robustness against background clutter.

2. Prediction-Level Distillation:

   • Goal: To enhance spatial localization accuracy.
   • Mechanism: A masked Kullback–Leibler (KL) divergence loss aligns the confidence distributions of the student and teacher specifically within the target region.
   • Effect: Guides the student to replicate the teacher's precise localization behavior and boundary consistency.

2.3. Training and Loss Function

The total training loss combines standard tracking objectives with the distillation terms:
$$L = L_{cls} + \lambda_1 L_1 + \lambda_2 L_{GIoU} + \lambda_3 L_{feat} + \lambda_4 L_{pred}$$
Where $L_{feat}$ and $L_{pred}$ are the feature-level and prediction-level distillation losses, respectively. Crucially, the teacher model and distillation losses are discarded after training; the student model incurs no extra computation during inference.

3. Key Contributions

The paper outlines three primary contributions:

1. Dual-Branch Distillation Framework: A novel design that selectively transfers target-specific knowledge via spatially weighted feature supervision and masked prediction-level alignment. This addresses the limitations of global-only distillation by concentrating supervision within the target region.
2. Fine-Grained Target-Aware Training: A strategy that explicitly guides the model to capture appearance variations by focusing supervision on the foreground. This enhances robustness to subtle and dynamic changes without increasing inference latency.
3. Asymmetric Lightweight Framework: The integration of an asymmetric backbone with a temporal enhancement mechanism (Target Store) creates a framework that balances tracking accuracy and computational efficiency, allowing pruned transformers to maintain competitive accuracy.

4. Experimental Results

The authors evaluated EATrack on five UAV benchmarks: DTB70, UAVDT, VisDrone2018, UAV123, and UAV123@10fps.

• Performance: EATrack-DeiT (using a DeiT-tiny backbone) achieved state-of-the-art (SOTA) results on all five datasets, outperforming previous lightweight trackers like Aba-ViTrack and SGLATrack. For instance, on UAV123, it achieved a success rate of 68.1%, surpassing the previous best by a significant margin.
• Efficiency: The model runs at 33.6 FPS on an embedded NVIDIA Jetson TX2 platform without acceleration tools (e.g., TensorRT), demonstrating real-time capability. On a high-end GPU (A100), it reaches 241.9 FPS.
• Ablation Studies:
  • Distillation: Removing distillation resulted in a drop of 1.4% in precision and 1.1% in success rate on UAV123. Both feature-level and prediction-level branches were shown to be complementary.
  • Temporal Cues: Incorporating temporal cues improved performance significantly on dynamic datasets (e.g., +3.2% on DTB70, +5.8% on VisDrone), confirming the value of temporal modeling in complex motion scenarios.
  • Depth Analysis: The study identified an optimal balance between layer depth and efficiency, selecting a configuration that maximizes performance while maintaining the lightweight objective.

5. Significance and Claims

The paper claims that EATrack effectively bridges the gap between efficiency and accuracy in UAV tracking. By leveraging a teacher-guided, target-aware distillation strategy, the method allows structurally pruned transformers to preserve the discriminative and localization capabilities of larger models without the associated inference overhead.

The authors emphasize that their approach distinguishes itself from existing distillation-based trackers by:

• Moving beyond global guidance to fine-grained, target-aware supervision.
• Combining feature-level and prediction-level guidance to address both representation quality and localization precision simultaneously.
• Achieving robustness against appearance variations through a temporal adaptation mechanism that operates at ultra-low cost.

The results suggest that EATrack is a viable solution for real-time aerial applications where both high tracking accuracy and strict computational constraints are required.