UAVDB: Point-Guided Masks for UAV Detection and Segmentation

1. Problem Statement

Accurate detection and segmentation of Unmanned Aerial Vehicles (UAVs) are critical for surveillance and security. However, existing datasets face significant limitations:

• Scale and Resolution: Many datasets lack high-resolution temporal data or downscale images, making it difficult to detect tiny, distant UAVs (often near single-pixel size).
• Annotation Cost: Manual annotation of small, moving objects is time-consuming and prone to noise.
• Weak Supervision Gaps: Existing point-guided methods often rely on complex pipelines, require retraining, or fail to generalize to small objects due to spatial noise in trajectory points.
• Environmental Diversity: Current datasets often feature large UAVs from short ranges, failing to represent real-world long-range surveillance scenarios with occlusion and clutter.

2. Methodology

The authors propose UAVDB, a benchmark dataset constructed via a fully automated, point-guided weak supervision pipeline. The core innovation is a two-stage process that converts raw trajectory points into high-quality bounding boxes and segmentation masks without manual labeling.

A. Patch Intensity Convergence (PIC)

PIC is a lightweight, training-free algorithm designed to convert 2D trajectory points into precise bounding boxes. It operates via an adaptive inward-outward expansion strategy:

1. Initialization: A small square patch is centered at the trajectory point $(x_0, y_0)$.
2. Iterative Expansion: The patch expands outward by a fixed step size $\delta$ in all directions.
3. Patch Intensity Computation: The mean pixel intensity ($\mu_t$) of the current patch is calculated.
4. Convergence Assessment: Expansion halts when the change in mean intensity between consecutive iterations falls below a threshold $\epsilon$ ($|\mu_{t+1} - \mu_t| < \epsilon$).
   • Logic: As the box expands beyond the UAV, the inclusion of background pixels dilutes the intensity, signaling the boundary.
   • Parameters: The paper uses $w_0=h_0=8$, $\delta=5$, and $\epsilon=4$.

B. Mask Generation with SAM2

To generate instance segmentation masks, the bounding boxes produced by PIC are used as box prompts for SAM2 (Segment Anything Model 2).

• Why Box Prompts? Direct point prompts from trajectory data are often spatially noisy (due to motion blur or annotation errors), leading to unstable masks for small objects. PIC-generated boxes provide a spatially constrained prior, guiding SAM2 to focus on the relevant region.
• Output: This pipeline transforms trajectory data into precise instance masks, creating a multi-task dataset (detection + segmentation).

C. Dataset Construction (UAVDB)

• Source: Built upon the multi-view drone tracking dataset [26], utilizing high-resolution RGB videos (up to 4K) from fixed ground cameras.
• Scale: Covers diverse conditions, from clearly visible UAVs to near single-pixel instances.
• Composition:
  • Training: 10,763 images.
  • Validation: 2,720 images.
  • Test: 4,578 images.
  • Unseen Scenario: Dataset 5 (from source [26]) is excluded from training to test generalization.

3. Key Contributions

1. UAVDB Benchmark: A high-resolution, multiscale dataset specifically designed for long-range UAV detection and segmentation, addressing the lack of small-object data in existing benchmarks.
2. PIC Method: A novel, training-free annotation technique that converts trajectory points into bounding boxes with high IoU (Intersection over Union) and negligible computational overhead.
3. Automated Pipeline: A plug-and-play workflow (PIC + SAM2) that eliminates manual labeling while preserving spatial precision, enabling scalable dataset creation.
4. Comprehensive Benchmarking: Extensive evaluation of YOLO-series detectors (from YOLOv8 to YOLO26) on UAVDB, establishing new baselines for the community.

4. Experimental Results

Annotation Accuracy & Efficiency

• Performance: The proposed PIC method achieved an mIoU of 0.462 for bounding box generation, outperforming fixed-size boxes (0.278), image thresholding (0.316), and GrabCut (0.425).
• SAM2 Integration: Using PIC boxes to prompt SAM2.1-Large yielded the highest mIoU (0.582) for mask generation, significantly outperforming direct point prompting (0.244).
• Efficiency: PIC takes only 0.007 seconds per image, compared to 19 seconds for manual annotation.

Detection & Segmentation Benchmarks

The authors fine-tuned YOLO models on UAVDB:

• Object Detection: YOLO26x achieved the highest $AP_{50}$ on the test set (0.775), while YOLO12l offered a strong balance of speed and accuracy.
• Instance Segmentation: YOLO26n-seg and YOLO12n-seg demonstrated strong capabilities, with mask $AP_{50}$ scores reaching 0.562 and 0.569 respectively on the test set.
• Generalization: Models trained on the dataset successfully detected and tracked small, blurry UAVs in the unseen Dataset 5, demonstrating robustness to domain shifts and varying scales.

5. Significance and Impact

• Scalability: The PIC-SAM2 pipeline allows researchers to generate massive, high-quality datasets from existing trajectory data without manual effort, solving the "data bottleneck" for small object detection.
• Real-World Applicability: By focusing on long-range, low-resolution UAVs in complex environments, UAVDB better reflects real-world surveillance challenges than previous ground-target or short-range datasets.
• Versatility: The video-based nature of the dataset supports temporal tasks like tracking, moving beyond static image detection.
• Future Research: The work establishes a new standard for weakly supervised computer vision, proving that simple intensity-based heuristics combined with foundation models can outperform complex learning-based annotation strategies.