Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks

1. Problem Statement

The paper addresses the critical challenges in Multi-Unmanned Aerial Vehicle (UAV) cooperative perception within low-altitude economy networks (e.g., intelligent transportation, urban surveillance).

• The Core Conflict: While cooperative perception using multiple UAVs significantly improves accuracy through complementary multi-view observations, it generates massive volumes of visual data. Transmitting raw images or dense features over wireless links creates severe communication bottlenecks, leading to high latency and resource inefficiency.
• Limitations of Existing Solutions:
  • UAV-centric compression: UAVs are resource-constrained edge devices with limited compute power, making complex onboard compression or feature extraction ineffective.
  • Traditional Models: Conventional Convolutional Neural Networks (CNNs) often lack the representation capability to reconstruct high-quality perception from highly compressed or sparse data.
  • Large Vision Models (LVMs): While LVMs (like Vision Transformers) offer superior generalization and feature extraction, their high computational cost makes them unsuitable for direct deployment on UAVs.
• Objective: To design a framework that balances perception utility (accuracy) and communication efficiency (latency/bandwidth) by offloading heavy computation to the ground while minimizing uplink data transmission.

2. Methodology: The BHU Framework

The authors propose the Base-Station-Helped UAV (BHU) framework, which shifts the heavy lifting of perception to a ground server while utilizing a sparsified transmission strategy from UAVs.

A. System Model

• Communication: Modeled as a Multi-User MIMO (MU-MIMO) uplink system where multiple UAVs transmit to a ground Base Station (BS) simultaneously.
• Channel: A 3D geometry-based wideband Rician channel model is used to account for Line-of-Sight (LoS) and Non-Line-of-Sight (NLoS) components, including Doppler shifts and array responses.
• Optimization Goal: Maximize a weighted utility function (combining Intersection-over-Union and Panoptic Quality) while minimizing the maximum transmission latency.

B. Core Components

1. Top-K Sparsification (UAV Side):

   • Instead of transmitting full images, each UAV uses a lightweight onboard network to generate an importance map.
   • A Top-K selection mechanism identifies the most informative pixels (based on a sliding window of neighbor scores) and transmits only a fraction ($\kappa_u$) of the image data.
   • The ground server reconstructs the sparse image using a learnable Gaussian interpolation module.

2. LVM-Based Perception Pipeline (Ground Server):

   • Encoder: A MaskDINO encoder (based on Swin-Large backbone) processes the reconstructed sparse images. It leverages the self-attention mechanism of Large Vision Models to extract rich, multi-scale features and handle long-range dependencies.
   • BEV Fusion: Features are projected from the camera view to a unified Bird's-Eye-View (BEV) coordinate system using a Lift-Splat-Shoot (LSS) paradigm. Multi-view features from different UAVs are fused via element-wise summation.
   • Prediction: A U-Net decoder performs multi-task prediction, outputting semantic segmentation (vehicle vs. background) and instance flow (motion prediction).

3. DDIM-Based Deep Reinforcement Learning (DRL) Algorithm:

   • Challenge: Jointly optimizing UAV association, Top-K ratios, and precoding matrices is a non-convex, NP-hard mixed-integer problem.
   • Solution: A DRL agent determines the UAV selection and sparsification ratios.
   • Diffusion Integration: To handle the high-dimensional action space of precoding matrices, the authors employ a Denoising Diffusion Implicit Model (DDIM).
     • The DDIM models the conditional distribution of optimal precoding vectors.
     • It uses a reverse denoising process to generate high-quality precoding actions efficiently, avoiding the computational cost of exhaustive search during inference.
   • Reward Function: Balances perception utility (IoU/PQ) against communication latency penalties.

3. Key Contributions

1. BHU Framework: A novel architecture that offloads LVM-based perception to the ground while using Top-K pixel selection on UAVs to drastically reduce uplink data volume.
2. LVM Integration: Demonstrates the use of MaskDINO (Swin-Large) for cooperative BEV feature fusion, proving that LVMs can effectively reconstruct and interpret sparse visual data better than traditional CNNs.
3. DDIM-Based DRL: Proposes a novel algorithm combining DRL with Diffusion Models to solve the joint optimization of UAV selection, sparsification, and precoding. The DDIM component significantly improves training stability and efficiency compared to standard diffusion or exhaustive search methods.
4. Comprehensive Evaluation: Extensive simulations on the Air-Co-Pred dataset validate the framework's superiority in both perception accuracy and communication efficiency.

4. Simulation Results

The framework was evaluated using the Air-Co-Pred dataset (simulated urban traffic with 4 UAVs).

• Perception Performance:

  • The BHU framework with Swin-Large backbone improved perception performance (Mean IoU) by over 5% compared to traditional CNN-based (EfficientNet) baselines.
  • It achieved a 4.94% relative improvement in IoU over EfficientNet when using 4 UAVs.
  • LoRA Fine-tuning: Using Low-Rank Adaptation (LoRA) on the Swin-Large model reduced trainable parameters by ~85% while maintaining performance within 2% of the fully fine-tuned model.

• Communication Efficiency:

  • The Top-K sparsification (25% ratio) reduced communication overhead by 85% compared to the baseline Drone-Help-Drone (DHD) scheme that transmits dense BEV features.
  • Specifically, BHU achieved a communication cost of $9.22 \times 10^6$ bps, whereas the DHD benchmark required $6.14 \times 10^7$ bps for similar tasks.

• DRL Performance:

  • The DDIM-based DRL algorithm converged faster and achieved higher rewards than DRL without diffusion guidance.
  • The algorithm successfully learned to trade off between latency and utility; increasing the latency weight ($\lambda$) led to higher SINR and lower latency, with a clear "turning point" identified for optimal system configuration.

5. Significance

This paper provides a transformative solution for low-altitude economy networks where bandwidth is scarce but perception accuracy is critical.

• Scalability: By moving heavy LVM processing to the ground, the system scales to complex urban environments without overloading UAV hardware.
• Efficiency: The combination of Top-K sparsification and MU-MIMO transmission makes real-time cooperative perception feasible in resource-constrained wireless environments.
• AI-Driven Optimization: The integration of Diffusion Models into DRL offers a new paradigm for solving high-dimensional, stochastic optimization problems in wireless communications, bridging the gap between advanced AI models and practical network constraints.

In summary, the BHU framework successfully demonstrates that Large Vision Models can be effectively leveraged for cooperative perception if the system architecture is designed to separate data acquisition (UAVs) from heavy computation (Ground Server), supported by intelligent, diffusion-driven resource management.