💻 computer science

Task Offloading in Multi-UAV Edge Computing Using an Improved Multi- Agent Proximal Policy Optimization Approach

This paper proposes SS-MAPPO, an improved Multi-Agent Proximal Policy Optimization framework incorporating top-K sparse attention and Gaussian latent policies, to solve the energy-efficient, decentralized task offloading problem in dynamic multi-UAV edge computing environments modeled as a Dec-POMDP.

Original authors: Changhao Duan, Zheng Yao, ChangJun Deng, Rui Li

Published 2026-11-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Changhao Duan, Zheng Yao, ChangJun Deng, Rui Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the skies above cities and remote landscapes, small flying robots known as drones are becoming essential workers. They inspect power lines, deliver packages, and monitor disaster zones, gathering vast amounts of data as they move. However, these machines carry a heavy burden: their batteries are limited, and their onboard computers are often too weak to process complex information quickly. If a drone tries to solve a difficult math problem or analyze a high-resolution image while flying, it drains its battery rapidly and may run out of power before finishing its job. To solve this, engineers have developed a concept called mobile edge computing. Instead of forcing the drone to do all the work itself, it can send its data to nearby powerful computers on the ground or to even larger servers in the cloud. This allows the drone to stay light and efficient, letting the heavy lifting be done by machines with unlimited power.

The challenge arises when dozens or hundreds of these drones operate in the same area at the same time. Imagine a swarm of one hundred and twenty drones, all trying to send data to a limited number of ground stations. The airwaves become crowded, the ground computers get overwhelmed with too many requests, and the drones must constantly decide whether to work alone, send data to a nearby station, or wait for a distant cloud server. If they make the wrong choice, they waste energy or drop important tasks. This is a problem of coordination. Traditional methods for organizing such groups often fail because the environment changes too fast; a drone might move out of range, or a ground station might suddenly become full. Researchers needed a way for these drones to learn how to make smart, energy-saving decisions on their own, even when they cannot see what the others are doing.

A team of researchers from Hubei University of Automotive Technology and Yunnan Hanzhe Technology Co., Ltd. has developed a new method to help these drone swarms work together more efficiently. They created a system where the drones learn through trial and error, using a type of artificial intelligence that allows them to improve their decisions over time. The researchers built a simulation of a complex world with one hundred and twenty drones, twenty-four ground servers, and one powerful cloud server. In this digital environment, the drones generate tasks of varying sizes, ranging from small data packets of three hundred kilobytes to large files of three thousand three hundred kilobytes. The goal was simple but difficult: get every task done while using as little battery power as possible.

The team tested their new approach against several existing methods, including older algorithms that rely on fixed rules and other learning systems that have been used in the past. They found that their new system, which they named SS-MAPPO, outperformed the others in almost every way. The key to its success was how it handled the sheer number of drones. In a crowded swarm, every drone does not need to pay attention to every other drone; in fact, trying to do so creates confusion and slows down the system. The researchers taught their new system to focus only on the most relevant neighbors, ignoring the rest. This is similar to how a person in a loud room might focus on the voice of a friend standing nearby while tuning out the background chatter. By filtering out unnecessary information, the drones could make faster and more stable decisions.

Furthermore, the researchers improved how the drones learned to choose between local processing, nearby servers, or the cloud. Instead of making abrupt jumps from one choice to another, the new system smoothed out these decisions, allowing the drones to adapt gently to changing conditions like wind or shifting network signals. In their simulations, this approach allowed the swarm to reach a stable, efficient state much faster than other methods. While older systems took longer to learn and often fluctuated wildly in their energy use, the new method settled into a rhythm quickly. When the number of drones increased from a small group to a large swarm of one hundred and twenty, the new system maintained its efficiency, whereas other methods struggled and consumed significantly more energy.

The results of these simulations showed that the new approach could reduce the total energy consumption of the entire swarm. It also proved to be more robust when the size of the tasks changed. Whether the drones were handling tiny data files or massive video streams, the system adjusted without wasting power. The researchers noted that the method was particularly good at preventing the ground servers from becoming overloaded, ensuring that no single drone was left waiting too long for a response. This balance is crucial for real-world applications where a delay could mean a missed delivery or a failed inspection.

Despite these promising results, the researchers are careful to note that their findings come from computer simulations. They modeled the drones moving along fixed paths and assumed that tasks were sent in full chunks rather than being split up. While this provides a strong foundation, the real world is often messier, with unpredictable weather and more complex flight patterns. The team suggests that future work will need to test these ideas with drones that can change direction freely and handle partial data transfers. For now, however, the study offers a clear path forward for managing large groups of autonomous machines. By teaching drones to ignore the noise and focus on what truly matters, we may soon see swarms that work together seamlessly, conserving their energy to keep the skies safe and efficient.

Technical Summary: Task Offloading in Multi-UAV Edge Computing Using an Improved Multi-Agent Proximal Policy Optimization Approach

Problem Statement
The paper addresses the challenge of energy-efficient task offloading in large-scale, heterogeneous Unmanned Aerial Vehicle (UAV) swarms operating within Mobile Edge Computing (MEC) environments. As UAV applications in inspection, logistics, and emergency communication grow, onboard computational and energy resources become bottlenecks. While offloading tasks to edge or cloud servers offers a solution, the multi-UAV environment presents significant difficulties:

Partial Observability: UAVs only have access to local information (e.g., local queue states, channel conditions) rather than global system states.
Dynamic Conditions: Wireless link quality, server accessibility, and task arrival rates fluctuate rapidly due to UAV mobility.
Resource Contention: Concurrent access by multiple agents leads to channel interference and queue congestion at shared edge servers.
Scalability Issues: Traditional optimization methods suffer from high computational complexity in real-time, while existing Multi-Agent Reinforcement Learning (MARL) approaches often face unstable policy learning and inefficient interaction modeling as swarm sizes increase.

The objective is to minimize the total energy consumption of the UAV swarm while satisfying task execution requirements under these dynamic and partially observable constraints.

Methodology
The authors formulate the problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). To solve this, they propose SS-MAPPO, an improved Multi-Agent Proximal Policy Optimization (MAPPO) framework. The system architecture consists of three layers:

UAV Layer: Heterogeneous UAVs (light, medium, heavy) generating tasks.
Edge Server Layer: A shared pool of edge servers (ESs) with task priority queues to manage load balancing.
Cloud Layer: A powerful cloud server for heavy offloading.

The SS-MAPPO framework introduces two specific architectural improvements to the standard MAPPO (which follows a Centralized Training and Decentralized Execution paradigm):

Gaussian-Based Latent Policy Representation (Actor): Instead of mapping observations directly to discrete action logits, the actor first encodes local observations into a latent Gaussian distribution. A latent variable is sampled via reparameterization and then projected to a discrete action distribution. This intermediate stochastic layer is designed to smooth policy updates and reduce abrupt action changes caused by rapid fluctuations in queue states and channel conditions.
Top-K Sparse Attention Mechanism (Critic): In the centralized critic, a top-K sparse attention mechanism replaces dense attention. Each agent focuses only on the $K$ most relevant neighboring agents based on attention scores. This reduces the computational overhead of modeling pairwise interactions from $O(N^2)$ to $O(N \cdot K)$ , enhancing scalability for large swarms while preserving critical inter-agent dependencies related to resource contention.

The reward function is designed as a negative normalized energy cost, augmented with penalty terms for constraint violations (e.g., energy limits) and server overflows.

Key Contributions

System Architecture: Construction of a heterogeneous three-layer (UAV-Edge-Cloud) offloading architecture that incorporates a shared edge server pool and task priority queues to model resource contention and queue dynamics realistically.
Algorithmic Innovation: Development of the SS-MAPPO framework, which integrates a top-K sparse attention mechanism to improve scalability and a Gaussian-based latent policy representation to stabilize discrete decision-making in dynamic environments.
Empirical Validation: Comprehensive simulation experiments comparing SS-MAPPO against baselines including standard MAPPO, MADDPG, MATD3, and classical heuristics (Genetic Algorithm, Simulated Annealing, Round-Robin).

Results
Simulation results, conducted with up to 120 UAVs and varying task loads, demonstrate that SS-MAPPO outperforms baseline methods in several key metrics:

Energy Efficiency: SS-MAPPO achieves lower total energy consumption compared to MAPPO, MADDPG, MATD3, and heuristic algorithms across different swarm sizes and task data volumes.
Convergence Speed: The algorithm converges faster (stabilizing around 50 episodes) compared to the original MAPPO (approx. 60 episodes) and significantly faster than MADDPG/MATD3, which showed slower convergence and higher volatility.
Scalability: As the number of UAVs increases, the energy consumption of SS-MAPPO rises more gradually than that of the baselines, indicating better scalability.
Stability: SS-MAPPO exhibits a lower standard deviation in energy consumption over time, suggesting more consistent and stable decision-making. The Gaussian latent representation and sparse attention are credited for reducing decision fluctuations.
Robustness: The method maintains performance across different task sizes (300 KB to 3300 KB), showing less sensitivity to task scale variations compared to heuristic approaches.

Significance and Claims
The paper claims that SS-MAPPO offers a promising approach for energy-aware collaborative offloading in dynamic multi-UAV edge computing environments. The significance lies in addressing the specific limitations of existing MARL methods in large-scale systems:

It mitigates the "curse of dimensionality" and computational overhead associated with dense agent interactions through sparse attention.
It improves the stability of training and decision-making for discrete offloading actions under partial observability through latent policy representation.
It provides a scalable solution that balances the trade-off between computational efficiency and decision quality in resource-constrained, time-varying networks.

The authors acknowledge limitations, noting that the current model assumes fixed flight routes and full offloading modes. They suggest future work will explore flexible mobility patterns, partial offloading strategies, and complex task dependencies.

More like this