💻 computer science

A Linear Mapping-Enhanced PPO Framework for Dynamic Load Balancing in Smart City Edge Systems

This paper proposes a Linear Mapping-Enhanced Proximal Policy Optimization (LME-PPO) framework that leverages deep reinforcement learning to optimize dynamic load balancing and reduce system latency in smart city edge computing environments by effectively mapping complex task spaces and ensuring computational equilibrium across edge servers.

Original authors: Fenghui Zhang, Yuhang Jiang, Yuhao Xu, Huaqiang Xi, Qiu Xu, Shijian Zheng, Maosheng Fu

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fenghui Zhang, Yuhang Jiang, Yuhao Xu, Huaqiang Xi, Qiu Xu, Shijian Zheng, Maosheng Fu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Smart City's Traffic Jam

Imagine a Smart City as a massive, bustling metropolis. In this city, thousands of devices (like traffic cameras, smart meters, and sensors) are constantly generating data. This data is like a flood of cars trying to get to a destination.

In the past, all these "cars" had to drive all the way to a giant, central "Cloud City" (the main data center) to be processed. This caused massive traffic jams and long delays.

To fix this, the city built Edge Servers. Think of these as local neighborhood service stations scattered all over the city. They are much closer to the devices, so they can fix problems faster.

The Problem: The Uneven Workload

Here is the catch: The "cars" (data tasks) don't arrive evenly.

Sometimes, a sudden traffic accident sends a huge wave of data to Edge Server A.
At the same time, Edge Server B in a quiet neighborhood has almost nothing to do.

If the city doesn't manage this, Server A gets overwhelmed and slows down (causing delays), while Server B sits idle, wasting money. This is called Load Imbalance.

The city needs a Central Dispatcher (a smart scheduler) to look at all the servers and say, "Server A is too busy; let's send some of those cars to Server B."

The Old Ways vs. The New Way

The paper looks at how to build this Dispatcher:

Static Rules (The Old Way): Like a traffic light that turns red and green on a fixed timer. It's simple, but it can't handle a sudden accident or a parade. It often leaves some servers overloaded.
Optimization Algorithms (The Middle Way): Like a GPS that calculates the perfect route every time. It's smart, but it takes a long time to calculate the route. By the time it figures it out, the traffic situation has already changed.
Deep Reinforcement Learning (The New Way): This is like training a super-intelligent traffic cop who learns from experience. Instead of following a rulebook, the cop watches the traffic, tries different strategies, and learns which moves get the cars moving fastest over time.

The Paper's Solution: The "LME-PPO" Framework

The authors propose a specific type of "super-intelligent cop" called LME-PPO. Here is how it works, broken down into three simple parts:

1. The "Linear Mapping" (The Translator)

The biggest problem with training a smart AI for this job is that the number of cars (tasks) changes every second.

The Issue: Imagine you are teaching a robot to distribute cookies. If you tell the robot, "Give out 5 cookies," but suddenly there are 500 cookies, the robot gets confused. If you tell it, "Give out 500," but there are only 5, it breaks. The robot's brain (the AI model) usually expects a fixed number, but the real world is messy and changing.
The Fix: The authors added a Linear Mapping Layer. Think of this as a translator or a scale.
- The AI brain says: "I think Server A should get 30% of the work, and Server B should get 70%." (These are just proportions).
- The Translator looks at how many actual tasks arrived right now. If 100 tasks arrived, the translator says, "Okay, 30% of 100 is 30 tasks for Server A."
- This allows the AI to learn the strategy (the percentages) without getting confused by the volume (the total number of tasks).

2. The "PPO" (The Smart Trainer)

PPO (Proximal Policy Optimization) is the training method.

Imagine you are teaching a dog to fetch. If you yell too hard when it makes a mistake, the dog gets scared and stops learning. If you are too soft, it doesn't learn the trick.
PPO is a "Goldilocks" trainer. It makes sure the AI learns step-by-step without making huge, scary mistakes that ruin its progress. It keeps the learning stable and steady.

3. The "Internal Balancer" (The Local Manager)

Once the Central Dispatcher sends tasks to a specific Edge Server, that server still has to do the work.

Inside each Edge Server, there are smaller workers called Virtual Machines (VMs).
The paper includes a special rule to make sure the tasks are split evenly among these internal workers. It's like a foreman making sure that if one worker is fast (high CPU speed), they get the heavy lifting, while slower workers get lighter tasks, so everyone finishes at the same time.

The Results: What Happened?

The authors tested their new system against the old methods (like "Greedy" algorithms, which just grab the first available server, and "PSO," a type of optimization).

Faster Speed: Their system reduced the average time it took to process tasks.
Better Balance: The work was spread much more evenly. No single server was left drowning in work while others were idle.
Stability: Even when the city got chaotic (high traffic, sudden spikes in data), their system kept working smoothly without crashing or getting confused.

Summary

In short, this paper presents a new way to manage data in Smart Cities. It uses a smart AI that learns to balance workloads dynamically. The secret sauce is a translator layer that helps the AI handle changing amounts of data, and a stable training method that ensures the AI learns correctly without going crazy. The result is a city where data flows faster, and services (like traffic control or emergency response) work better.

Technical Summary: A Linear Mapping-Enhanced PPO Framework for Dynamic Load Balancing in Smart City Edge Systems

Problem Statement
The integration of Mobile Edge Computing (MEC) into smart city infrastructures aims to reduce latency and improve bandwidth efficiency by processing data closer to IoT sources. However, the stochastic and heterogeneous nature of task offloading from urban IoT devices (e.g., traffic cameras, smart grids) creates significant load imbalances among Edge Servers (ESs). In a typical deployment where each ES hosts multiple Virtual Machines (VMs), random task arrivals can cause some servers to become overwhelmed while others remain underutilized. This imbalance degrades service quality and increases system latency. Existing solutions face limitations: static scheduling algorithms lack adaptability to dynamic workloads; traditional optimization methods (e.g., Particle Swarm Optimization) suffer from iterative delays unsuitable for real-time constraints; and standard Deep Reinforcement Learning (DRL) approaches struggle with the mismatch between continuous state variations and fixed action spaces, particularly when task counts fluctuate significantly.

Methodology
The authors propose a Linear Mapping-Enhanced Proximal Policy Optimization (LME-PPO) framework to address these challenges. The approach is structured as follows:

System Modeling: The smart city resource management system is modeled as a Markov Decision Process (MDP).
- State Space ( $S$ ): Defined by the number of pending tasks in the queues of $M$ edge servers.
- Action Space ( $A$ ): Represents the task scheduling matrix across all ESs.
- Reward Function: Designed to minimize the variance between individual ES execution times and the system-wide average execution time, thereby promoting load balancing.
- Constraints: The model accounts for queue capacity limits, CPU frequency variations across VMs, and maximum allowable execution times.
LME-PPO Framework:
- Actor-Critic Architecture: The system utilizes a standard PPO algorithm with Actor and Critic networks (two-layer fully connected networks).
- Linear Mapping Layer: A core innovation of this work. Since PPO typically outputs continuous values, but task allocation requires discrete integers, the authors introduce a linear mapping layer. The Actor network outputs a vector of relative allocation proportions ( $m_t$ ). This vector is mapped to actual integer task counts ( $N_t$ ) based on the total number of tasks arriving in the current time slot ( $n$ ) using the formula: $N_i = n \times \frac{m_i}{\sum m_i}$ . This mechanism ensures the action space adapts dynamically to the current workload, preventing training instability caused by fixed action spaces.
- Internal Task Scheduling: Within each ES, a Workload-Balanced Task Allocation Algorithm is employed to distribute tasks among multiple VMs. Based on Theorem 1 and Theorem 2, the algorithm prioritizes assigning tasks to VMs with lower CPU frequencies first to equalize execution times, ensuring the minimum possible completion time for the queue.
Long-Term Utility: The framework optimizes for long-term discounted utility rather than single-slot performance, utilizing Generalized Advantage Estimation (GAE) to balance immediate rewards with future system states.

Key Contributions

Stochastic System Modeling: The paper models the task allocation process as a stochastic system with a DRL framework that incorporates scheduling at the VM level within each ES, capturing key computational characteristics for precise load balancing.
Linear Mapping Enhancement: The authors introduce a linear mapping layer within the PPO framework to resolve the conflict between the continuous output of the policy network and the discrete, time-varying nature of task arrival counts. This ensures stable learning and feasible integer task assignments.
Workload-Balanced Allocation Algorithm: A specific algorithm is devised for internal ES scheduling that minimizes completion time by intelligently assigning tasks to VMs based on their CPU frequencies and the current queue state.
Comprehensive Evaluation: The study provides a rigorous comparison against Particle Swarm Optimization (PSO) and greedy baselines, demonstrating the framework's effectiveness in dynamic environments.

Experimental Results
The proposed LME-PPO framework was evaluated using a Python/PyTorch simulator with 20 edge servers. Key findings include:

Convergence: The algorithm converges faster with fewer ESs (200 episodes for 20 ESs) and demonstrates superior stability compared to SAC and TD3 algorithms.
Latency Reduction: The LME-PPO approach achieved a significantly lower average system execution time (approx. 3.7 ms) compared to PSO (4.5 ms) and Greedy (5.7 ms) algorithms over 100 episodes.
Load Balancing: Under varying workload conditions (low, normal, high), LME-PPO consistently maintained the lowest average computation time and the most balanced runtime distribution across ESs. In contrast, Greedy and PSO algorithms exhibited higher runtimes and greater fluctuations due to task accumulation.
Scalability: The framework maintained stable performance across different network configurations (300, 400, 500 neurons) and discount factors, with a discount factor of 0.99 providing the best long-term performance.

Significance and Claims
The paper claims that the LME-PPO framework provides a robust and scalable solution for next-generation smart city environments. By bridging the gap between continuous control policies and discrete execution requirements through linear mapping, the approach effectively handles the high variability of IoT task offloading. The authors assert that their method not only achieves superior load balancing but also significantly reduces average system latency, offering a practical alternative to heuristic and traditional optimization methods in dynamic edge computing scenarios. The work highlights the necessity of addressing both inter-server load balancing and intra-server (VM-level) task scheduling to maximize resource utilization. Future work is suggested to explore decentralized architectures and more heterogeneous workload patterns.