💻 computer science

Dynamic Task Offloading in Vehicular Edge Networks: A Mobility-Driven A3C-Based Framework

This paper proposes a mobility-driven task offloading framework for Vehicular Edge Computing that utilizes the Asynchronous Advantage Actor-Critic (A3C) deep reinforcement learning algorithm to optimize scheduling decisions, effectively minimizing latency and energy overhead while ensuring reliability in dynamic vehicular environments.

Original authors: Manoj Kumar Somesula

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Manoj Kumar Somesula

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy highway where every car is a smartphone trying to do heavy math problems, like navigating a complex route or rendering a 3D game. These cars (vehicles) have small engines (processors) that can't handle the heavy lifting alone. They need help.

Enter the Roadside Units (RSUs). Think of these as "pit stops" or "super-charged service stations" located along the highway. They have massive engines and can solve these math problems instantly.

The Problem: The "Moving Target" Dilemma

Here's the catch: The cars are moving fast.

The Deadline: Some math problems are urgent. If you don't solve them in 100 milliseconds, the car might crash or the game might freeze.
The Mobility: A car might drive into the range of a pit stop, hand over a task, and then zoom out of range before the pit stop finishes the job. If the car leaves too soon, the task fails, and the car has to start over with the next pit stop.
The Energy: If the car tries to do the math itself, it drains its battery. If it sends the data to the pit stop, it uses radio energy.

The big question is: Who should do the work? Should the car do it itself? Should it ask the current pit stop? Or should it wait for a different pit stop? And how do we decide this when the car is moving at 50 km/h?

The Solution: A "Smart Traffic Cop" (A3C)

The authors of this paper propose a new way to manage this chaos using a type of Artificial Intelligence called Deep Reinforcement Learning, specifically an algorithm named A3C (Asynchronous Advantage Actor-Critic).

Think of A3C as a super-smart, multi-tasking traffic cop who doesn't just look at the cars right in front of him, but predicts where they will be in the next few seconds.

The "Actor": This is the decision-maker. It looks at a car, sees how fast it's going, how long it will stay near a pit stop, and how much battery is left. It then decides: "Okay, you (Car A) will do this task yourself," or "You (Car B) should send that task to Pit Stop X."
The "Critic": This is the judge. It watches the Actor's decisions and says, "Good job, that saved energy!" or "Bad move, that task took too long and failed."
The "Asynchronous" part: Instead of one traffic cop making all the decisions, imagine a whole team of cops working simultaneously on different parts of the highway. They learn faster because they are all practicing at the same time, sharing what they learn with a central brain.

How It Works (The Analogy)

Imagine you are at a crowded airport with a heavy suitcase (the data task).

Old Way (Traditional Methods): You ask the first person you see to carry it. If they walk away before you get to the gate, you have to find someone else and start over. Or, you try to carry it yourself until your arms give out.
The A3C Way: You have a smart assistant who knows the airport layout. The assistant predicts: "That person walking away won't make it to the gate in time. But that person standing still has a strong back and is going to be there for 5 minutes. Hand the suitcase to them."

The assistant learns from every mistake. If it hands a suitcase to someone who drops it, it learns not to do that again. If it finds a perfect helper, it remembers that pattern.

What the Paper Found

The researchers built a computer simulation of this highway scenario with 50 cars and various numbers of tasks. They compared their "Smart Traffic Cop" (A3C) against other methods (like a simple greedy algorithm or other AI types).

Here is what happened in their simulation:

Faster Results: The A3C method finished tasks faster (lower delay) than the other methods, even when there were many cars on the road. It reduced delays by up to 19% compared to the next best method.
Battery Saver: It saved a lot of energy. The cars used up to 35% less energy because the AI knew exactly when to offload a task and when to do it locally, avoiding wasted radio signals.
More Success: More tasks got finished on time. The success rate was up to 13% higher than the other methods. This is because the AI predicted how long a car would stay near a pit stop (sojourn time) and didn't assign tasks to cars that would leave too soon.

The Bottom Line

The paper claims that by using this specific AI (A3C) and by paying close attention to how long a car stays near a service station (mobility), we can make smart cars run smoother, faster, and with less battery drain. It's like upgrading from a manual map to a GPS that predicts traffic jams before they happen, ensuring your "tasks" arrive at their destination on time.

Technical Summary: Dynamic Task Offloading in Vehicular Edge Networks

Problem Statement

The paper addresses the critical challenge of dynamic task offloading within Vehicular Edge Computing (VEC) networks. As the Internet of Vehicles (IoV) expands, applications such as autonomous driving, immersive gaming, and holographic communication demand stringent Quality of Service (QoS), specifically end-to-end latencies between 10–100 ms, high bandwidth, and reliability.

The core difficulty lies in the high mobility of vehicles and the dynamic nature of network resources. When a vehicle moves between Roadside Unit (RSU) coverage areas, ongoing tasks may fail or require retransmission if the sojourn time (dwell time) within an RSU's range is not accurately predicted. Existing scheduling strategies often rely on simplified assumptions, such as constant velocities or single-path mobility, failing to account for the complex, bidirectional, and multi-path movement patterns of real-world traffic. Furthermore, vehicles have limited onboard computational resources, necessitating efficient offloading decisions that balance task deadlines against energy consumption and latency.

Methodology

1. System and Mobility Modeling

The authors model the VEC environment as a network of $V$ mobile vehicles and $R$ RSUs connected to a central cloud.

Mobility Model: Vehicle movement is modeled using a Markov Renewal Process (MRP). This allows for the prediction of vehicle trajectories and, crucially, the sojourn time ( $\Delta^r_v$ ) a vehicle spends within a specific RSU's coverage. The expected sojourn duration is calculated based on transition probabilities and time distributions between RSUs.
Task Model: Tasks are defined by input data size ( $d_{r,t}$ ), computational load ( $c_{r,t}$ ), and deadlines ( $\delta_{r,t}$ ). A task can be executed locally at the RSU or offloaded to a specific vehicle.
Utility Function: A per-task utility function is designed to reward meeting deadlines and penalize energy consumption:
$U_{r,t} = w_d \max\left(0, 1 - \frac{D_{r,t}}{\delta_{r,t} + \epsilon}\right) - w_e \frac{E_{r,t}}{E_{max}}$
where $D_{r,t}$ is the total delay (transmission + execution) and $E_{r,t}$ is the energy cost.

2. Problem Formulation

The task offloading problem is initially formulated as an Integer Linear Programming (ILP) problem aimed at maximizing overall system utility while satisfying constraints on:

Assignment (each task must be processed exactly once).
Deadline compliance.
Sojourn time (offloaded tasks must complete before the vehicle leaves the RSU's range).
Computational capacity of both RSUs and vehicles.

The authors note that this ILP is NP-hard, and solving it directly via conventional optimization is computationally prohibitive for large-scale, dynamic networks due to the exponential growth of decision variables and the need for real-time adaptation.

3. Proposed Framework: A3C-Based DRL

To overcome the limitations of static optimization, the paper proposes a Deep Reinforcement Learning (DRL) framework using the Asynchronous Advantage Actor-Critic (A3C) algorithm.

Markov Decision Process (MDP): The problem is mapped to an MDP tuple $(S, A, P, R, \gamma)$ $(S, A, P, R, γ)$ .
- State Space ( $S$ ): Includes vehicle locations, velocities, available computational/storage resources at RSUs and vehicles, queued tasks, and predicted sojourn times.
- Action Space ( $A$ ): Decisions to offload a task to a specific vehicle or process it locally.
- Reward Function ( $R$ ): Derived from the utility function, balancing delay reduction and energy efficiency.
A3C Architecture: The framework employs multiple parallel worker agents (representing vehicles/RSUs) that interact with the environment asynchronously. Each worker maintains local policy and value networks, accumulating gradients based on computed advantages before updating a shared global network. This asynchronous approach accelerates learning and improves stability in dynamic environments compared to synchronous methods.

Key Contributions

Mobility-Aware Modeling: The integration of a Markov Renewal Process to accurately predict vehicle sojourn times and travel paths, moving beyond simplified constant-velocity assumptions.
ILP Formulation: Development of a mathematical ILP formulation for the task offloading problem that explicitly incorporates task deadlines, vehicle sojourn times, and resource constraints.
A3C-Based Scheduling: Design of a deep reinforcement learning framework using A3C to solve the NP-hard scheduling problem. The approach leverages parallel learning to adapt to dynamic network conditions and optimize the trade-off between latency and energy.
Comprehensive Evaluation: Extensive simulations comparing the proposed A3CRL approach against four baseline algorithms: DDPG, DDQN, SA-PSO (Simulated Annealing Particle Swarm Optimization), and GA (Genetic Algorithm).

Performance Results

Simulations were conducted in a Python environment with varying numbers of vehicles (10–50), tasks (50–250), and vehicle speeds (30–70 km/h). The proposed A3CRL framework demonstrated superior performance across all metrics:

Service Delay: A3CRL achieved up to a 19% reduction in service delay compared to the second-best method (DDQN) when varying vehicle density, and up to a 29% reduction when varying task load.
Energy Consumption: The framework improved energy efficiency by up to 35% (vs. vehicle density changes) and 40% (vs. task load changes) compared to baselines, attributed to its energy-aware objective function and efficient task allocation.
Successful Offloading Rate: A3CRL maintained the highest percentage of successfully completed tasks, showing improvements of up to 13% over the second-best strategy. This is attributed to the algorithm's ability to select vehicles with stable links and sufficient resources based on predicted sojourn times.
Convergence: In terms of training convergence, A3CRL achieved a higher mean reward (11.64 vs. 11.21 for DDQN) with lower variance (0.39 vs. 0.46), indicating more stable and effective learning.

Significance and Claims

The paper claims that the proposed A3C-based framework offers a scalable, adaptive, and efficient solution for real-time task scheduling in VEC networks. By integrating mobility prediction (sojourn time) directly into the scheduling decision process, the framework addresses the limitations of existing methods that often overlook the dynamic nature of vehicle movement or rely on greedy strategies that fail to scale.

The authors emphasize that while traditional optimization techniques (like ILP) are theoretically sound, they are impractical for large-scale, dynamic IoV environments due to computational overhead. The proposed DRL approach effectively bridges this gap, enabling intelligent agents to learn optimal offloading policies through interaction with the environment. The results suggest that deep reinforcement learning, specifically A3C, holds significant potential for managing the complexities of large-scale task offloading in vehicular edge networks.

The paper concludes modestly, noting that future work will investigate multi-objective optimization, cooperative offloading strategies, and experimental validation to further enhance robustness in practical dynamic environments.