Technical Summary: Federated Meta Deep Reinforcement Learning for Heterogeneous Task Offloading in VEC

1. Problem Statement

The paper addresses the challenge of heterogeneous task offloading in Vehicular Edge Computing (VEC) systems. While VEC offers low-latency processing by offloading tasks to nearby edge servers, real-world vehicular applications (e.g., autonomous driving) are modeled as Directed Acyclic Graph (DAG) tasks. These tasks possess complex dependency structures where subtasks have varying computational demands and strict precedence constraints.

The core challenges identified are:

• Complex Optimization: Jointly optimizing offloading decisions, computation scheduling, and resource allocation for DAG tasks is an NP-hard Mixed-Integer Nonlinear Program (MINLP).
• Dynamic Environments: Traditional heuristic or game-theoretic methods lack adaptability to high-dimensional, time-varying network conditions and heterogeneous task structures.
• Privacy and Scalability: Centralized training of learning-based policies requires sharing raw vehicular data (including location and behavior), raising privacy concerns and incurring high communication overhead. Existing Deep Reinforcement Learning (DRL) approaches often assume simplified, independent and identically distributed (i.i.d.) task models, limiting their generalization to complex DAG scenarios.

2. Methodology: The FedMAGS Framework

The authors propose FedMAGS (Federated Meta Deep Reinforcement Learning with GAT-Seq2Seq modeling), a framework designed to handle heterogeneous DAG workloads while preserving data privacy. The system architecture consists of three layers: a vehicular application layer, a vehicular layer, and an MEC layer with a federated aggregation server.

A. Dependency-Aware Modeling (GAT-Seq2Seq)

To address the structural complexity of DAG tasks, the framework employs a specific neural architecture:

• Graph Attention Network (GAT): Used to extract topological features from the DAG. It aggregates high-dimensional feature representations of subtasks by attending to their parent and child nodes, capturing the complex dependency constraints.
• Seq2Seq Policy: The extracted features are fed into a Bidirectional Gated Recurrent Unit (BiGRU) encoder and an attentional GRU decoder. This structure maps the non-Euclidean DAG topology into a serialized offloading plan.
• Action Space: The policy generates a sequence of categorical decisions for each subtask, selecting between local execution or specific MEC resources (uplink channels and processors).

B. Federated Meta-Learning

To enable collaborative learning without sharing raw data, the framework integrates Model-Agnostic Meta-Learning (MAML) with Federated Learning (FL):

• Local Adaptation (Inner Loop): Each MEC server trains a local policy using Proximal Policy Optimization (PPO) on locally observed task distributions. The server performs a few gradient steps to adapt to specific regional dynamics.
• Federated Meta-Update (Outer Loop): Instead of sharing raw task data, MEC servers upload only model parameter updates (deltas) to a central aggregation server. The server performs weighted averaging (FedAvg-style) to update the global meta-parameters.
• Fast Adaptation: The global meta-model serves as a robust initialization, allowing individual MEC servers to quickly adapt to new, unseen DAG scenarios with minimal local training steps.

3. Key Contributions

The paper outlines four primary contributions:

1. Problem Formulation: It formulates heterogeneous DAG-based task offloading in VEC as a joint optimization problem that integrates dependency constraints, resource allocation, and privacy considerations.
2. FedMAGS Framework: It designs a federated meta-DRL framework that facilitates collaborative and adaptive policy learning across distributed MEC servers without raw data exchange.
3. Dependency-Aware Modeling: It introduces a GAT-Seq2Seq architecture specifically to capture DAG topology and critical-path information, enabling structured offloading decisions.
4. Extensive Evaluation: It provides simulation results demonstrating the framework's superiority in convergence speed, scalability, and latency performance compared to state-of-the-art baselines.

4. Experimental Results

The authors evaluated FedMAGS using a custom simulator with synthetic DAG tasks generated by DAGGEN. The system was compared against three baselines: SMRL-MTO (Seq2Seq meta-RL), PPO (local training), and DQN.

• Convergence: FedMAGS achieved the fastest convergence and the lowest steady-state Average Execution Time (AET) during pretraining, outperforming standalone PPO and DQN due to federated meta-initialization.
• Scalability (Varying Subtask Numbers): When the number of subtasks increased (from 10 to 30), FedMAGS maintained stable performance with minimal degradation. In contrast, PPO and SMRL-MTO converged to higher delays, and DQN exhibited instability for large DAGs.
• Robustness (Varying Topologies): Under complex DAG topologies (high density and low parallelism), FedMAGS consistently achieved the lowest AET and stable convergence. Other methods, particularly PPO and DQN, showed strong oscillations or failed to converge effectively.
• Privacy: The framework successfully preserved data privacy by transmitting only model parameters, thereby eliminating the need to share sensitive raw vehicular data (trajectories, workload patterns) while reducing backhaul communication overhead.

5. Significance and Claims

The paper claims that FedMAGS offers a unified solution for adaptive, scalable, and privacy-preserving task offloading in dynamic VEC environments. Its significance lies in bridging the gap between:

• Heterogeneous DAG Modeling: Moving beyond simplified task models to handle real-world dependency structures.
• Rapid Adaptation: Utilizing meta-learning to generalize across diverse vehicular workloads without retraining from scratch.
• Privacy Preservation: Enabling collaborative intelligence in distributed MEC systems without compromising user data privacy.

The authors conclude that the proposed framework is well-suited for large-scale VEC deployments where task heterogeneity, dynamic network conditions, and data privacy are critical constraints.