⚡ electrical engineering

RL-Based Delay Minimization for Computation Offloading in IRS- and RHS-Enhanced MEC Systems

This paper proposes a Deep Deterministic Policy Gradient (DDPG)-based reinforcement learning framework to jointly optimize IRS phase shifts, RHS antenna amplitudes, and task offloading decisions in an IRS- and RHS-enhanced mobile edge computing system, thereby significantly minimizing overall task execution delay compared to conventional approaches.

Original authors: Zahra Amiri, Mohsen Eslami, Ebrahim Nikmaleki

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zahra Amiri, Mohsen Eslami, Ebrahim Nikmaleki

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a busy kitchen (the Mobile Edge Computing system) where customers (the users) want their food prepared as quickly as possible. Some customers have their own tiny blenders at their tables (local computing), while others send their orders to the main kitchen chef (the MEC server) to be cooked.

The goal of this paper is to figure out the fastest way to get every order done, minimizing the total time from "order placed" to "food served."

Here is how the authors solve the problem, broken down into simple concepts:

1. The Two "Magic Tools"

The kitchen is in a building with bad acoustics and weird walls, making it hard for the chef to hear the customers clearly. To fix this, the authors introduce two high-tech tools:

The "Smart Mirror" (IRS - Intelligent Reflecting Surface): Imagine a wall covered in thousands of tiny, adjustable mirrors. If a customer's voice is blocked by a pillar, these mirrors can catch the sound and bounce it perfectly toward the chef. In the paper, this is an Intelligent Reflecting Surface that adjusts how radio signals bounce to make the connection stronger.
The "Shape-Shifting Speaker" (RHS - Reconfigurable Holographic Surface): This is a special speaker system at the chef's station. Instead of just blasting sound in one direction, it can instantly change its shape to focus the sound exactly where the customer is standing, cutting through the noise. This is the Reconfigurable Holographic Surface, which acts like a super-precise antenna.

2. The Big Decision: Cook Here or Send There?

Every customer has a choice:

Cook locally: Use their own small blender. This is fast if the task is small, but slow if the task is huge because their blender is weak.
Send to the chef: Send the order to the main kitchen. This is great for big tasks, but it takes time to shout the order across the room (transmission delay).

The tricky part is that the "room" (the wireless signal) changes constantly. Sometimes the mirrors are aligned perfectly; sometimes they aren't. The chef's ability to hear also changes.

3. The Problem: Too Many Choices

The authors tried to write a math equation to find the perfect moment to send an order and the perfect angle for every single mirror. But the math was too messy and complicated (non-convex and high-dimensional). It was like trying to solve a puzzle with a million moving pieces all at once. Traditional math methods got stuck.

4. The Solution: The "Smart Trainee" (DDPG)

Instead of solving the math equation directly, the authors taught a computer program to learn by trial and error. They used a method called Deep Deterministic Policy Gradient (DDPG).

Think of this program as a smart trainee chef:

The State (What it sees): The trainee looks at the room. It sees where the customers are, how loud the noise is, and how much work the main chef has left to do.
The Action (What it does): The trainee makes three decisions at once:
1. Tilt the tiny mirrors (adjust IRS phase shifts).
2. Focus the speaker (adjust RHS amplitudes).
3. Decide how much of the order to cook locally vs. sending to the chef (task offloading ratio).
The Reward (The Score): If the food is served quickly, the trainee gets a high score. If it's slow, the score drops. If the trainee tries to send too many orders and the main chef gets overwhelmed, the score gets a big penalty.

Over time, the trainee learns the best habits. It figures out, "Oh, when the customer is far away, I should tilt the mirrors this way and send 80% of the order to the chef."

5. What They Found

The authors ran simulations (computer tests) to see if their "smart trainee" worked.

The Trainee Wins: The system using this learning method was much faster than systems that didn't use offloading or didn't use the smart mirrors.
More Mirrors = Faster: They found that adding more tiny mirrors to the wall (increasing the number of IRS elements) made the system even faster, shaving off about 10% of the delay when they went from 8 mirrors to 80.
Convergence: The trainee started slow but quickly learned the best strategy and stopped making mistakes, stabilizing at a very low delay time.

Summary

The paper claims that by combining a Smart Mirror (IRS) and a Shape-Shifting Speaker (RHS) with a Learning AI (DDPG), you can significantly reduce the time it takes to process data in a wireless network. The AI learns how to juggle the mirrors, the speakers, and the task distribution to make everything run as fast as possible, outperforming older, static methods.

Technical Summary: RL-Based Delay Minimization for Computation Offloading in IRS- and RHS-Enhanced MEC Systems

Problem Statement
The paper addresses the challenge of minimizing overall task execution delay in Mobile Edge Computing (MEC) systems operating under unfavorable wireless channel conditions. While MEC brings computational resources closer to users, performance is often limited by unreliable uplink transmission. To overcome this, the authors propose a system enhanced by two emerging technologies: Intelligent Reflecting Surfaces (IRS) and Reconfigurable Holographic Surfaces (RHS).

In the proposed framework, a Base Station (BS) equipped with an RHS serves multiple single-antenna users. An IRS is deployed to assist the uplink communication. Users employ a partial offloading strategy, where tasks are split between local processing and offloading to the MEC server. The core challenge is a joint optimization problem involving three coupled, continuous decision variables:

IRS Phase Shifts: To optimize the reflected signal path.
RHS Antenna Amplitudes: To control the active beamforming at the BS.
Task Offloading Ratios: Determining the fraction of each task processed locally versus offloaded.

The problem is characterized as highly non-convex and high-dimensional, rendering conventional optimization methods intractable for dynamic network environments.

Methodology
To solve the formulated optimization problem, the authors develop a Deep Deterministic Policy Gradient (DDPG) based Reinforcement Learning (RL) framework. The system is modeled as a Markov Decision Process (MDP) with the following components:

State Space ( $S$ ): Includes instantaneous Channel State Information (CSI) for the direct user-to-BS, user-to-IRS, and IRS-to-BS links, as well as the available computational resources at the MEC server.
Action Space ( $A$ ): Consists of continuous control variables: the IRS phase-shift matrix, the RHS amplitude control matrix, and the vector of task offloading ratios for all users.
Reward Function ( $r$ ): Defined as the negative sum of total system latency (local computing delay + edge computing delay). To ensure feasibility, a penalty term is added to the reward function for violations of the MEC server's total computational resource constraint.
Algorithm: The DDPG algorithm utilizes an actor-critic architecture. The actor network generates continuous actions, while the critic network evaluates them. Training is stabilized using experience replay buffers and target networks with soft updates.

Key Contributions
The paper outlines four primary contributions:

Novel Framework: Proposal of a joint IRS–RHS assisted MEC framework specifically designed for delay minimization.
Problem Formulation: Mathematical formulation of a joint optimization problem covering IRS phase shifts, RHS amplitudes, and task offloading decisions under physical and resource constraints.
Algorithmic Solution: Development of a DDPG-based solution capable of handling the high-dimensional continuous action space inherent in the problem.
Performance Validation: Demonstration through simulations that the proposed integrated design significantly outperforms conventional approaches in reducing task execution delay.

Simulation Results
Numerical simulations were conducted using Python and PyTorch with a system comprising a BS with 16 RHS elements, an IRS with 20 reflecting elements, and multiple users. Key findings include:

Convergence: The proposed DDPG-based algorithm demonstrates a clear convergence trend, with system delay decreasing over iterations and stabilizing at a lower value compared to a non-offloading baseline.
IRS Impact: Comparing scenarios with and without IRS assistance shows that IRS significantly reduces system delay by improving channel conditions and signal propagation.
Scalability: Increasing the number of IRS elements from 8 to 80 resulted in an approximate 10% reduction in delay, highlighting the effectiveness of scaling the reflecting surface.

Significance and Claims
The authors claim that the proposed approach effectively addresses the limitations of static optimization frameworks in dynamic, time-varying network conditions. By integrating IRS (for uplink enhancement) and RHS (for efficient downlink/beamforming) with DDPG, the system achieves a synergistic reduction in latency. The paper concludes that this integrated design is a promising solution for next-generation MEC systems, offering significant latency reductions compared to conventional schemes.

Future work, as noted by the authors, is intended to extend this framework to more dynamic scenarios, including user mobility, multi-cell environments, and advanced resource allocation strategies.