🔢 mathematics

Active Beyond-Diagonal RIS Empowered Heterogeneous Edge Computing: A Distributional Reinforcement Learning Approach

This paper proposes DSAC-T, a distributional reinforcement learning framework that optimizes energy-aware offloading and resource allocation in heterogeneous mobile edge computing systems assisted by reciprocal active beyond-diagonal RISs, effectively addressing the challenges of cross-sector energy leakage and high-dimensional nonconvex optimization to achieve superior energy-latency performance and feasibility.

Original authors: Tianyu Pang, Hongyu Li

Published 2026-07-16

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Tianyu Pang, Hongyu Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a bustling city where your smartphone is a delivery driver trying to drop off heavy packages (data) to a central warehouse (the cloud). Sometimes, the roads are clear, and the driver zooms straight to the warehouse. But often, a giant skyscraper (a building or a wall) blocks the path, causing the driver to get stuck or take a long, slow detour. This is the world of mobile edge computing, where we try to process data quickly on nearby servers instead of waiting for a distant cloud. To fix the blocked roads, scientists invented "smart mirrors" called Reconfigurable Intelligent Surfaces (RIS). Think of these as magical walls that can catch a signal and bounce it around a corner to reach the receiver.

However, there's a catch. Traditional smart mirrors are passive; they just bounce signals without making them stronger, like a regular mirror reflecting a dim light. If the signal is already weak because of a long distance or a thick wall, the reflection is too faint to be useful. To solve this, engineers developed "active" mirrors that can amplify the signal, like a megaphone attached to the mirror. But here's the tricky part: these active mirrors are built with real electronics that have a physical rule called "reciprocity." It's like a two-way street where if you shout from one side, the sound leaks back into the street, wasting energy and creating noise. This paper tackles the messy reality of using these active, amplifying mirrors in a city full of different types of traffic (some tasks need fast CPUs, others need powerful GPUs) and figuring out how to keep the energy bill low while ensuring no package arrives late.

The researchers, Tianyu Pang and Hongyu Li, set out to solve a massive puzzle: how to decide which tasks to send to the edge, how much power to use, and exactly how to tilt the active mirror, all while dealing with the energy leaks caused by the mirror's own hardware. They found that the old way of solving this—trying to calculate the perfect answer for every single situation one by one—is too slow and gets stuck easily. Instead, they built a new kind of artificial intelligence agent called DSAC-T.

Think of DSAC-T not as a calculator that finds one "best" answer, but as a curious teenager learning to play a complex video game. Instead of just guessing the average score, this AI learns the entire range of possible outcomes. It understands that sometimes a move might lead to a huge win, but other times it might lead to a total crash, especially near the "edge" of what's possible (like running out of battery or missing a deadline). By learning these distributions, the AI becomes much more stable and cautious when things get risky.

In their simulations, the team tested this AI against other methods in a virtual city with 10 users, 4 of whom were "deeply blocked" behind obstacles. They gave the AI a mix of heavy CPU tasks (like crunching numbers) and GPU tasks (like rendering graphics). The results were impressive. The DSAC-T AI managed to find a solution that worked for 81.67% of the scenarios, which was the highest success rate among all the methods they tested. In contrast, other popular AI methods only succeeded about 64% to 71% of the time. Furthermore, the DSAC-T AI was incredibly fast, making a decision in just 0.0267 seconds per scenario. Compare this to the traditional mathematical approach, which took over 41 seconds to solve the same problem—making the AI roughly 1,500 times faster.

The paper explicitly argues against the idea that simply adding more "active" power or using older, simpler AI models is enough. They show that ignoring the "cross-sector energy leakage" (the signal leaking back into the system due to the mirror's hardware) leads to poor energy efficiency. Their simulations suggest that you must account for this leakage and the specific mix of CPU and GPU tasks to get a good result. While these findings are based on computer simulations rather than a real-world physical test, the results strongly suggest that using this "distributional" learning approach is a promising way to manage the complex, energy-hungry networks of the future, ensuring that our digital packages get delivered fast without burning out the system.

Technical Summary: Active Beyond-Diagonal RIS Empowered Heterogeneous Edge Computing

Problem Statement
This paper addresses the challenge of energy-aware task offloading and resource allocation in heterogeneous Mobile Edge Computing (MEC) systems assisted by Active Beyond-Diagonal Reconfigurable Intelligent Surfaces (BD-RIS). The system involves a mix of CPU-oriented and GPU-oriented tasks, users with varying degrees of blockage (some deeply blocked requiring RIS assistance, others with residual direct links), and an active BD-RIS operating in a hybrid transmitting and reflecting mode.

A critical physical constraint identified is that practical active BD-RIS devices are realized via reciprocal circuits. This reciprocity inherently generates cross-sector energy leakage when the RIS assists users across different sectors (e.g., reflecting signals to one side while transmitting to the other). This leakage reshapes the system-level energy-latency tradeoff, compressing the energy budget available for uplink transmission and edge execution. Furthermore, the optimization problem is a high-dimensional, mixed-integer, nonconvex problem involving tightly coupled variables: offloading decisions, CPU/GPU computation allocation, uplink transmit powers, receive processing, and active BD-RIS configuration. Conventional per-instance optimization methods struggle with the computational complexity and the strong scenario dependence of this problem.

Methodology
To solve this complex optimization problem efficiently, the authors propose an end-to-end joint optimization framework based on Distributional Soft Actor-Critic (DSAC-T), a refined version of the distributional reinforcement learning algorithm.

System Modeling:
- Channel Model: The system models asymmetric blockage where blocked users rely on the active BD-RIS, while non-blocked users have severely attenuated (but non-zero) direct links. The active BD-RIS is modeled with a reciprocal scattering matrix, explicitly capturing cross-sector coupling ( $\Theta_{1,2} = \Theta_{2,1}$ ) and the resulting energy leakage.
- Heterogeneous Computation: The edge server maintains decoupled CPU and GPU resource pools to handle distinct task types.
- Energy Model: The model accounts for the static circuit power, amplifier efficiency, and the dynamic power consumption of the active RIS, including the cost of amplifying noise and cross-sector leakage.
MDP Reformulation:
The problem is reformulated as a slot-wise Markov Decision Process (MDP).
- State: Includes task features, channel information, and resource conditions.
- Action: Generated by an actor network, covering offloading decisions, edge computation allocation, transmit powers, and RIS control vectors. These are projected into feasible binary and continuous constraints (e.g., ensuring the RIS matrix is diagonal and reciprocal).
- Reward: A penalty-based surrogate of the original objective. It maximizes the negative weighted sum of local, offloading, and RIS energy consumption while explicitly penalizing latency violations.
Algorithm Selection (DSAC-T):
The authors argue that standard expectation-based actor-critic methods (like SAC or DDPG) are insufficient due to reward heterogeneity (caused by varying blockage patterns and task types) and feasibility-boundary sensitivity (where small decision changes drastically alter feasibility).
- Distributional Approach: DSAC-T models the return distribution using twin Gaussian distributions rather than just expected values. This allows the critic to learn both the mean return and the uncertainty (variance).
- Robustness: By using the smaller of the two target means for updates, the algorithm reduces overestimation bias, leading to more stable policy updates in environments with sharp feasibility boundaries.

Key Contributions

Physics-Consistent System Model: The paper establishes a rigorous model for reciprocal active BD-RIS-assisted heterogeneous MEC. It explicitly captures the coupling between transmitting and reflecting sectors due to circuit reciprocity, residual direct-link attenuation for blocked users, and decoupled CPU/GPU edge resources.
Joint Optimization Framework: The authors formulate and solve a strongly coupled mixed-integer nonconvex problem. They develop a DSAC-T-based framework that handles the high dimensionality and nonconvexity by learning a policy that jointly optimizes offloading, computation, power, and RIS configuration.
Algorithmic Adaptation: The study demonstrates the suitability of distributional RL (specifically DSAC-T) for problems with heterogeneous rewards and strict feasibility constraints, showing that modeling return distributions improves stability compared to expectation-based baselines.

Performance Results
Simulations were conducted over 300 evaluation scenarios with 10 users (4 deeply blocked), an 8-antenna base station, and a 128-cell active BD-RIS.

Reward and Feasibility: DSAC-T achieved the best energy-latency reward (-2.828) and the highest feasibility ratio of 81.67%. This outperformed baselines including SAC (71.33%), TD3 (70.67%), DDPG (64.33%), and a mathematical Alternating Optimization (AO) method based on Successive Convex Approximation (66.67%).
Efficiency: The online decision time for DSAC-T was 0.0267 seconds per scenario. In contrast, the AO-SCA method required 41.865 seconds, highlighting that the learning-based approach is orders of magnitude faster for real-time decision-making.
Convergence: DSAC-T converged rapidly and maintained the highest reward, whereas other DRL methods either stabilized at lower levels or converged more slowly.
RIS Impact: Increasing the number of active BD-RIS cells improved both reward and feasibility, confirming that learned RIS control provides gains beyond simply increasing aperture size.

Significance and Claims
The paper claims that its primary significance lies in addressing the specific physical limitations of reciprocal active BD-RIS in heterogeneous MEC, a scenario often over-idealized in prior works. By rigorously modeling cross-sector energy leakage and residual direct links, the proposed framework captures real-world constraints that degrade energy efficiency if ignored.

The authors assert that the proposed DSAC-T framework offers a superior performance-complexity tradeoff. It not only achieves higher feasibility and better energy-latency performance than conventional optimization and other DRL baselines but also enables real-time operation, which is critical for latency-sensitive MEC applications. The work validates that distributional value learning is an effective tool for optimizing systems where reward heterogeneity and feasibility boundaries make traditional expectation-based learning unstable.

Technical Summary: Active Beyond-Diagonal RIS Empowered Heterogeneous Edge Computing

More like this