⚡ electrical engineering

Robust SAC-Enabled UAV-RIS Assisted Secure MISO Systems With Untrusted EH Receivers

This paper proposes a robust secure communication framework for UAV-RIS-assisted MISO systems with untrusted energy-harvesting receivers and imperfect channel state information, utilizing a tailored soft actor-critic (SAC) deep reinforcement learning approach to maximize worst-case secrecy energy efficiency while outperforming conventional optimization and other deep reinforcement learning benchmarks.

Original authors: Hamid Reza Hashempour, Le-Nam Tran, Duy H. N. Nguyen, Hien Quoc Ngo

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hamid Reza Hashempour, Le-Nam Tran, Duy H. N. Nguyen, Hien Quoc Ngo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a high-stakes game of "Hide and Seek" played in a complex, invisible maze. In this story, the goal is to send secret messages to a group of friends (the Legitimate Users) while keeping them safe from a group of spies (the Untrusted Energy-Harvesting Receivers). These spies have a dual nature: they want to steal your energy (like a solar panel) but also want to eavesdrop on your secrets.

Here is how the paper solves this problem, broken down into simple concepts:

The Setting: The Invisible Maze

The "maze" is the air between a ground-based tower (the Base Station) and the users. The problem is that there are tall buildings blocking the direct path, so the signal can't get through.

To fix this, the system uses a Drone (UAV) carrying a special "magic mirror" called a Reconfigurable Intelligent Surface (RIS).

The Drone: It hovers in the sky. Its job is to find the perfect spot to catch the signal from the tower and bounce it down to the users.
The Magic Mirror (RIS): This surface has hundreds of tiny tiles. Each tile can twist the signal like a prism, steering it exactly where it needs to go.

The Challenge: The "Perfect Storm" of Problems

The authors faced three massive headaches trying to make this work:

The Spies are Sneaky: The system doesn't know exactly where the spies are or how strong their listening equipment is. It only has a "best guess" (Imperfect Information). The system must plan for the worst-case scenario where the spies are as good as they could possibly be.
The Mirror is Digital: In the real world, the mirror tiles can't twist the signal to any angle; they can only snap to specific, pre-set angles (like a digital clock that only shows whole minutes, not seconds).
The Balancing Act: The system has to do three things at once:
- Move the drone to the perfect spot.
- Adjust the mirror tiles to focus the signal.
- Decide how much power to use for each user.
- And do all this while making sure the spies get enough energy to keep their batteries charged (so they don't complain) but not enough to steal the secrets.

The Solution: Two Different Approaches

The paper proposes two ways to solve this puzzle.

Approach 1: The "Step-by-Step" Calculator (SCA-BCD)

Think of this as a very smart, methodical human engineer.

How it works: It tries to solve the puzzle by fixing one piece at a time. It says, "Okay, let's keep the drone still and just adjust the mirror. Now, let's keep the mirror still and just move the drone." It repeats this over and over, getting slightly better each time.
The Result: It works well if the conditions are perfect (like a calm day with no wind). However, it's slow, and if the spies are very tricky (uncertain information), it can get stuck in a local "good enough" spot rather than finding the best spot.

Approach 2: The "Super-Intelligent Video Game Bot" (SAC)

This is the paper's main innovation. They used a type of Artificial Intelligence called Soft Actor-Critic (SAC).

How it works: Imagine a video game character that has played the "Hide and Seek" game millions of times. It doesn't calculate every step mathematically. Instead, it has "learned" through trial and error what moves work best.
The Secret Sauce: Unlike other AI bots that just try to win, this one is trained to be curious. It tries random moves occasionally (exploration) to see if there's a better strategy, which helps it avoid getting stuck in bad spots.
The Result: Once this AI is "trained" (which happens offline, like studying for a test), it can make decisions instantly. It looks at the situation and instantly knows exactly where to put the drone and how to twist the mirror, even if the spies are being unpredictable.

The Big Win

The authors tested their "Super-Intelligent Bot" against the "Step-by-Step Calculator" and other AI methods. Here is what they found:

Better Performance: The SAC bot achieved up to 28% better results than the traditional calculator method and 16% better than other AI bots.
Robustness: Even when the information about the spies was very fuzzy (high uncertainty), the SAC bot kept performing well, while the others struggled.
Speed: Once trained, the SAC bot makes decisions instantly, whereas the calculator method has to do heavy math every single time a new situation arises.

The Bottom Line

The paper shows that using a smart, learning-based AI (SAC) to control a drone and a digital mirror is a much better way to send secure, energy-efficient messages than using old-school math methods. It's like replacing a slow, manual calculator with a seasoned expert who has seen every possible trick the enemy can throw at them.

Technical Summary: Robust SAC-Enabled UAV-RIS Assisted Secure MISO Systems With Untrusted EH Receivers

Problem Statement
This paper addresses the challenge of secure downlink transmission in Unmanned Aerial Vehicle (UAV)-assisted Reconfigurable Intelligent Surface (RIS) enabled multiuser Multiple-Input Single-Output (MISO) systems. The specific scenario involves a hovering UAV-mounted RIS assisting a multi-antenna Base Station (BS) in serving legitimate Information-Harvesting Receivers (IHRs) while coexisting with Untrusted Energy-Harvesting Receivers (UEHRs). These UEHRs act as potential eavesdroppers and harvest energy simultaneously (SWIPT).

The system operates under significant practical constraints:

Imperfect Channel State Information (CSI): The BS possesses only imperfect estimates of the UEHR channels, modeled via a norm-bounded uncertainty set, necessitating a worst-case security formulation.
Discrete RIS Phase Shifts: Practical RIS hardware is limited to finite-resolution phase shifts rather than ideal continuous values.
Coupled Optimization: The problem requires the joint optimization of three highly coupled variables: the UAV's horizontal location, the transmit power allocation at the BS (using Zero-Forcing precoding), and the discrete RIS phase shifts.
Objective: The goal is to maximize the Worst-Case Secrecy Energy Efficiency (WCSEE), defined as the ratio of the minimum secrecy rate among legitimate users to the total power consumption, subject to energy harvesting requirements for the UEHRs.

The resulting optimization problem is highly nonconvex due to the fractional objective, the coupling between geometric channel relationships and control variables, the discrete nature of RIS phases, and the robustness constraints imposed by CSI uncertainty.

Methodology
To tackle the intractability of the WCSEE maximization problem, the authors propose two complementary approaches:

Model-Based Benchmark (BCD-SCA):
- A Block Coordinate Descent (BCD) framework is developed, alternating between optimizing power allocation, RIS phase shifts, and UAV location.
- Successive Convex Approximation (SCA) is employed to handle nonconvexities. For power allocation, the Charnes–Cooper transformation converts the fractional program into a convex form. For RIS phases and UAV location, first-order Taylor expansions and arithmetic-geometric mean (AGM) inequalities are used to construct convex surrogate constraints.
- This approach assumes idealized conditions (perfect CSI and continuous phases) to serve as a structured benchmark.
Learning-Based Solution (Soft Actor-Critic - SAC):
- To address the general WCSEE problem with discrete phases and CSI uncertainty without iterative solving for every channel realization, a customized Deep Reinforcement Learning (DRL) framework based on Soft Actor-Critic (SAC) is proposed.
- MDP Formulation: The problem is modeled as a Markov Decision Process where the state includes channel estimates and UAV position, the action jointly controls power, phases, and location, and the reward is the WCSEE (zero if energy harvesting constraints are violated).
- Architecture: The SAC agent utilizes an actor network (Gaussian policy) and twin critic networks to mitigate overestimation bias. Entropy regularization is used to encourage exploration and ensure stable convergence.
- Handling Constraints: Discrete RIS phases are managed by mapping continuous SAC outputs to the discrete set, and hard constraints (like energy harvesting) are enforced through the reward shaping mechanism.

Key Contributions

System Formulation: The paper formulates a novel secure UAV-RIS assisted SWIPT system that jointly considers UAV location, RIS configuration, and power allocation under imperfect CSI and untrusted receivers, aiming to maximize WCSEE.
Optimization Framework: It develops a BCD-SCA based algorithm that efficiently solves a tractable secrecy energy efficiency (SEE) formulation, providing a fast-converging benchmark.
DRL Innovation: It proposes a tailored SAC-based approach for the general WCSEE problem. This method effectively captures the high-dimensional coupling among variables and avoids the repeated iterative optimization required by conventional methods, shifting computational burden to offline training.
Performance Validation: The study provides a comprehensive comparison against conventional optimization (SCA) and other DRL benchmarks (DDPG, TD3).

Results
Simulation results demonstrate the superiority of the proposed SAC approach:

Performance Gain: The SAC method achieves up to a 28% improvement in SEE compared to the SCA-based benchmark and approximately 16% improvement over Deep Deterministic Policy Gradient (DDPG). It also outperforms Twin Delayed DDPG (TD3).
Robustness: The SAC approach maintains stable performance and robustness across varying system configurations, including different numbers of RIS elements and transmit power levels. It demonstrates strong resilience to CSI uncertainty (varying uncertainty radii $\nu$ ).
Convergence: SAC exhibits faster convergence and higher average rewards compared to DDPG and TD3, even with small mini-batch sizes.
Scalability: Unlike SCA, which becomes computationally intensive with high-dimensional coupled variables, the SAC method offers low-complexity online deployment after offline training.

Significance
The paper claims that its primary significance lies in bridging the gap between theoretical optimization and practical deployment in secure UAV-RIS systems. While existing works often treat SWIPT, security, or UAV deployment in isolation or assume idealized models, this framework unifies these elements under realistic constraints (imperfect CSI, discrete phases). By demonstrating that a model-free DRL approach (SAC) can outperform rigorous model-based optimization (SCA) in complex, coupled, and uncertain environments, the paper suggests a paradigm shift toward learning-based control for next-generation secure wireless networks. The results indicate that SAC is particularly well-suited for handling the nonconvexity and high dimensionality inherent in joint UAV-RIS resource allocation, offering a scalable solution for future beyond-5G networks.