⚡ electrical engineering

DRL-Based Phase Optimization for O-RIS in Dual-Hop Hard-Switching FSO/RIS-aided RF and UWOC Systems

This paper proposes a DRL-based framework utilizing TD3 and DDPG algorithms to optimize phase shifts in an optical reconfigurable intelligent surface (O-RIS) for a dual-hop hybrid FSO/RF-UWOC system, demonstrating significant improvements in outage probability and channel capacity under realistic oceanic turbulence conditions.

Original authors: Aboozar Heydaribeni, Hamzeh Beyranvand, Sahar Eslami

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aboozar Heydaribeni, Hamzeh Beyranvand, Sahar Eslami

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a secret message from a submarine deep underwater to a satellite orbiting high above the Earth. This is the ultimate "cross-domain" communication challenge, and it's incredibly difficult because the ocean and the sky are both hostile environments.

This paper proposes a clever, futuristic solution to this problem, acting like a high-tech relay race that uses the best tools for each leg of the journey, controlled by an AI coach.

Here is the breakdown of their system using simple analogies:

1. The Two-Leg Relay Race

The system is a "dual-hop" network, meaning the message travels in two distinct stages:

Leg 1: The Sky (FSO & RF)
- The Problem: The first part of the journey is through the air. Usually, we use lasers (Free-Space Optical or FSO) because they are super fast, like a bullet train. But lasers hate bad weather (fog, rain, clouds).
- The Backup: To fix this, the system has a "Plan B." If the laser gets blocked by fog, it instantly switches to Radio Frequency (RF), which is like a sturdy, slow-moving truck that can drive through the storm.
- The "Mirror" (RIS): To make sure the signal reaches its destination, they use a Reconfigurable Intelligent Surface (RIS). Think of this as a giant, smart mirror on a building. If the direct path is blocked, the mirror catches the signal and bounces it perfectly to the receiver.
Leg 2: The Ocean (UWOC & O-RIS)
- The Problem: The second part is underwater. Here, lasers are used again (Underwater Wireless Optical Communication or UWOC). But underwater is messy: salt, temperature changes, and tiny particles create "turbulence" (like looking at the bottom of a swimming pool on a windy day). This distorts the light.
- The Solution: They use an Optical RIS (O-RIS). This is a special underwater mirror made of thousands of tiny, adjustable tiles.

2. The "Smart Coach" (Deep Reinforcement Learning)

This is the most exciting part. In the past, engineers had to manually calculate how to angle every single tiny tile on the underwater mirror to focus the light. But the ocean is always changing (waves, currents, temperature shifts), so the calculations would be outdated the second they were made.

Instead, the authors used Deep Reinforcement Learning (DRL), which is essentially an AI Coach.

How it works: Imagine the AI Coach is playing a video game where the goal is to keep the light beam focused.
- The State: The AI looks at the current conditions (how choppy the water is, how much light is getting lost).
- The Action: The AI instantly adjusts the angle of every tiny tile on the O-RIS mirror.
- The Reward: If the message gets through clearly, the AI gets a "point." If the signal breaks, it loses a point.
The Learning: The AI tries millions of times (in simulation) to figure out the perfect way to tilt the mirrors. It learns that "When the water is salty and cold, tilt the mirrors this way." When the water is warm, it learns to tilt them that way.

They tested two specific AI algorithms: DDPG and TD3. Think of TD3 as the "veteran coach" who is more careful and less likely to make mistakes, while DDPG is the "enthusiastic rookie." The paper found that the veteran coach (TD3) was better at keeping the connection stable.

3. Why This Matters (The "So What?")

The authors simulated this system and found some amazing results:

Fewer Dropped Calls: The system rarely loses the signal (low "outage probability"), even in rough seas or bad weather.
Faster Speeds: Because the mirrors are perfectly aligned by the AI, the data capacity (how much information can be sent) is much higher.
Future-Ready: This is a blueprint for 6G networks. It envisions a world where your phone, a drone, a submarine, and a satellite are all connected seamlessly, regardless of whether you are in a storm, deep underwater, or in space.

Summary Analogy

Think of this system as a high-stakes game of "Pin the Tail on the Donkey," but instead of a blindfolded child, you have:

Two different playing fields (Sky and Ocean).
A backup plan (switching from laser to radio if it rains).
A wall of thousands of tiny mirrors (the RIS) that need to be angled perfectly to bounce the message to the target.
An AI Coach that watches the wind and waves in real-time and instantly tells every mirror where to point, ensuring the message never misses its target.

The paper proves that using this AI coach is much smarter and more reliable than trying to calculate the angles with old-school math, paving the way for a truly connected future across land, sea, and sky.

1. Problem Statement

The paper addresses the challenge of establishing reliable, high-capacity, cross-domain communication networks for the Internet of Underwater Things (IoUT) and 6G ecosystems. Specifically, it tackles the limitations of isolated communication links in heterogeneous environments:

FSO/RF Links: Terrestrial Free-Space Optical (FSO) links offer high bandwidth but suffer from atmospheric turbulence, weather, and pointing errors. While hybrid FSO/Radio Frequency (RF) systems with hard-switching provide a backup, optimizing the RF link using Reconfigurable Intelligent Surfaces (RIS) in dynamic conditions remains complex.
Underwater Links: Underwater Wireless Optical Communication (UWOC) is hindered by absorption, scattering, and severe turbulence caused by oceanic factors (temperature, salinity).
Optimization Gap: Existing methods for optimizing Optical RIS (O-RIS) phase shifts often rely on model-based or heuristic approaches that struggle with the high-dimensional, non-convex, and time-varying nature of underwater channels. There is a lack of an integrated End-to-End (E2E) framework that jointly considers realistic oceanic turbulence modeling, hard-switching protocols, and adaptive, real-time O-RIS control.

2. Methodology

The authors propose a unified dual-hop hybrid framework and a Deep Reinforcement Learning (DRL) control strategy.

A. System Architecture

The system consists of two hops connected via a Decode-and-Forward (DF) relay:

First Hop (Underwater): An O-RIS-assisted UWOC link. The signal travels from a source to an O-RIS and then to a DF relay.
Second Hop (Terrestrial): A hybrid FSO/RIS-aided RF link operating under a hard-switching protocol.
- The FSO link is the primary channel.
- If the FSO Signal-to-Noise Ratio (SNR) drops below a predefined threshold, the system switches to the RF link (assisted by a standard RIS) as a backup.

B. Channel Modeling

Underwater Channel: Modeled using the Oceanic Turbulence Optical Power Spectrum (OTOPS). This physics-based model incorporates environmental variables (temperature, salinity, depth) via the Gibbs SeaWater (GSW) toolbox to accurately predict turbulence-induced fading. The model accounts for:
- Path loss (Beer–Lambert law).
- Turbulence fading (modeled as a cascaded Gamma–Gamma distribution).
- Pointing errors (due to transmitter jitter and surface misalignment).
Terrestrial Channel:
- FSO: Modeled using the Gamma–Gamma distribution for atmospheric turbulence.
- RF: Modeled with Rayleigh fading and 3GPP Urban Micro (UMi) path loss, enhanced by an RIS for coherent signal addition.

C. DRL-Based Phase Optimization

To solve the non-convex problem of optimizing O-RIS phase shifts in real-time, the authors formulate the problem as a Markov Decision Process (MDP):

State Space ( $S$ ): Includes instantaneous channel coefficients (magnitude and phase) and temporally correlated outdated estimates to capture channel dynamics.
Action Space ( $A$ ): The continuous phase shifts of $N$ O-RIS elements, constrained by hardware limitations (quantized to discrete levels).
Reward Function ( $R$ ): Designed to maximize throughput and reliability while penalizing abrupt phase changes. It incorporates Finite Block Length (FBL) effects to calculate decoding error probability and achievable rates.
Algorithms: Two actor-critic DRL algorithms are implemented and compared:
1. Deep Deterministic Policy Gradient (DDPG).
2. Twin Delayed DDPG (TD3): Selected for its superior stability, utilizing clipped double Q-learning, target policy smoothing, and delayed policy updates to mitigate Q-value overestimation.

3. Key Contributions

Integrated Hybrid Framework: Proposes a novel E2E architecture combining O-RIS-assisted UWOC with hard-switching FSO/RIS-aided RF, bridging the gap between underwater and terrestrial 6G networks.
Realistic Oceanic Modeling: Moves beyond simplified turbulence models by employing the OTOPS model, which dynamically accounts for temperature and salinity profiles, providing a more accurate representation of underwater channel fluctuations.
DRL-Driven Control: Introduces a model-free DRL approach (specifically TD3) for continuous O-RIS phase optimization. This avoids the computational complexity of traditional convex optimization in dynamic environments.
Hardware-Aware Design: The action space and reward function explicitly consider practical hardware constraints, such as phase resolution bits and amplitude-phase coupling in O-RIS elements.

4. Simulation Results

The study uses Monte-Carlo simulations ( $10^6$ runs) to validate the analytical models and compare performance:

Impact of O-RIS Elements: Increasing the number of O-RIS elements significantly reduces the Outage Probability (OP) and improves channel capacity.
Environmental Factors: Lower water temperature and salinity levels lead to better performance (lower OP), validating the sensitivity of the OTOPS model.
Hard-Switching Performance: The hybrid FSO/RIS-aided RF system shows substantial gains over standalone links. The optimal switching threshold balances the reliance on the high-capacity FSO link versus the robustness of the RF backup.
DRL Performance Comparison:
- TD3 vs. DDPG: TD3 consistently outperforms DDPG and non-learning baselines. It achieves a steeper decline in outage probability and higher average rewards.
- Robustness: TD3 demonstrates superior adaptability to channel variations, achieving the highest channel capacity across various SNR regimes.
- Capacity: The DRL-enabled system significantly boosts channel capacity compared to systems without learning or with static optimization.

5. Significance and Future Outlook

6G Enabler: This work provides a critical blueprint for cross-domain 6G networks, demonstrating how AI can manage the complexity of integrating underwater, aerial, and terrestrial links.
IoUT Reliability: By leveraging O-RIS and DRL, the proposed system offers a viable solution for mission-critical applications like autonomous underwater vehicles (AUVs) and environmental monitoring, where link reliability is paramount.
Future Directions: The authors suggest extending the framework to cooperative multi-RIS scenarios and integrating Integrated Sensing and Communication (ISAC) techniques to further enhance situational awareness and spectrum efficiency in 6G networks.

In conclusion, the paper successfully demonstrates that TD3-based DRL is a highly effective tool for optimizing O-RIS in complex, hybrid underwater-terrestrial communication systems, offering a robust path toward reliable high-speed 6G connectivity.