Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

Technical Summary: Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

Problem Statement
Industrial 6G networks face critical challenges in meeting ultra-reliable, low-latency communication (URLLC) and energy efficiency requirements within dynamic, blockage-prone environments. Traditional terrestrial cellular infrastructures often fail to provide stable coverage in dense industrial settings characterized by large-scale factories, logistics hubs, and harsh propagation conditions. While integrating non-terrestrial networks (NTN) via Unmanned Aerial Vehicles (UAVs) and High-Altitude Platforms (HAPs) offers a solution, the joint optimization of resource allocation, Reconfigurable Intelligent Surface (RIS) phase shifts, spectrum sharing, and computing offloading in such heterogeneous Terrestrial/Non-Terrestrial (TN/NTN) environments creates a high-dimensional, strongly coupled problem. Conventional centralized optimization techniques are computationally intractable for these dynamic scenarios, and existing learning-based approaches often lack the integration of RIS-enabled propagation control, TN/NTN coordination, and Open-RAN intelligence.

Methodology
The authors propose a RIS-enabled Open-RAN framework for integrated TN/NTN industrial 6G networks. The system architecture comprises:

• Terrestrial Layer: Ground Radio Units (GRUs) and distributed Industrial IoT (IIoT) devices.
• Non-Terrestrial Layer: A HAP equipped with a Multi-Access Edge Computing (MEC) server and multiple UAVs mounted with RISs.
• Control Plane: Open-RAN architecture utilizing Near-RT and Non-RT RAN Intelligent Controllers (RICs) for disaggregated control.

The core technical approach involves modeling the joint optimization of data rates, latency, and energy consumption as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP).

• System Model: The framework considers direct links (GRU-IIoT, HAP-IIoT) and indirect RIS-assisted links (UAV-RIS-IIoT). Channel models account for Line-of-Sight (LoS) and Non-Line-of-Sight (NLoS) conditions, utilizing Rician fading for aerial links and path loss models for terrestrial links.
• Optimization Objective: The goal is to maximize a utility function that balances communication throughput (logarithmic rate utility) against penalties for latency and energy consumption, subject to constraints on bandwidth, transmit power, computing capacity, RIS phase shifts, and QoS requirements.
• Solution Algorithm: To solve the Dec-POMDP, the authors employ a Multi-Agent Proximal Policy Optimization (MAPPO) framework.
  • Agents: GRUs, HAP, and UAV-mounted RIS controllers act as independent agents.
  • Observations: Agents operate based on local observations (channel conditions, traffic demands, residual energy) rather than global state information.
  • Training: A centralized critic utilizes global state information to estimate the joint value function, mitigating non-stationarity, while agents update their policies (actors) using a clipped surrogate objective.
  • Actions: Agents jointly determine user association, bandwidth allocation, transmission power, computing resource allocation, and RIS phase shift configurations.

Key Contributions

1. Architecture: Development of a RIS-enabled Open-RAN architecture for integrated TN/NTN industrial 6G networks, leveraging UAV-mounted RISs to enhance coverage and reliability in challenging environments.
2. Formulation: A comprehensive optimization problem formulation that captures computing resources, RIS configuration, spectrum sharing, and QoS provisioning under practical latency and energy constraints.
3. Algorithm: Proposal of a Multi-Agent Deep Reinforcement Learning (MADRL) framework (specifically MAPPO) that enables scalable, adaptive, and decentralized control of learned policies without requiring full global state knowledge.
4. Validation: Extensive simulations comparing the proposed approach against state-of-the-art baselines, including Multi-Agent Actor-Critic (MAAC), MAPPO without RIS, and a greedy baseline.

Results
Simulation results, conducted in a 1000m × 1000m industrial area with 100 IIoT devices, demonstrate the effectiveness of the proposed framework:

• Data Rate: The proposed MAPPO with RIS achieves a steady-state data rate of approximately 220 Mbps, representing a 75% improvement over MAPPO without RIS and a 25% improvement over the MAAC scheme.
• Latency: The framework achieves up to a 25% reduction in latency compared to MAPPO without RIS and a 20% reduction compared to MAAC.
• Energy Consumption: The approach yields up to a 16% energy saving compared to the MAAC approach and nearly 20% improvement over MAPPO without RIS.
• Convergence: The proposed method demonstrates rapid convergence and superior performance across varying network loads compared to greedy and non-RIS baselines.

Significance and Claims
The paper claims that the integration of RIS-assisted Open-RAN intelligence offers a viable solution for scalable and energy-efficient industrial 6G networks. By enabling decentralized control through MADRL, the framework overcomes the computational intractability of centralized optimization in dynamic industrial environments. The authors assert that their approach effectively balances the trade-offs between throughput, latency, and energy consumption, addressing the stringent requirements of mission-critical IIoT applications. The work highlights the potential of UAV-mounted RISs to mitigate blockage and coverage holes while maintaining spectral efficiency. Future work is identified as extending the framework to include dynamic UAV mobility, downlink transmission, and federated learning mechanisms to further enhance scalability and real-time adaptability.