⚡ electrical engineering

Deep Reinforcement Learning for Interference Suppression in RIS-Aided Space-Air-Ground Integrated Networks

This paper proposes a deep deterministic policy gradient (DDPG) algorithm for a reconfigurable intelligent surface (RIS)-aided space-air-ground integrated network to dynamically optimize HAPS beamforming weights, effectively suppressing cross-tier interference from antenna back-lobes and achieving up to an 11.3% throughput improvement over conventional zero-forcing methods.

Original authors: Pujitha Mamillapalli, Shikhar Verma, Tiago Koketsu Rodrigues, Abhinav Kumar

Published 2026-07-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pujitha Mamillapalli, Shikhar Verma, Tiago Koketsu Rodrigues, Abhinav Kumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world's internet is about to get a massive upgrade, stretching from the deepest oceans to the farthest stars. This is the dream of "6G," a future where your phone never loses signal, whether you are in a subway, on a mountain, or floating in space. To make this happen, engineers are building a giant, three-layered web called a Space-Air-Ground Integrated Network (SAGIN). Think of it like a relay race: satellites pass the baton to high-flying balloons or drones (called High-Altitude Platform Stations, or HAPS), which then pass it to people on the ground.

But there's a catch. In this crowded relay race, the runners are trying to talk to each other using the same walkie-talkie frequency. The satellite talking up to the HAPS can accidentally blast noise into the HAPS's ear, drowning out the signal the HAPS is trying to send down to you. It's like trying to listen to a friend whisper while someone next to you is screaming into a megaphone. To fix this, scientists are using a new trick called a Reconfigurable Intelligent Surface (RIS). Imagine a wall covered in thousands of tiny, magical mirrors that can instantly change their angle to bounce signals exactly where they need to go, dodging the noise. The big question is: how do we tell these mirrors and the flying station exactly what to do when the wind changes and the users move? That's where the story of this paper begins.

This paper tackles the tricky problem of that "screaming megaphone" interference in our future 6G sky-web. The authors, a team of researchers from India and Japan, propose a clever solution using a type of artificial intelligence called Deep Reinforcement Learning (DRL). Specifically, they use an algorithm named DDPG (Deep Deterministic Policy Gradient).

To understand their approach, picture the HAPS not just as a tower, but as a conductor of a massive orchestra. The conductor needs to tell the musicians (the antennas) exactly how loud to play and in which direction to aim their sound so that the audience (the ground users) hears a beautiful melody, while the noise from the satellite (the back-lobe interference) is completely silenced. Traditional methods try to solve this by using a rigid rulebook (called Zero-Forcing or ZF) that tries to mathematically cancel out the noise. However, the authors argue that in a fast-changing, dynamic sky, this rulebook is too slow and often gets the math wrong, leading to a messy performance.

Instead, the authors trained their "conductor" (the DDPG AI) to learn by doing. They set up a simulation where the AI acts as the brain of the HAPS. Every time the AI tries a new way to aim the antennas, it gets a score (a reward). If it successfully silences the interference and keeps the connection strong, it gets a high score. If it fails, it gets a penalty. Over thousands of tries, the AI learns the perfect dance moves to steer the signal beams, creating "spatial nulls"—invisible pockets of silence that push the interference away while keeping the desired signal strong.

The paper simulates this scenario with a specific setup: one satellite, one HAPS flying at 20,000 meters, and a 4x4 grid of intelligent mirrors (RIS) on the ground. The results from these simulations show that the AI-driven approach is indeed better than the old rulebook method. In their tests, the DDPG framework managed to boost the data speed (throughput) by up to 11.3% compared to the traditional method when using a 4x4 RIS configuration. The authors also found that giving the system more power to work with helped the AI learn faster and perform better, but even with limited power, the AI adapted well.

The paper explicitly rules out the idea that traditional, static mathematical solutions (like the Zero-Forcing codebooks) are sufficient for these highly dynamic environments. They argue that because the sky is always changing, you need a system that can learn and adapt in real-time rather than one that just follows a pre-written script. However, the authors are careful to note that these are simulation results, not real-world field tests. They acknowledge that their current model assumes perfect knowledge of the signals, which is a simplification, and that future work will need to deal with hardware imperfections and real-world unpredictability.

In short, this paper suggests that by teaching our future flying cell towers to "think" and adapt using deep learning, we can silence the noise of our crowded sky and ensure that the 6G internet remains fast and reliable, no matter how chaotic the environment gets. It's a promising step toward a world where your connection is as seamless as the air you breathe.

Technical Summary: Deep Reinforcement Learning for Interference Suppression in RIS-Aided Space–Air–Ground Integrated Networks

Problem Statement
The paper addresses the challenge of inter-link interference in Space–Air–Ground Integrated Networks (SAGINs) envisioned for 6G. Specifically, it focuses on High-Altitude Platform Station (HAPS) systems where the satellite uplink and ground downlink share the same frequency spectrum to maximize efficiency. This frequency sharing creates severe interference because the back-lobe radiation from the HAPS uplink antenna (directed toward the satellite) couples into the co-located downlink array, degrading the signal quality for ground users.

Traditional mitigation techniques, such as Zero-Forcing (ZF) beamforming and codebook-based null steering, face significant limitations in this context. They rely on discrete codebooks and require full channel state information (CSI), leading to quantization errors and suboptimal performance in highly dynamic channel conditions. Furthermore, conventional optimization methods are computationally expensive and struggle to scale with the exponential configuration space introduced by Reconfigurable Intelligent Surfaces (RIS).

Methodology
To overcome these limitations, the authors propose a RIS-aided HAPS framework optimized using a Deep Deterministic Policy Gradient (DDPG) algorithm, a model-free Deep Reinforcement Learning (DRL) approach.

System Model: The architecture consists of a satellite layer, an aerial HAPS layer equipped with Uniform Planar Arrays (UPAs) for both uplink and downlink, and a ground layer featuring a RIS and multiple users. The RIS, composed of passive reflecting elements, reshapes the propagation environment to enhance desired signals and suppress interference without additional power.
Optimization Objective: The goal is to minimize the total transmit power of the HAPS while satisfying a minimum Signal-to-Interference-plus-Noise Ratio (SINR) for all ground users and enforcing zero interference (nulling) toward unintended receivers.
DDPG Framework:
- State: The system state is defined by the composite channel matrix ( $H_{HAPS}$ ), including real and imaginary components of the CSI.
- Action: The agent outputs the continuous beamforming matrix ( $W$ ) for the HAPS antenna array.
- Reward: The reward function is shaped to penalize constraint violations (SINR and power limits) and reward high spectral efficiency and interference suppression.
- Architecture: The framework utilizes four neural networks (Training Actor, Training Critic, Target Actor, Target Critic) with fully connected layers, batch normalization, and PReLU activation functions. The agent learns optimal beamforming strategies through interaction with the environment, adjusting weights to form spatial nulls toward interference sources while maintaining robust links to desired users.

Key Contributions

Novel Framework: The paper introduces a DRL-driven approach to optimize HAPS beamforming in RIS-aided SAGINs, specifically targeting the suppression of back-lobe interference in shared-spectrum scenarios.
Adaptive Null Directivity: Unlike static ZF methods, the proposed DDPG framework dynamically learns to form antenna nulls toward interference sources while preserving Quality of Service (QoS) for intended users, adapting to time-varying channel conditions.
Performance Validation: The study provides a comparative analysis demonstrating that the DDPG approach outperforms conventional ZF beamforming across different RIS configurations and user distributions.

Results
Simulations were conducted in a Python environment using a 28 GHz carrier frequency, a 4×4 RIS configuration, and varying numbers of ground users. Key findings include:

Throughput Improvement: The DDPG framework consistently achieved higher throughput than ZF beamforming. For a 4×4 RIS configuration, the proposed method yielded a throughput improvement of up to 11.3% (specifically 11.34% for $\alpha=1$ and 10.99% for $\alpha=2$ in fairness-weighted scenarios).
Scalability and Robustness: The DDPG approach maintained superior performance across different user distributions (Poisson, Normal, Uniform) and demonstrated a widening performance gap over ZF as the number of ground users increased.
Convergence: The system showed stable convergence, with higher transmit power budgets enabling faster learning and higher asymptotic rewards. The study identified that a two-layer hidden neural network architecture offered the optimal balance between expressivity and generalization for this specific problem.

Significance and Claims
The paper claims that the proposed DDPG-based framework provides a scalable, energy-efficient, and adaptive solution for interference mitigation in next-generation 6G non-terrestrial networks. By leveraging the design of HAPS beamforming and the flexibility of RIS, the approach effectively suppresses interference arising from frequency sharing between uplink and downlink transmissions without requiring full channel knowledge or complex iterative solvers.

The authors position this work as a step toward enhancing spectral efficiency and spatial isolation in dynamic SAGIN environments. They note that while the current study focuses on a reduced-scale configuration for computational feasibility, the learned strategies are theoretically sound and provide a reliable scaling approach for larger deployments. Future work is identified as extending the framework to larger-scale networks with more users and RIS elements, as well as integrating practical considerations such as hardware impairments and imperfect CSI.

Technical Summary: Deep Reinforcement Learning for Interference Suppression in RIS-Aided Space–Air–Ground Integrated Networks

More like this