⚡ electrical engineering

RL based Beamforming Optimization for 3D Pinching Antenna assisted ISAC Systems

This paper proposes a 3D pinching antenna deployment scheme for ISAC systems and develops a heterogeneous graph neural network-based reinforcement learning algorithm to jointly optimize antenna positioning, time allocation, and transmit power, demonstrating superior performance over 1D/2D counterparts and other baselines in maximizing sum communication rates under sensing and energy constraints.

Original authors: Qian Gao, Ruikang Zhong, Yue Liu, Hyundong Shin, Yuanwei Liu

Published 2026-01-29

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Qian Gao, Ruikang Zhong, Yue Liu, Hyundong Shin, Yuanwei Liu

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to host a party where you need to do two things at once: talk to your guests (communication) and keep an eye on the room to make sure everyone is safe (sensing). In the world of wireless technology, this is called ISAC (Integrated Sensing and Communication).

Usually, the "speakers" (antennas) that do this job are stuck in a straight line or a flat grid, like soldiers standing in a row. This paper proposes a new way to arrange them: 3D Pinching Antennas.

Here is the breakdown of the paper's ideas using simple analogies:

1. The Problem: The "Flat" Limitation

Think of traditional antennas like a row of people holding flashlights on a single straight line. They can shine light forward, but they can't easily look up, down, or to the side without moving their whole bodies. If a guest is standing on a balcony or a target is high up in the air, a flat line of antennas struggles to reach them efficiently. They waste energy trying to hit a target that is "out of their lane."

2. The Solution: The "Pinching" Antenna

The authors introduce a special technology called Pinching Antennas.

The Metaphor: Imagine a long, flexible garden hose (a waveguide) that has water flowing through it. Normally, the water just flows out the end. But with this special hose, you can "pinch" it at any point along its length, and water will spray out right there.
The Benefit: Instead of having fixed speakers, you have a flexible hose where you can turn on a "speaker" (antenna) anywhere you want. This is cheap and very flexible.

3. The Big Idea: The 3D "Starfish" Shape

Previous studies only used these hoses in a straight line (1D) or a flat grid (2D). This paper suggests arranging three hoses to form a 3D corner, like the corner of a room where the floor meets two walls (the X, Y, and Z axes).

Why it's better: It's like turning a flat flashlight into a 3D starfish. Now, the system can reach guests on the floor, on a balcony, or even flying in the air. It gives the system "spatial freedom" to point its beams exactly where they are needed without wasting energy.

4. The Challenge: The "Too Many Choices" Problem

Because the antennas can move anywhere in 3D space, and you also have to decide when to talk to each guest and how loud to speak, there are millions of possible combinations.

The Analogy: It's like trying to solve a Rubik's cube that keeps changing shape while you are also juggling. A human (or a standard computer program) would get overwhelmed trying to calculate the perfect move every second.

5. The Brain: HGRL (The "Smart Coach")

To solve this, the authors created a new AI coach called HGRL (Heterogeneous Graph Neural Network Reinforcement Learning).

How it works: Imagine a coach who doesn't just look at the players (users) and the ball (targets) as a messy crowd. Instead, the coach sees them as a special map (a graph) where every connection is labeled.
- "This antenna is talking to that user."
- "This antenna is watching that target."
- "These two antennas are interfering with each other."
The "Heterogeneous" part: This is the key. The coach knows the difference between a "User" and a "Target." It doesn't treat them all the same. This allows the AI to learn much faster and make smarter decisions than older AI methods that treat everything as a generic blob.

6. The Results: The Winner's Circle

The authors ran simulations (computer tests) to see how this new 3D setup compares to the old 1D and 2D setups.

The Outcome: The 3D Pinching Antenna setup won every time.
- Communication: It delivered more data (talked faster) to the users.
- Sensing: It kept the targets in view just as well, but used less power to do it.
- Efficiency: The old 1D and 2D setups wasted a lot of energy just trying to reach the targets. The 3D setup was so precise that it could meet the safety requirements with very little power left over, allowing it to focus that saved energy on talking to the users.

Summary

In short, this paper says: Stop arranging your antennas in flat lines. By arranging them in a 3D corner using flexible "pinching" technology and training a smart AI coach that understands the difference between people and objects, you can make wireless systems that are faster, smarter, and use less energy. The AI learned to move the antennas in 3D space to perfectly balance talking and watching, beating all the older, flatter methods.

Technical Summary: RL-based Beamforming Optimization for 3D Pinching Antenna assisted ISAC Systems

Problem Statement
Integrated Sensing and Communication (ISAC) systems aim to unify spectrum and hardware usage to improve spectral efficiency and situational awareness. While pinching antennas (PAs)—low-cost, reconfigurable dielectric-waveguide-based leaky antennas—offer significant potential for ISAC due to their mobility and ability to form on-demand Line-of-Sight (LoS) beams, existing research has largely confined PA deployment to 1D (single waveguide) or 2D (parallel waveguides in a plane) configurations. These lower-dimensional layouts limit angular coverage and elevation diversity, restricting the system's ability to serve airborne or elevated users and targets effectively.

Furthermore, optimizing a 3D PA-assisted ISAC system presents a complex challenge. The problem involves the joint optimization of continuous variables: 3D antenna positioning, Time-Division Multiple Access (TDMA) time allocation, and transmit power distribution. This creates a high-dimensional, non-convex, and non-linear optimization problem (due to fractional SNR expressions and exponential phase terms) that is difficult to solve using conventional hand-crafted optimization or standard Reinforcement Learning (RL) methods.

Methodology
To address these challenges, the authors propose a novel framework combining a 3D deployment scheme with a Heterogeneous Graph Neural Network-based Reinforcement Learning (HGRL) algorithm.

System Model: The authors model a downlink ISAC system where a Base Station (BS) utilizes three orthogonal waveguides (aligned with x, y, and z axes) equipped with pinching antennas. The system serves $K$ users and detects $L$ targets located in the $x-y$ plane. The optimization objective is to maximize the sum communication rate subject to constraints on target sensing rates (SNR), total energy budget, and minimum antenna spacing.
MDP Formulation: The optimization problem is formulated as a Markov Decision Process (MDP).
- State Space: Represents the system topology as a time-varying heterogeneous graph containing nodes for antennas, users, and targets, along with edges representing communication, sensing, and interference relationships.
- Action Space: A continuous vector comprising antenna displacements, TDMA weights, and power allocation.
- Reward Function: Designed to maximize the sum communication rate while penalizing violations of sensing SNR thresholds and physical constraints (e.g., antenna spacing).
HGRL Algorithm: The core solution is an HGRL framework utilizing the Advantage Actor-Critic (A2C) algorithm.
- Heterogeneous Graph Representation: Unlike homogeneous graph approaches, the proposed method explicitly encodes distinct node types (antenna, user, target) and relation types (communicates, senses, interference) using a Heterogeneous GNN (HGNN) encoder. This allows for precise modeling of ISAC-specific interactions.
- Policy and Value Networks: The encoded graph embeddings are fed into an actor network (for action selection) and a critic network (for value estimation) to jointly learn the optimal policy for beamforming and resource allocation.

Key Contributions

3D Deployment Scheme: The paper proposes and analyzes a 3D pinching antenna configuration using three orthogonal waveguides. This extends existing ISAC studies beyond 1D/2D layouts, offering full-space beam steering and additional spatial degrees of freedom to decouple communication and sensing beams.
HGRL Framework: The authors develop the first heterogeneous GNN-based RL framework specifically for 3D pinching antenna-assisted ISAC. This approach enables scalable optimization over graph-structured states and mixed continuous action spaces, addressing the limitations of conventional RL in handling complex topological relationships.
Joint Optimization Formulation: The study formulates a comprehensive joint optimization problem for antenna positioning, time allocation, and power control, capturing fine-grained resource control under realistic constraints.

Simulation Results
Numerical experiments were conducted in a simulated environment with 6 users and 1 target, comparing 1D, 2D, and 3D deployment strategies under different power constraints ( $p=0.1$ W and $p=0.02$ W).

Algorithm Performance: The proposed HGRL algorithm demonstrated superior performance compared to baselines (standard A2C, homogeneous GRL, and random configuration). While HGRL exhibited slightly slower convergence due to its complex hierarchical reasoning, it achieved the highest final reward and stability, validating the effectiveness of heterogeneous relational modeling.
Deployment Comparison: The 3D deployment consistently outperformed 1D and 2D counterparts in terms of sum communication rate.
- Under high power ($0.1$W), 3D deployment achieved an average rate of 7.13 bps/Hz, compared to 6.37 bps/Hz (2D) and 6.21 bps/Hz (1D).
- Under strict power constraints ($0.02$W), the advantage of 3D deployment became more pronounced. The 3D scheme achieved 4.70 bps/Hz, while 1D and 2D achieved 3.80 and 4.50 bps/Hz, respectively.
Resource Efficiency: The results indicate that 3D deployment allows the system to operate closer to the sensing SNR constraint threshold (e.g., 5.41 dB vs. the required 5 dB), thereby reducing unnecessary power consumption for sensing and reallocating more power to communication. In contrast, 1D and 2D deployments resulted in significantly higher sensing SNRs, indicating inefficient power usage that limited communication throughput.

Significance
The paper claims that the proposed 3D pinching antenna configuration, optimized via HGRL, offers a superior balance between sensing and communication performance, particularly in power-limited scenarios. By leveraging the spatial diversity of tri-axial waveguides, the system achieves flexible beam steering and efficient beam separation. The study concludes that the 3D structure maximizes communication rates while maintaining sensing capabilities near the required threshold, highlighting its potential for next-generation, resource-constrained ISAC applications.