⚡ electrical engineering

Tertiary-Mode STAR-RIS for Secure NOMA: Integrating Transmission, Reflection, and Jamming

This paper proposes a secure NOMA system leveraging a tertiary-mode STAR-RIS capable of simultaneous transmission, reflection, and jamming, where a penalty-based alternating optimization algorithm jointly designs beamforming and mode selection to significantly enhance secrecy performance and sum rates compared to conventional benchmarks.

Original authors: Mansi Nema, Kuntal Deka, Sanjeev Sharma, Tharmalingam Ratnarajah

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mansi Nema, Kuntal Deka, Sanjeev Sharma, Tharmalingam Ratnarajah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to have a secret conversation with a friend in a crowded, noisy room, but there's a spy listening in. This paper presents a clever new way to use a "smart wall" to make sure your friend hears you clearly while the spy hears only static.

Here is the breakdown of the paper using simple analogies:

1. The Problem: The "One-Sided" Mirror

Traditionally, wireless networks use RIS (Reconfigurable Intelligent Surfaces). Think of these as smart mirrors on a wall.

How they worked before: They could only reflect signals. If you stood on one side of the mirror, it could bounce your signal to your friend. But if your friend stood on the other side of the mirror, the mirror was useless to them. It was like a mirror that only worked for people standing in front of it, not behind it.
The New Idea (STAR-RIS): The authors propose a STAR-RIS. Imagine a magical sheet of glass that is half-mirror and half-window. It can reflect signals to people on one side and transmit (let them pass through) to people on the other side simultaneously. This covers the whole room.

2. The Superpower: The "Triple-Threat" Mode

Most smart surfaces can only reflect or transmit. This paper introduces a Tertiary-Mode system. This means every single tiny piece of the smart surface can do three things at once, depending on what the network needs:

Reflect: Bounce the signal to a friend on the left.
Transmit: Let the signal pass through to a friend on the right.
Jam: This is the secret sauce. Instead of helping the signal, this mode acts like a noise machine. It blasts static interference specifically at the spy (the eavesdropper) to drown out their ability to listen in.

3. The Setup: The "NOMA" Dinner Party

The system uses a technique called NOMA (Non-Orthogonal Multiple Access).

The Analogy: Imagine a waiter (the Base Station) carrying a tray with two drinks: a strong coffee for User A and a weak tea for User B.
The Trick: The waiter pours both drinks into the same cup but mixes them in a specific way. User A (who is closer and has a better "taste") can drink the coffee and filter out the tea. User B (who is further away) can only taste the tea, but because User A removed the coffee first (a process called SIC or Successive Interference Cancellation), User B gets their tea clearly.
The Spy: The spy is trying to taste the drinks too. The goal is to make sure the spy can't figure out what either user is drinking.

4. The Solution: The "Smart Conductor"

The hardest part is figuring out exactly how to tilt every tiny piece of the smart wall. Should piece #1 reflect? Should piece #2 jam the spy? Should piece #3 let the signal through?

The Challenge: There are millions of combinations. It's like trying to solve a Rubik's cube while blindfolded.
The Algorithm: The authors created a "Smart Conductor" (an algorithm). It works in two steps, back and forth:
1. Step 1 (The Waiter): It adjusts how the waiter (Base Station) holds the tray (beamforming) to aim the signal best.
2. Step 2 (The Wall): It adjusts the smart wall. It decides which pieces should reflect, which should transmit, and which should blast noise at the spy.
3. The Penalty: The algorithm uses a "penalty" system. If it tries a solution that isn't perfect (like a piece of the wall trying to do two things at once), it gets "fined" (a penalty score). The algorithm keeps adjusting to lower the fine until it finds the perfect balance.

5. The Results: Why It Matters

The simulations in the paper show that this new system is a game-changer:

More Speed: Because the wall can talk to people on both sides, the total data speed (sum-rate) goes up significantly compared to old mirrors.
Better Security: By turning some parts of the wall into "noise machines" specifically for the spy, the spy hears nothing but garbage. The legitimate users hear everything clearly.
Adaptability: If the spy gets closer, the wall automatically shifts more pieces to "Jamming" mode. If the signal is weak, it shifts more pieces to "Transmission" mode. It's like a chameleon changing its colors to survive.

Summary

This paper introduces a super-smart wall that doesn't just bounce signals; it can bounce, pass through, and blast noise all at the same time. By using a smart computer algorithm to decide exactly what each tiny piece of the wall should do, the system ensures that two friends can chat securely and quickly, while a spy listening in is completely confused by static noise. It's a major step forward for 6G security and speed.

1. Problem Statement

The paper addresses the challenge of enhancing Physical Layer Security (PLS) and spectral efficiency in Non-Orthogonal Multiple Access (NOMA) systems operating in complex wireless environments.

Context: Traditional Reconfigurable Intelligent Surfaces (RIS) are limited to reflection, serving users on only one side. While Simultaneously Transmitting and Reflecting RIS (STAR-RIS) solves the coverage issue, most existing studies use Energy Splitting (ES) or Time Switching (TS) protocols, which lack flexibility.
The Gap: There is a need for a more adaptive STAR-RIS protocol that can dynamically switch modes to not only serve legitimate users but also actively disrupt eavesdroppers.
Specific Scenario: The authors consider a Multi-Input Single-Output (MISO) NOMA system where a Base Station (BS) serves two legitimate users located on opposite sides of a STAR-RIS. A passive eavesdropper (Eve) is present on the reflection side. The goal is to maximize the sum-rate of legitimate users while ensuring the secrecy rate (limiting information leakage to Eve) by utilizing a subset of RIS elements for jamming.

2. Methodology

The authors propose a Tertiary-Mode STAR-RIS framework and a corresponding optimization algorithm.

A. System Model & Tertiary Modes

Unlike standard STAR-RIS which splits energy or time, this model allows each of the $K$ RIS elements to operate in one of three mutually exclusive modes:

Reflection: Serves the user on the reflection side.
Transmission: Serves the user on the transmission side.
Jamming: Actively reflects a jamming signal to degrade the eavesdropper's channel.

Constraint: For each element $k$ , the mode selection coefficients satisfy $\beta_{r,k} + \beta_{t,k} + \beta_{j,k} = 1$ , where $\beta \in \{0, 1\}$ .

B. Optimization Problem

The objective is to jointly optimize:

Active Beamforming: The transmit vectors ( $w_1, w_2$ ) at the multi-antenna BS.
Passive Beamforming: The phase shifts ( $\theta$ ) of the RIS elements.
Mode Selection: The binary selection of reflection, transmission, or jamming for each element.

The problem is formulated to maximize the sum-rate ( $R_{11} + R_{22}$ ) subject to:

BS transmit power constraints.
NOMA Successive Interference Cancellation (SIC) decoding order constraints.
Secrecy constraints (limiting the eavesdropper's achievable rate to a threshold $\tau$ ).
Binary and rank-one constraints on the RIS matrices.

C. Proposed Solution: Penalty-Based Alternating Optimization (AO)

The problem is non-convex due to variable coupling and discrete binary variables. The authors develop a Dual-Stage Penalty Optimization Algorithm:

Active Beamforming Optimization (Stage 1):
- Fixes RIS parameters and optimizes BS beamforming vectors.
- Uses Semidefinite Relaxation (SDR) to handle rank-one constraints on covariance matrices.
- Introduces slack variables and Second-Order Cone (SOC) constraints to convexify the fractional SINR expressions and SIC ordering constraints.
Passive Beamforming & Mode Selection (Stage 2):
- Fixes BS beamforming and optimizes RIS phase shifts and mode selection.
- Binary Relaxation: Handles the discrete $\{0, 1\}$ mode selection by relaxing it to $[0, 1]$ and adding a penalty term to the objective function. This penalty is iteratively increased to force convergence to binary values.
- Rank-One Relaxation: Handles the rank-one constraint on RIS matrices by minimizing the difference between the nuclear norm and the spectral norm (using a first-order gradient approximation).
- One-Hot Encoding: After solving the relaxed convex problem, a post-processing step selects the mode with the highest value for each element to ensure strict exclusivity.

The algorithm iterates between these two stages until convergence.

3. Key Contributions

Novel Framework: Proposes the first secure NOMA system utilizing a tertiary-mode STAR-RIS capable of simultaneous transmission, reflection, and jamming.
Joint Optimization Formulation: Formulates a complex joint optimization problem to maximize sum-rate while strictly enforcing secrecy constraints against an eavesdropper.
Algorithm Development: Develops a penalty-based alternating optimization algorithm that effectively handles the mixed-integer non-convex nature of the problem by unifying continuous (phase shifts) and discrete (mode selection) variables.
Performance Validation: Demonstrates through simulations that the proposed design significantly outperforms conventional RIS, standard STAR-RIS (without jamming), and systems without RIS.

4. Simulation Results

The paper validates the proposed scheme using MATLAB simulations with the following key findings:

Sum-Rate Improvement: The proposed "Tertiary-Mode STAR-RIS with Jamming" consistently outperforms all benchmarks (Conventional RIS, STAR-RIS without jamming, No-RIS) across all transmit power levels (0–40 dBm).
Secrecy Gain: The system achieves a notable improvement of approximately 1 bit/s/Hz in secrecy rate compared to conventional RIS-assisted systems.
Impact of Element Count ( $K$ ): Increasing the number of RIS elements improves performance, but with diminishing returns observed beyond $K=50$ .
Adaptive Mode Allocation: The system dynamically adjusts element modes based on transmit power. At low power, most elements serve transmission/reflection; at high power, a significant portion switches to jamming mode to suppress the eavesdropper.
Convergence: The proposed algorithm converges rapidly (typically within 6–10 iterations), demonstrating computational efficiency.

5. Significance

This work highlights the potential of active security measures integrated into passive intelligent surfaces. By enabling RIS elements to act as jammers, the system transforms the RIS from a passive signal enhancer into an active security asset. This approach offers a robust solution for 6G networks, providing full-space coverage (via STAR-RIS) and enhanced physical layer security without the need for additional active hardware or complex relay networks. The proposed penalty-based AO algorithm provides a practical, low-complexity method for deploying such advanced secure communication systems.