Trustworthy AI-Driven Dynamic Hybrid RIS: Joint Optimization and Reward Poisoning-Resilient Control in Cognitive MISO Networks

This paper proposes a trustworthy, energy-aware hybrid reconfigurable intelligent surface (RIS) for cognitive MISO networks that dynamically switches between passive and active modes, optimized via a soft actor-critic deep reinforcement learning algorithm with a novel defense mechanism against reward poisoning attacks to ensure secure and efficient spectrum utilization.

The Gist

Imagine a bustling city where the airwaves are like a crowded highway. There are "Primary Users" (PUs)—the emergency vehicles and official buses—that have the right-of-way on specific lanes. Then there are "Secondary Users" (SUs)—your regular cars and delivery trucks—that need to get somewhere but can't use the main lanes without causing a crash.

This paper proposes a smart, energy-saving solution to help these "regular cars" get through the traffic safely and quickly, while also protecting the system from hackers trying to trick the traffic lights.

Here is the breakdown of the paper's ideas using simple analogies:

1. The Problem: The "Dead Zone" and the "Battery"

Sometimes, the road between your car (the transmitter) and your destination (the receiver) is blocked by a huge building or a mountain. You can't see the road, and the signal dies.

• The Old Solution: You could build a giant, powerful radio tower (an "Active" repeater) to shout the message over the building. But this tower eats a lot of electricity and costs a fortune to run.
• The Passive Solution: You could put up a giant, shiny mirror (a "Passive" Reconfigurable Intelligent Surface or RIS) to bounce the signal around the building. This uses almost no electricity, but if the signal is too weak to begin with, the mirror just bounces a whisper. It doesn't make the signal louder.

2. The Innovation: The "Smart Hybrid Mirror"

The authors created a Dynamic Hybrid RIS. Think of this as a smart mirror with a built-in solar panel and a battery.

• How it works: The mirror is connected to a "Power Beacon" (like a solar charger).
  • When the battery is low: It acts as a Passive Mirror. It just bounces the signal. It saves energy but doesn't boost the volume.
  • When the battery is full: It switches to Active Mode. It acts like a mini-amplifier, catching the signal, boosting its volume, and then bouncing it.
• The Magic: It decides in real-time which mode to use based on how much energy it has harvested. It's like a smart home thermostat that turns on the heater only when the house is cold and the power bill allows it.

3. The Brain: The "AI Traffic Cop"

To decide exactly how to angle the mirror and how hard to shout, the system uses Deep Reinforcement Learning (DRL).

• The Analogy: Imagine a new traffic cop who has never seen this intersection before. Every time a car gets through successfully, the cop gets a "gold star" (Reward). If a car crashes or hits a red light, the cop gets a "frown" (Penalty).
• The Method (SAC): The authors used a specific type of AI called Soft Actor-Critic (SAC). Think of this as a cop who is curious. Instead of just memorizing one route, the cop tries many different angles and strategies, learning which ones give the most gold stars over time. This helps the system adapt to changing weather, moving cars, and blocked roads better than older, rigid methods.

4. The Threat: The "Fake Gold Star" Attack

Here is the tricky part. What if a hacker stands on the corner and tells the traffic cop, "Great job!" when the cop actually made a terrible mistake?

• Reward Poisoning: This is an attack where a hacker manipulates the "gold stars." They might flip a "Good Job" into a "Bad Job" (Inversion) or make a "Great Job" seem like a "So-So Job" (Scaling).
• The Result: The AI cop gets confused. It starts thinking that bad driving is good, and it learns the wrong rules. The whole network slows down or crashes.

5. The Defense: The "Skeptic Filter"

The paper proposes a lightweight defense to stop these fake gold stars.

• The Analogy: Imagine the traffic cop has a statistical filter. Before accepting a gold star, the cop checks: "Wait a minute. I usually get about 10 stars an hour. This guy just gave me 1,000 stars in one second, or he gave me a negative star. That's suspicious!"
• The Mechanism: The system looks at the history of rewards. If a new reward is way too high or way too low compared to the average, the system clips it (cuts it off) or discards it entirely. It only trusts the "normal" feedback. This keeps the AI on the right track even when a hacker is trying to trick it.

6. The Results: Why It Matters

The authors ran simulations (computer tests) and found:

• Better Speed: The Hybrid Mirror + AI Traffic Cop got more data through the network than using just a passive mirror or a fixed active tower.
• Energy Savings: Because it only amplifies when it has the battery power, it saves a massive amount of energy compared to a system that is always amplifying.
• Security: Even when hackers tried to poison the rewards, the "Skeptic Filter" kept the system running smoothly, maintaining high speeds.

Summary

This paper presents a smart, energy-harvesting mirror that helps wireless signals bypass obstacles. It uses a curious AI to figure out the best way to bounce signals, and it has a built-in lie detector to ignore fake feedback from hackers. It's a step toward faster, greener, and more secure 6G networks for the future.

Technical Summary

1. Problem Statement

The paper addresses critical challenges in Cognitive Radio Networks (CRNs), specifically underlay Multiple-Input Single-Output (MISO) systems where Secondary Users (SUs) share spectrum with Primary Users (PUs). The core problems identified are:

• Unreliable Direct Links: SUs often suffer from signal blockage, necessitating Reconfigurable Intelligent Surfaces (RIS) to enhance coverage.
• Energy Constraints: Traditional active RISs provide signal amplification but consume significant power, which is unsustainable for energy-constrained devices (e.g., IoT). Conversely, passive RISs are energy-efficient but suffer from the "multiplicative fading" effect, leading to performance degradation.
• Static vs. Dynamic: Existing hybrid RIS designs are often "fixed" (some elements active, some passive), lacking the adaptability to real-time energy availability.
• Security Vulnerabilities: Deep Reinforcement Learning (DRL) agents used for optimization are vulnerable to reward poisoning attacks, where adversaries manipulate reward signals to degrade learning and network performance.

2. Methodology

The authors propose a comprehensive framework integrating system modeling, optimization, and security defense:

A. System Model

• Dynamic Hybrid RIS: A novel RIS architecture that dynamically switches between passive (reflection only) and active (reflection + amplification) modes based on real-time Energy Harvesting (EH) availability.
  • Energy Source: The RIS harvests energy from a dedicated power beacon.
  • Mode Switching: If total harvested energy ($E_{total}$) is below a threshold ($\tau$), the RIS operates in passive mode. If $E_{total} \geq \tau$, it switches to active mode with an energy-dependent amplification gain.
• Realistic Channel & Hardware Modeling:
  • Cascaded Fading: Uses a cascaded Rayleigh fading model for links (SU-Tx $\to$ RIS $\to$ SU-Rx) to accurately represent multipath scattering.
  • Non-Ideal Reflection: Incorporates phase-dependent amplitude models for passive elements and thermal noise for active amplifiers.
• Optimization Goal: Maximize the ergodic sum rate of SUs while adhering to PU interference thresholds and energy constraints.

B. Optimization via Deep Reinforcement Learning (DRL)

• Algorithm: The authors employ the Soft Actor-Critic (SAC) algorithm, chosen for its robustness in continuous state and action spaces.
• State Space: Includes transmission power, interference thresholds, channel matrices, and previous actions.
• Action Space: Jointly optimizes the SU transmit beamforming matrix and the RIS phase shift/amplification coefficients.
• Reward Function: Designed to maximize the sum rate while penalizing the agent if it attempts active amplification without sufficient harvested energy.

C. Security Defense Mechanism

• Threat Model: The paper investigates Reward Poisoning Attacks, where an attacker inverts or scales the reward signal when the agent performs well (high Q-values), misleading the agent into learning suboptimal policies.
• Defense Strategy: A lightweight, real-time defense combining:
  1. Reward Clipping: Bounding rewards within a predefined range $[r_{min}, r_{max}]$.
  2. Statistical Filtering: Comparing incoming rewards against a running mean and standard deviation (calculated during a warm-up phase). Rewards deviating significantly (outliers) are discarded.

3. Key Contributions

1. Dynamic Energy-Aware Hybrid RIS: First proposal of a RIS that dynamically toggles between passive and active modes based on harvested energy, bridging the gap between energy efficiency and signal reliability.
2. Joint Optimization Framework: A unified DRL approach (SAC) that jointly optimizes beamforming and RIS coefficients under realistic hardware impairments (non-ideal reflection, amplifier noise) and cascaded channels.
3. Systematic Security Analysis: The first study to systematically analyze reward poisoning attacks in RIS-enhanced CRNs and propose a computationally efficient defense mechanism (clipping + statistical filtering) that requires no access to multiple environments.
4. Comprehensive Evaluation: Extensive simulations comparing the proposed dynamic hybrid RIS against fully passive, fully active, and fixed hybrid RIS architectures, as well as various DRL baselines (DDPG, TD3).

4. Simulation Results

• Performance vs. DRL Baselines: The SAC-based approach consistently outperforms DDPG, TD3, and random policies, demonstrating faster convergence and higher cumulative rewards due to its entropy regularization (efficient exploration).
• Dynamic Hybrid vs. Fixed/Static:
  • The dynamic hybrid RIS achieves a superior trade-off between throughput and energy consumption compared to fully active (high power) and fully passive (low performance) RISs.
  • It significantly outperforms fixed hybrid RIS (where 50% of elements are permanently active) by adaptively utilizing amplification only when energy permits.
• Impact of Parameters:
  • Increasing the number of RIS elements ($R$) and minimum reflection amplitude ($\beta_m$) improves the sum rate.
  • Higher cascade levels ($\kappa$) and more PU receivers degrade performance due to increased fading and stricter interference constraints.
• Energy Efficiency: The hybrid model achieves up to 74.2% energy savings compared to a fully active RIS while maintaining high data rates, depending on the energy threshold $\tau$.
• Defense Effectiveness: The proposed defense mechanism successfully mitigates reward poisoning attacks. Even under severe inversion attacks, the defended agent maintains a stable, high average rate, whereas the undefended agent converges to a suboptimal policy.

5. Significance and Impact

• Practical Deployment: By addressing energy constraints and hardware imperfections, this work moves RIS technology closer to practical implementation in energy-limited 6G and IoT networks.
• Security in AI-Driven Networks: It highlights the critical vulnerability of DRL in physical layer security and provides a low-complexity, deployable solution for maintaining network reliability against adversarial attacks.
• Design Insights: The study offers crucial design guidelines for network operators, such as tuning the energy threshold ($\tau$) to balance performance and energy efficiency based on specific deployment scenarios (e.g., dense urban vs. remote IoT).

In conclusion, this paper presents a robust, secure, and energy-efficient framework for next-generation cognitive radio networks, leveraging adaptive AI and hybrid RIS technologies to overcome spectrum scarcity and hardware limitations.