🔢 mathematics

Bayesian-Guided Cooperative RL Beamforming for Wireless Adversarial User Detection

This paper proposes a Bayesian-guided cooperative reinforcement learning framework for wireless beamforming that effectively balances high data rates and security by outperforming random selection and achieving an optimal trade-off between attacker detection accuracy and computational efficiency, with Q-learning emerging as the most effective method.

Original authors: Parmida Geranmayeh, Onur Günlü

Published 2026-07-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Parmida Geranmayeh, Onur Günlü

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the air around us is filled with invisible rivers of information, carrying your favorite songs, videos, and messages to your phone. For a long time, these rivers just flowed everywhere, like water from a hose sprayed across a whole field. But as more and more people try to drink from the same hose, the water gets muddy, and the flow slows down. To fix this, engineers are learning to turn that hose into a high-powered laser pointer. This is called beamforming: instead of spraying water everywhere, you aim a tight, powerful stream directly at the person you want to talk to. It's like using a flashlight in a dark room to find a friend instead of turning on the ceiling lights for everyone.

However, there's a catch. In the future, the "dark room" might be full of troublemakers. Imagine a mischievous kid in the crowd who starts shouting to drown out your friend's voice or pretending to be your friend to steal your secrets. This is the world of wireless security. We need a way to not only aim our laser perfectly but also spot the troublemakers instantly and ignore them. This is where machine learning comes in. Think of it as a super-smart coach that watches the game, learns from every mistake, and figures out the best moves to win, even when the other team is trying to cheat. The paper you are about to read explores how to teach this coach to be a detective and a strategist all at once.

The Paper: A Smart Coach for Wireless Laser-Pointers

This paper introduces a new, clever system designed to help wireless networks (like the 5G and future 6G networks) stay fast and secure, even when bad actors are trying to mess things up. The authors, Parmida Geranmayeh and Onur Günlü, propose a method that combines three big ideas: beamforming (aiming the signal), reinforcement learning (learning by trial and error), and Bayesian reasoning (updating your guess about who the bad guy is based on new evidence).

The Setup: A Digital City with Laser Beams

The researchers built a digital simulation of a city (based on the real streets of Dortmund, Germany) filled with cell towers (transmitters) and phones (receivers). They used a very detailed map of the city, including buildings and streets, to see how radio waves bounce off walls and get blocked. This is called ray tracing. In their simulation, they had two towers trying to talk to several phones. But, they also added "attackers"—phones that are secretly trying to jam the signal or pretend to be legitimate users.

The goal was simple: get the most data to the good phones while spotting the bad ones. To do this, the towers need to pick the perfect angle to shine their "lasers."

The Problem: Too Many Choices

Imagine you have two flashlights and 25 different angles you can point them at. If you have just a few phones, you could try every single combination of angles to see which one works best. This is called an exhaustive search. The researchers did this first to find the "perfect" answer for a small group of phones. They found that with 4 phones, the best setup could send about 2.41 Gbps of data.

But here's the problem: if you add more phones, the number of possible angle combinations explodes. With 8 phones, there are 25 to the power of 10 possible combinations. That is a number so huge that even the fastest supercomputer would take forever to check them all. This is why the researchers needed a smarter way to find the answer without checking every single possibility.

The Solution: The Smart Coach (Reinforcement Learning)

Instead of checking every angle, the authors taught the network to learn like a video game character. They used two types of "coaches" (algorithms):

Q-learning: A coach that is very confident. It tries an action, sees what happens, and assumes it will always pick the best move next time. It's like a player who thinks, "If I jump here and get a coin, I'll always jump here."
SARSA: A more cautious coach. It learns based on what it actually does next, even if it makes a mistake. It's like a player who thinks, "I jumped here, but then I slipped, so maybe I shouldn't jump here next time."

Both coaches were given a special tool: a Bayesian belief system. This is like a detective's notebook. Every time the network sees something strange (like a sudden drop in speed), the notebook updates the "suspicion score" for every phone. If a phone's score gets high enough, the system assumes it's an attacker.

The Results: Who Wins the Game?

The researchers ran their simulation many times (20 different "Monte Carlo" runs) with different numbers of phones (4, 6, and 8) and attackers (1 or 2). Here is what they found:

The Random Player Loses: If you just pick angles and guess who the bad guys are at random, the network is slow and you rarely catch the attackers (only about 20% to 40% accuracy).
The Smart Coaches Win: Both Q-learning and SARSA were much better. They learned to aim the beams correctly and spot the attackers.
- Q-learning was the champion. It achieved the highest data speeds (reaching up to 3.42 × 10⁹ bps with 8 phones) and caught the most attackers (about 84% to 95% accuracy, depending on the group size).
- SARSA was also good but slightly slower and less accurate than Q-learning.
- The "Random" method was the worst, catching attackers only about 20% of the time in the largest group.

The paper also looked at how long it took to run. The smart coaches took a bit longer to compute (about 50 to 92 seconds depending on the group size) compared to the random method (which took only 1 to 2 seconds). However, the authors argue that waiting a few extra seconds is worth it to get much faster internet and better security.

The Verdict

The paper suggests that by combining a "detective" (Bayesian reasoning) with a "learner" (Reinforcement Learning), wireless networks can become much smarter. They can automatically figure out the best way to send signals and identify troublemakers without needing a human to check every single possibility.

While the results are very promising, the authors are careful to note that these findings come from computer simulations, not real-world tests on actual phones in a real city. They suggest that in the future, this system could be expanded to also control how much power the towers use, making the whole network even more efficient. For now, though, the study shows that a little bit of artificial intelligence can go a long way in keeping our digital connections fast and safe.

Technical Summary: Bayesian-Guided Cooperative RL Beamforming for Wireless Adversarial User Detection

Problem Statement
In next-generation wireless networks, particularly those operating in millimeter-wave (mmWave) bands, there is a critical need to simultaneously achieve high data rates, spectral efficiency, and robust security. Traditional beamforming optimization often relies on exhaustive search methods, which become computationally prohibitive as network size and complexity increase. Furthermore, in dynamic and adversarial environments, distinguishing between legitimate users and malicious entities (attackers) is challenging due to channel variations, fading, and intentional interference. Existing literature has largely treated beamforming optimization, channel modeling, and security mechanisms as independent problems. This paper addresses the gap by proposing an integrated framework that jointly optimizes beamforming and attacker detection in a multi-user, multi-antenna mmWave network under adversarial conditions.

Methodology
The authors propose a unified framework combining 3GPP-compliant channel modeling, game theory, Bayesian inference, and Reinforcement Learning (RL).

System Model: The study utilizes a 5G NR system model operating at 28 GHz with a 400 MHz bandwidth. The network consists of two transmitters (TXs) and a variable number of receivers (RXs, $N_{RX} \in \{4, 6, 8\}$ ). The environment is modeled using the 3GPP TR 38.901 Urban Micro-cellular (UMi) scenario, incorporating a deterministic Ray-Tracing (Shooting and Bouncing Rays) algorithm to capture realistic propagation paths, reflections, and diffractions. Spatially correlated shadow fading is also included.
User Association: A non-cooperative game-theoretic mechanism is employed for RX-TX association. Each RX updates its association based on a utility function that balances Signal-to-Interference-plus-Noise Ratio (SINR) and load balancing, using a Softmax-based probabilistic best-response rule.
Bayesian Belief Updating: To detect malicious users without prior knowledge of their identities, the system maintains a Bayesian belief vector. This vector represents the posterior probability that each RX is an attacker. Beliefs are updated iteratively based on observed discrepancies between predicted and actual network performance (capacity).
Reinforcement Learning Framework: The core decision-making engine uses RL to select cooperative beamforming angles for the two TXs.
- State Space: Defined by the most probable attacker index and a Bayesian confidence level.
- Action Space: The joint selection of discrete beam steering angles for both TXs (from a codebook of 25 angles).
- Exploration: A Bayesian-guided $\epsilon$ -greedy policy is used, where exploratory actions are biased by the current belief vector to avoid high-risk configurations.
- Algorithms: The paper compares Q-learning (off-policy) and SARSA (on-policy). The reward function is multi-objective, combining efficiency (capacity), penalties for performance drops, and security risks proportional to the probability of malicious nodes.
Evaluation: The framework is evaluated via Monte Carlo simulations against an exhaustive search benchmark (for small networks) and a random baseline.

Key Contributions

Integrated Framework: The paper proposes a novel architecture that unifies ray-tracing-based channel modeling, game-theoretic user association, and Bayesian-guided RL for secure beam management. Unlike previous works that infer a single attacker from aggregate capacity, this work generates link-specific evidence to support multiple attackers and incorporates these beliefs into cooperative beam selection.
Scalability and Cooperation: The study develops a scalable framework evaluated under varying RX densities ( $N_{RX} = 4, 6, 8$ ) and attacker counts ($1$ or $2$). It introduces a cooperative beamforming strategy where two TXs jointly determine beam directions, demonstrating improved coordination over independent operation.
Benchmarking: An exhaustive search is implemented for the baseline scenario to establish the optimal beam steering angles, serving as a reference for evaluating learning-based methods.
Algorithmic Comparison: A comprehensive comparison between Q-learning, SARSA, and random selection is provided, analyzing throughput, detection accuracy, convergence, and computational cost.

Results

Attacker Detection: The Bayesian belief mechanism successfully converges to distinguish malicious nodes from legitimate users. In scenarios with 4, 6, and 8 RXs, the belief values for attackers stabilize at high levels (0.65–0.9), while legitimate nodes drop near zero. Attacker detection accuracy approaches 100% for smaller networks and exceeds 98% for larger, more complex networks after sufficient iterations.
Throughput and Efficiency: Both RL algorithms significantly outperform the random baseline in total channel capacity. Q-learning consistently achieves the highest throughput (e.g., ~3.42 Gbps for $N_{RX}=8$ ) compared to SARSA and the random policy.
Convergence and Stability: Q-learning demonstrates faster convergence to high-capacity regions compared to SARSA. While SARSA is noted for being more conservative and stable in uncertain environments, Q-learning provides a superior trade-off between detection accuracy and computational efficiency in this specific context.
Computational Cost: While RL methods incur higher execution times (approx. 50–92 seconds) compared to the random baseline (1–2 seconds) due to the learning process, they remain feasible. Q-learning generally requires slightly less execution time than SARSA while delivering better performance.

Significance and Claims
The paper claims that the proposed Bayesian-guided cooperative RL framework provides a stable, scalable, and effective solution for joint beamforming and security-aware decision-making in dynamic wireless environments. The authors assert that integrating RL with Bayesian inference allows the system to adaptively learn beam selection policies that maximize network capacity while simultaneously identifying and mitigating adversarial threats. The results indicate that Q-learning offers the most favorable balance among the evaluated methods, achieving high detection accuracy and throughput with manageable computational overhead. The study concludes that this approach is a viable step toward intelligent, security-aware network management in 6G and beyond, though it acknowledges that future work could extend this to include power control and deep RL for continuous action spaces.