FGGM: Formal Grey-box Gradient Method for Attacking DRL-based MU-MIMO Scheduler

1. Problem Statement

The paper addresses the security vulnerabilities inherent in Deep Reinforcement Learning (DRL) based schedulers for Multi-User Multiple-Input Multiple-Output (MU-MIMO) systems in 5G networks.

• Context: MU-MIMO schedulers must select a subset of users for data transmission in real-time to maximize spectral efficiency and fairness. This is an NP-hard combinatorial optimization problem, often solved using DRL agents (e.g., Soft Actor-Critic) that infer near-optimal solutions.
• The Vulnerability: These DRL agents rely on Channel State Information (CSI) reported by users as input observations. If adversarial users manipulate their CSI, they can degrade the scheduler's decision-making, leading to reduced throughput for legitimate (victim) users.
• The Gap in Existing Research: Most prior adversarial attacks on MU-MIMO assume the attacker knows the exact CSI values of victim users (White-box assumption). However, in uplink LTE/5G systems, the Base Station (BS) estimates CSI directly, and physical layer encryption often prevents attackers from intercepting exact victim values. Existing methods fail in this "Grey-box" scenario where attackers only know the DRL policy but must estimate victim observation ranges.

2. Methodology: Formal Grey-box Gradient Method (FGGM)

The authors propose FGGM, a novel attack strategy that generates adversarial CSI without knowing the exact victim observations. It combines DRL policy knowledge with formal verification techniques.

A. Threat Model (Grey-box)

• Attacker Capabilities: The adversary knows the trained DRL policy (White-box access to the model) but does not know the exact CSI of victim users.
• Attacker Limitations: The adversary only knows the statistical bounds (mean and variance) of the victim observations, derived from the DRL agent's observation normalizer (a standard component used to stabilize training).
• Goal: Find a single set of adversarial CSI values that minimizes the probability of victim selection across the entire range of possible victim observations.

B. Core Technical Approach

1. Observation Normalization Exploitation: The attacker extracts the mean ($\mu$) and standard deviation ($\sigma$) from the DRL agent's observation normalizer. They define the victim's observation space as an interval: $[\mu - \delta\sigma, \mu + \delta\sigma]$.
2. Polytope Abstraction (Formal Verification):
   • Instead of sampling specific victim CSIs, FGGM uses polytope abstract domains (e.g., DeepPoly) to over-approximate the output of the DRL neural network.
   • The method propagates the input intervals (adversarial inputs + victim intervals) through the neural network layers (Linear and ReLU) to compute linear upper and lower bounds for the network's output (the "proto-action" and subsequently the Q-value).
3. Optimization Objective:
   • The attack formulates a non-linear optimization problem: Minimize the upper bound of the Q-value for any action that includes a victim.
   • By minimizing the upper bound of the Q-value for victim-inclusive actions, the attacker ensures that no matter what the actual victim CSI is within the estimated range, the DRL agent will likely assign a low score to selecting them.
4. Gradient-Based Solution: Since the polytope abstraction is differentiable, the problem is solved using Stochastic Gradient Descent (SGD). The attacker computes the optimal adversarial CSI once at the start of the attack and reuses it, making the attack computationally efficient during runtime.

3. Key Contributions

1. Grey-box Threat Model: Introduced a realistic threat model for DRL-based MU-MIMO where attackers know the policy but only have statistical bounds (not exact values) of victim observations.
2. FGGM Algorithm: Proposed a method leveraging polytope abstraction to generate adversarial inputs that are robust against the entire range of possible victim observations, rather than just specific samples.
3. Performance Superiority: Demonstrated that FGGM outperforms both black-box attacks (Random Noise) and sampling-based white-box attacks (Projected Gradient Descent - PGD/SPGD).
4. Significant Impact: Showed that a single attacker can reduce victim throughput by 9.5%, and when 50% of users are attackers, victim throughput can be degraded by up to 70%.

4. Experimental Results

The authors evaluated FGGM on a DRL-based MU-MIMO scheduler (using Soft Actor-Critic and Wolpertinger architecture) with simulated channel models (QuaDRiGa).

• RQ1 (Search Space Sensitivity): FGGM consistently achieved lower victim selection probabilities than SPGD and Noise across various search space sizes ($\delta$). It performed best when the search space was moderate ($\delta_{adv} \approx 2.0$), proving effective even with estimation errors.
• RQ2 (Number of Attackers): As the number of attackers increased, FGGM's effectiveness grew significantly. With 50% of users acting as attackers, FGGM reduced victim transmission rates to roughly half of what was achieved by Noise attacks.
• RQ3 (50% Attack Scenario):
  • In a scenario with 8 users (4 attackers), FGGM reduced the worst-case victim data rate from 88.5 MB/s (clean) to 26.6 MB/s (a 70% reduction).
  • In a scenario with 16 users (8 attackers), FGGM reduced throughput by 46% compared to the clean DRL scheduler, significantly outperforming SPGD.
  • The attack notably degraded the Jain Fairness Index while maintaining proportional fairness metrics, indicating a specific targeting of user selection fairness.

5. Significance and Implications

• Security Warning: The study reveals that DRL-based network schedulers are critically vulnerable to "grey-box" attacks where adversaries do not need perfect information. The reliance on observation normalizers for training stability inadvertently provides attackers with the statistical bounds needed to launch formal verification-based attacks.
• Methodological Shift: It is the first work to apply DNN verification techniques (polytope abstraction) to generate adversarial examples for DRL systems. This bridges the gap between formal verification (proving robustness) and adversarial machine learning (breaking robustness).
• Future Directions: The results highlight the urgent need for adversarial training in DRL schedulers and the development of robust observation mechanisms that do not leak statistical bounds exploitable by attackers. The FGGM framework is also applicable to other DRL-based resource allocation problems, such as knapsack-oriented scheduling.