Resource Allocation for Positive-Rate Covert Communications Using Optimization and Deep Reinforcement Learning

This paper proposes optimization-based and deep reinforcement learning (specifically DDQN) methods to achieve positive-rate keyless covert communications in Rayleigh block-fading channels by solving power and rate allocation problems under both non-causal and causal channel state information scenarios.

The Gist

Imagine you are trying to send a secret message to a friend across a noisy room, but there's a strict security guard (the "Warden") watching everyone. Your goal isn't just to keep the content of the message secret (like using a code); your goal is to make it so the guard doesn't even realize a conversation is happening at all. This is called Covert Communication.

Usually, there's a catch: if you try to hide your conversation too well, you have to whisper so quietly that the message becomes useless (zero rate). This paper solves that problem by figuring out how to whisper just enough to be heard by your friend, but not enough to trigger the guard's alarm, even when the room is full of static (fading channels).

Here is a breakdown of the paper's ideas using simple analogies:

1. The Setup: The Whispering Game

• The Players: You (Transmitter), your Friend (Receiver), and the Guard (Warden).
• The Environment: The room has "bad acoustics" (Rayleigh block-fading). Sometimes the sound carries well; sometimes it gets muffled.
• The Goal: Send a message with a positive rate (meaning, actually say something useful) without the guard noticing.
• The Secret: You and your friend know exactly how the acoustics are right now (Channel State Information or CSI). The guard only knows the average noise level, not the specific moment-to-moment changes.

2. The Two Big Challenges

The researchers tackled two main questions:

1. Power Allocation: "I have a limited battery (power budget). How should I split my energy across different time slots to whisper the most words possible without getting caught?"
2. Rate Allocation: "I need to whisper exactly 10 words. What is the minimum energy I need to use to do this safely?"

3. Scenario A: The "All-Knowing" Strategy (Non-Causal CSI)

Imagine you are given a script before the game starts. You know exactly how the acoustics will be for the next 10 minutes.

• The Problem: You have to decide right now how much to whisper in minute 1, minute 2, etc., to maximize your total words.
• The Solution: The authors created a clever three-step recipe:
  1. Check the Rules: First, check if it's even possible to whisper without getting caught. (If the guard's hearing is always better than your friend's, you can't win).
  2. The Easy Guess: Pretend the hardest rule doesn't exist and solve the easy math problem.
  3. The Fine-Tune: If your "easy guess" breaks the hard rule, use a "penalty" method. Imagine you are walking a tightrope; if you lean too far one way, a spring pushes you back. The computer keeps adjusting your whisper volume until you are perfectly balanced: loud enough for your friend, quiet enough for the guard.

4. Scenario B: The "Live" Strategy (Causal CSI)

Now, imagine you don't have the script. You only know the acoustics right now. You have to decide how much to whisper in the current minute, then wait to see what happens next.

• The Problem: This is like driving a car in fog. You can't see the whole road, only the patch right in front of you. You need to make decisions on the fly.
• The Solution (Power Allocation): They turned this into a video game for a computer.
  • They used a technique called Deep Reinforcement Learning (DDQN). Think of this as training a video game AI.
  • The AI plays the game thousands of times. Every time it whispers too loud and gets "caught" (simulated), it loses points. Every time it whispers successfully, it gains points.
  • Eventually, the AI learns the perfect strategy: "When the room is quiet, whisper a bit louder. When the room is noisy, whisper softer."
• The Solution (Rate Allocation): This was tricky because the math didn't fit the video game format perfectly. So, they used a hack: They took the AI trained to whisper loudly (Power Allocation) and tricked it into whispering specific words (Rate Allocation). It's like using a weightlifter's training program to teach someone how to run a marathon. It's not a perfect fit, but it works surprisingly well.

5. The Results: Who Won?

The researchers ran thousands of simulations to test their methods against older, simpler strategies.

• The "Trivial" Strategy: Just whispering the same amount all the time. (Like shouting "Hello" at a constant volume).
• The "Convex" Strategy: A standard math approach that ignores the hardest rules.
• The Authors' Strategy: The smart, adaptive AI and the three-step recipe.

The Verdict:

• The authors' methods always won. They managed to send more words with less energy.
• The advantage was biggest when the Guard had better hearing than the friend (a harder scenario).
• The "AI" (Deep Reinforcement Learning) was particularly good at handling the "live" scenario where you don't know the future.

Summary

This paper is about teaching a spy how to talk to their contact in a crowded, noisy room without the security guard noticing. They figured out two ways to do it:

1. If you know the future: Use a smart, step-by-step math recipe to plan your whispers perfectly.
2. If you only know the present: Train a digital "brain" (AI) to learn the best way to whisper on the fly.

The result is a way to communicate securely and efficiently, even when the odds are stacked against you.

Technical Summary

Here is a detailed technical summary of the paper "Resource Allocation for Positive-Rate Covert Communications Using Optimization and Deep Reinforcement Learning."

1. Problem Statement

The paper addresses the challenge of achieving keyless covert communication with a positive transmission rate over Rayleigh block-fading channels.

• Context: In traditional covert communication over point-to-point channels, the achievable rate scales as $O(\sqrt{n})$ (where $n$ is the block length), meaning the rate per channel use vanishes as $n \to \infty$. To overcome this, the authors leverage recent information-theoretic results showing that positive rates are possible if the transmitter and legitimate receiver possess Channel State Information (CSI) that the warden (adversary) lacks.
• System Model:
  • Transmitter & Receiver: Have access to CSI for both the legitimate channel ($H_\ell$) and the warden's channel ($G_\ell$).
  • Warden: Only knows the statistical distribution of the CSI, not the instantaneous values.
  • Goal: Maximize the sum covert rate under a power constraint OR minimize power consumption under a minimum rate constraint, while ensuring the warden cannot distinguish between communication and silence (covertness constraint).
• Scenarios: The study considers two distinct CSI availability scenarios:
  1. Non-causal CSI: The transmitter knows the channel states for all future blocks ($L$ blocks) at the start.
  2. Causal CSI: The transmitter only knows the channel states up to the current block $\ell$ and must allocate resources sequentially.

2. Methodology

The authors formulate two primary optimization problems based on information-theoretic constraints for Discrete Memoryless Channels (DMCs) adapted to block-fading:

1. Power Allocation: Maximize $\sum \log(1 + h_\ell P_\ell)$ subject to total power, covertness, and a "less noisy" constraint (legitimate channel must be stochastically better than the warden's).
2. Rate Allocation: Minimize $\sum P_\ell$ subject to a minimum total rate, covertness, and the "less noisy" constraint.

Both problems are non-convex due to the "less noisy" constraint ($I(X;Y|S) \geq I(X;Z|S)$).

A. Non-Causal CSI Approach (Optimization-Based)

When CSI is known non-causally, the authors propose a novel three-step method for both power and rate allocation:

1. Feasibility Check: Determine if a positive rate is possible (requires at least one block where $h_\ell \geq g_\ell$). If not, the problem is infeasible.
2. Convex Relaxation: Temporarily drop the non-convex "less noisy" constraint to solve a convex optimization problem (using Lagrangian multipliers and bisection search).
   • If the convex solution satisfies the dropped constraint, it is the global optimum.
   • If not, proceed to Step 3.
3. Penalized Optimization:
   • Power Allocation: Uses Projected Gradient Ascent (PGA). The non-convex constraint is converted into a penalty term added to the objective function. The algorithm starts from the convex solution and iteratively adjusts power to satisfy the constraint.
   • Rate Allocation: Uses Projected Gradient Descent (PGD). Similar to power allocation, it penalizes the violation of the "less noisy" constraint and iteratively minimizes power consumption.

B. Causal CSI Approach (Deep Reinforcement Learning)

When CSI is known causally, sequential decision-making is required.

1. Power Allocation (MDP Formulation):
   • The problem is formulated as a Markov Decision Process (MDP).
   • State: Remaining power, remaining covertness margin, cumulative "less noisy" metric, and current channel gains.
   • Action: Power allocation for the current block.
   • Reward: Instantaneous covert rate.
   • Algorithm: Solved using Double Deep Q-Network (DDQN). A primary Q-network learns the optimal policy, while a target network stabilizes training. The agent learns to balance immediate rewards with future constraints.
2. Rate Allocation (Approximation):
   • The rate allocation problem is non-Markovian because the total rate constraint links future decisions to the current state in a way that standard MDPs cannot easily handle.
   • Solution: The authors approximate the rate problem by converting the remaining rate requirement into an equivalent remaining power requirement using Jensen's inequality and expected channel values. They then utilize the DDQN trained for power allocation to solve the rate allocation problem sequentially.

3. Key Contributions

• First Analysis of Keyless Positive-Rate Covert Communication over Fading Channels: Unlike prior works relying on jammers or relays, this paper focuses on optimizing resource allocation using only CSI knowledge.
• Novel Three-Step Optimization Algorithms: Developed specific algorithms (PGA for power, PGD for rate) to solve non-convex optimization problems under non-causal CSI, effectively handling the "less noisy" constraint.
• DDQN for Causal Resource Allocation: Successfully formulated the causal power allocation as an MDP and solved it using DDQN.
• Approximate Solution for Causal Rate Allocation: Proposed a method to transform the non-Markovian rate allocation problem into a solvable form using the power allocation DDQN network.
• Comprehensive Performance Analysis: Provided extensive simulations comparing the proposed methods against baseline schemes (trivial allocation, convex relaxation, and average power splitting).

4. Simulation Results

The simulations (using $L=10$ blocks, Rayleigh fading) demonstrate:

• Non-Causal Power Allocation: The proposed PGA method consistently outperforms "convex" and "trivial" baselines, achieving higher sum covert rates, especially when the warden's channel is strong.
• Non-Causal Rate Allocation: The proposed PGD method achieves significantly higher feasibility probabilities (ability to meet the rate target) and lower power consumption compared to baselines.
• Causal Power Allocation: The DDQN approach outperforms causal baselines ("average" and "trivial"). While there is a performance gap compared to the non-causal case (due to lack of future knowledge), the gain over causal baselines is substantial.
• Causal Rate Allocation: The DDQN-based approximation yields higher feasibility and lower power consumption than causal baselines.
• Impact of Covertness ($\delta$): Stricter covertness constraints (smaller $\delta$) naturally reduce achievable rates and increase power consumption, but the proposed algorithms remain robust.

5. Significance

This paper bridges the gap between information-theoretic limits of covert communication and practical resource allocation in dynamic fading environments.

• Theoretical Advancement: It extends the concept of positive-rate covert communication from static or specific setups to general block-fading channels without external helpers (jammers).
• Practical Application: By utilizing Deep Reinforcement Learning, the paper provides a viable solution for real-time, sequential resource allocation where future channel states are unknown, a critical requirement for 6G and next-generation wireless security.
• Efficiency: The proposed methods offer a computationally efficient way to achieve secure, undetectable communication with positive throughput, addressing a fundamental limitation in physical layer security.