⚡ electrical engineering

End-to-End NOMA with Perfect and Quantized CSI Over Rayleigh Fading Channels

This paper proposes an end-to-end autoencoder framework for downlink NOMA over Rayleigh fading channels that learns interference-aware super-constellations and demonstrates superior bit error rate performance compared to existing schemes, particularly when utilizing Lloyd-Max quantized channel state information.

Original authors: Selma Benouadah, Mojtaba Vaezi, Ruizhan Shen, Hamid Jafarkhani

Published 2026-02-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Selma Benouadah, Mojtaba Vaezi, Ruizhan Shen, Hamid Jafarkhani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy radio tower (the Base Station) trying to talk to two different people (User 1 and User 2) at the exact same time, on the exact same radio frequency. This is the challenge of NOMA (Non-Orthogonal Multiple Access).

In the old days, to avoid them talking over each other, the tower would give them different time slots or different frequencies. But that's inefficient. NOMA is like a DJ mixing two songs into one track. The tower sends a "super-signal" that is a mix of both messages, but it gives one message more volume (power) than the other.

The receiver with the "louder" signal (User 2) can easily hear their own song and subtract the "quieter" song (User 1) out of the mix to hear their own clearly. The receiver with the "quieter" signal (User 1) has a harder time because they have to listen to the mix while the "louder" song is still playing in the background.

The Problem: The "Foggy" Weather

In the real world, radio signals don't travel through a vacuum; they travel through the air, which is full of "fog" and "wind" (called Rayleigh Fading). Sometimes the signal is strong, sometimes it's weak, and it changes randomly every second.

If you try to use a fixed recipe for mixing these signals (like a standard math formula), the "fog" messes everything up. The signals overlap in confusing ways, and the listeners start hearing static (errors).

The Solution: The "Smart DJ" (The Autoencoder)

This paper introduces a Smart DJ (an Artificial Intelligence called an Autoencoder) that learns how to mix these signals perfectly, even in the fog.

Instead of using a fixed recipe, this AI learns by trial and error. It acts like a musician who can instantly change the volume, pitch, and shape of the two songs depending on the weather outside.

The Magic: If the wind is blowing hard against User 1, the AI reshapes the signal so User 1 can still hear clearly. If User 2 is having a great day, the AI tweaks the mix to make sure User 2 gets the best possible quality.
The Result: The AI creates a custom "Super-Constellation" (a fancy map of how the signals look) that is perfectly adapted to the current weather, rather than using a one-size-fits-all map.

The Real-World Hurdle: "Whispering" the Weather

To do this magic, the Smart DJ needs to know the weather right now. In the real world, the radio tower can't see the weather perfectly; it has to ask the users to whisper back what the weather is like. But the "whisper" (feedback) is limited by bandwidth, so the users have to give a rough estimate.

The paper tests two ways of whispering:

Uniform Quantization: Like using a ruler with evenly spaced marks. It's simple, but it might miss the details where the weather changes the most.
Lloyd–Max Quantization: Like using a ruler that has tiny, detailed marks where the weather is usually tricky, and big, spaced-out marks where the weather is usually calm.

The Finding: The "smart ruler" (Lloyd–Max) works much better. It gives the AI just enough detail to do its job without needing a perfect, high-definition weather report.

The "High Score" Problem (The BER Floor)

When the AI was first trained, it got really good at low-to-medium signal strength, but when the signal got very strong, it hit a "glass ceiling" (called a BER floor). It stopped improving.

Why? The AI was trained in a specific "gym" (a specific signal strength level). It got too used to that specific environment.
The Fix: The researchers changed the training regimen. Instead of training in just one gym, they made the AI train in a variety of gyms (different signal strengths) all at once. This made the AI more flexible and robust, allowing it to break through the glass ceiling and perform perfectly even when the signal is super strong.

The Big Picture

This paper is a breakthrough because:

It's the first to teach an AI to handle the "fog" (fading) perfectly. Previous AI models only worked in perfect, clear conditions.
It works with imperfect information. It proves you don't need perfect weather reports to get great results; a smart, rough estimate is enough.
It beats the old math. The AI learns new, weird, and wonderful ways to mix signals that human mathematicians never thought of, resulting in fewer errors and clearer calls.

In short: The authors built a self-learning radio system that adapts to the weather in real-time, learns from imperfect reports, and outperforms all the traditional, rigid methods of sending data. It's like upgrading from a fixed-volume radio to a smart assistant that knows exactly how to shout or whisper so you can hear every word, no matter how windy it gets.

1. Problem Statement

The paper addresses the limitations of current Non-Orthogonal Multiple Access (NOMA) systems, specifically in downlink scenarios over Rayleigh fading channels.

The Challenge: Traditional NOMA relies on fixed constellations (e.g., QPSK) and Successive Interference Cancellation (SIC). Under Rayleigh fading, random channel variations cause inter-user interference and overlapping super-constellations, leading to severe Bit Error Rate (BER) degradation.
The Gap: Existing deep learning-based NOMA solutions (Autoencoders) have primarily been tested on Additive White Gaussian Noise (AWGN) channels with perfect Channel State Information (CSI). They fail to account for the stochastic nature of fading or the practical constraints of limited feedback (quantized CSI) at the transmitter.
Objective: To develop an end-to-end learning framework that adapts to fading channels and operates effectively with both perfect and quantized CSI, optimizing for BER performance without relying on fixed constellations or SIC.

2. Methodology

The authors propose a Deep Learning-based Autoencoder (AE) framework that jointly optimizes the transmitter (encoder) and receivers (decoders).

A. System Model

Scenario: A single Base Station (BS) serving two users (UE1 and UE2) via power-domain NOMA.
Channel: Independent Rayleigh fading channels ( $h_k$ ) with Gaussian noise.
Transmission: The BS superimposes symbols $x_1$ and $x_2$ with power allocation factors $\alpha$ and $(1-\alpha)$ .
CSI Handling:
- Perfect CSI: The encoder has full knowledge of instantaneous channel gains ( $h_1, h_2$ ).
- Quantized CSI: To emulate practical constraints, channel gains are quantized at the receiver and fed back to the transmitter. The paper compares Uniform Quantization and Lloyd–Max Quantization.

B. Network Architecture

Encoder (Transmitter): Takes binary input bits ( $s_1, s_2$ $s_{1}, s_{2}$ ) and channel state information ( $h_1, h_2$ $h_{1}, h_{2}$ or quantized $\hat{h}_1, \hat{h}_2$ $\hat{h}_{1}, \hat{h}_{2}$ ) as inputs. It outputs a complex-valued superimposed symbol.
- Structure: 8 hidden layers (fully connected with $tanh$ activation) to capture the non-linear mapping required by varying fading conditions.
- Adaptation: Unlike AWGN systems where the mapping is fixed, the encoder learns to shape the "super-constellation" dynamically based on the current channel realization.
Decoders (Receivers): Two separate decoders for UE1 and UE2.
- Structure: Similar deep architecture (8 hidden layers) but taking only the received noisy signal and their own channel gain as input.
- Output: Bit probabilities via sigmoid activation.
Loss Function: A weighted sum of binary cross-entropy losses for both users:
$L(\gamma) = w \max(L_1, L_2) + \min(L_1, L_2)$
Setting $w > 1$ enforces fairness by prioritizing the user with the higher error rate.

C. Training Strategy & BER Floor Mitigation

Standard Training: Trained at a fixed SNR (10 dB).
The Problem: Fixed SNR training leads to a "BER floor" at high SNRs (performance saturates) because the model is not optimized for high-SNR conditions.
Proposed Solution: A novel multi-SNR training method. The batch loss is calculated as a weighted sum of losses computed at multiple representative SNR values (e.g., 1, 5, 10, 15, 20 dB). This eliminates the BER floor without degrading low-SNR performance.

3. Key Contributions

First AE-NOMA for Rayleigh Fading: Extends end-to-end learning from AWGN to Rayleigh fading, enabling the encoder to adaptively shape super-constellations to mitigate interference under random channel conditions.
Quantized CSI Integration: Introduces the first AE-NOMA framework that incorporates limited feedback via Uniform and Lloyd–Max quantization, analyzing their impact on system performance.
Novel Training Algorithm: Proposes a multi-SNR loss function to mitigate the high-SNR BER floor, a common issue in fixed-SNR training.
Super-Constellation Learning: Demonstrates that the AE learns non-conventional, interference-aware mappings that outperform fixed QPSK/QAM schemes and adaptive power allocation baselines.

4. Key Results

Performance vs. Theory: The proposed AE significantly outperforms theoretical BER bounds for standard QPSK-QPSK NOMA and adaptive benchmarks (PAR and PANOMA) across the entire SNR range.
- Example: At a BER of $10^{-2}$ , the AE achieves the target at ~12.5 dB, whereas the theoretical NOMA baseline requires ~17.5 dB.
Quantization Impact:
- Lloyd–Max vs. Uniform: Lloyd–Max quantization consistently yields lower BER than uniform quantization because it minimizes Mean Squared Quantization Error (MSQE) by allocating finer resolution to high-probability channel regions.
- Resolution: Performance improves with more quantization levels. At $\geq 4$ bits, the gap between Lloyd–Max and Uniform narrows, but Lloyd–Max remains superior.
Scalability: The framework adapts seamlessly to different bit lengths (e.g., 2 bits vs. 3 bits per user) without structural changes, only requiring decoder output adjustments.
BER Floor Mitigation: The multi-SNR training strategy successfully removes the BER floor observed in high-SNR regions while maintaining robustness at low SNRs.

5. Significance

This work represents a critical step toward the practical deployment of NOMA in real-world fading environments.

Robustness: It proves that deep learning can learn signaling strategies that are inherently robust to channel randomness, outperforming hand-crafted analytical solutions.
Practicality: By addressing quantized CSI, the paper moves beyond idealized assumptions, showing that AE-NOMA remains effective even with limited feedback bandwidth.
Future Direction: The results suggest that jointly optimizing the quantizer and the AE (rather than using fixed quantizers) could further close the performance gap to the perfect CSI case, paving the way for next-generation, AI-native wireless communication systems.