🔢 mathematics

Knowledge Distillation Driven Semantic NOMA for Image Transmission with Diffusion Model

This paper proposes KDD-SemNOMA, a novel semantic NOMA framework for multi-user wireless image transmission that integrates a ConvNeXt-based adaptive coding architecture, a two-stage knowledge distillation strategy to mitigate interference without inference overhead, and a diffusion model-based refinement stage to achieve high-fidelity image reconstruction under diverse and challenging channel conditions.

Original authors: Qifei Wang, Zhen Gao, Shuo Sun, Zhijin Qin, Xiaodong Xu, Meixia Tao

Published 2026-01-27

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Qifei Wang, Zhen Gao, Shuo Sun, Zhijin Qin, Xiaodong Xu, Meixia Tao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a high-quality photo to a friend, but the internet connection is terrible. It's like shouting a message across a crowded, noisy room where multiple people are talking at once. In traditional communication, you'd have to break the photo into tiny digital bits (like 1s and 0s) and send them one by one. If the noise gets too loud, the bits get lost, and the photo arrives as a scrambled mess.

This paper proposes a smarter way to do this, specifically for a future "6G" world where many users are trying to send photos to a central tower (the base station) at the same time. The authors call their new system KDD-SemNOMA.

Here is how it works, broken down into three simple stages using everyday analogies:

1. The Smart Shouter (Semantic Communication & Channel Adaptation)

Instead of sending raw bits, this system sends the "meaning" or "essence" of the image (semantic features). Think of it like sending a detailed description of a painting rather than the pixels themselves.

However, sending these descriptions is hard when the "room" is noisy (bad weather or interference). To solve this, the system uses a specialized translator (a neural network based on something called ConvNeXt).

The Analogy: Imagine a translator who doesn't just speak the language but also listens to the background noise. If the room gets windy (Rayleigh fading) or loud (noise), this translator instantly changes how they speak to make sure the message gets through clearly. They adapt their volume and tone dynamically to fight the noise.

2. The Mentor and the Apprentice (Knowledge Distillation)

The biggest problem in this scenario is that everyone is shouting at once. This creates "interference," making it hard for the receiver to tell who said what.

The Problem: Training a system to decode these mixed-up shouts is difficult because the noise from other users confuses the learning process.
The Solution: The authors use a Teacher-Student approach.
- The Teacher: First, they train a "Teacher" model in a perfect, quiet room where everyone speaks one by one (orthogonal transmission). The Teacher learns perfectly how to describe the images without any interference.
- The Student: Then, they train a "Student" model to work in the noisy, crowded room. Instead of learning from scratch, the Student watches the Teacher. The Student tries to mimic the Teacher's internal "thoughts" (features) and predictions.
- The Result: Even though the Student is working in a noisy room, it learns the "tricks" the Teacher used in the quiet room. This allows the Student to filter out the noise and interference much better than if it had learned alone. Crucially, once training is done, the Teacher is thrown away, so the final system is fast and doesn't need extra computing power.

3. The Art Restorer (Diffusion Model Refinement)

Even with the smart translator and the mentor, the photo might still look a bit blurry or have some "static" because the channel was so bad.

The Analogy: Imagine the photo arrives looking like a sketch with some smudges. The system then uses a Generative AI Artist (a Diffusion Model) to fix it.
How it works: This artist doesn't just guess; it knows what a "perfect face" or "perfect landscape" should look like because it has seen millions of high-quality images before. It takes the blurry sketch, adds a little bit of "noise" to it (to reset the smudges), and then slowly "denoises" it back into a sharp, high-definition image.
The Magic: This step doesn't just fix the pixels; it restores the feeling and details of the image (like the texture of skin or the sharpness of an eye) that were lost in the transmission.

Why is this a big deal?

The paper claims that by combining these three steps, their system can send images to multiple users simultaneously over a crowded, noisy channel and get them back looking almost perfect.

Better than the old way: Traditional methods (like sending bits separately) fail completely when the connection is bad (the "cliff effect"). This system degrades gracefully.
Better than other AI methods: It beats other AI methods that try to do the same thing because it uses the "Teacher" to learn faster and the "Artist" to fix the details better.
Efficiency: It achieves high quality without needing a super-computer at the receiving end to do the heavy lifting during the actual call.

In short, the paper presents a system that acts like a smart, adaptive translator who learns from a perfect mentor and then hires a master art restorer to ensure your photos arrive looking crisp, even when the internet connection is terrible.

Technical Summary: Knowledge Distillation Driven Semantic NOMA for Image Transmission with Diffusion Model

Problem Statement

The paper addresses the challenges of multi-user uplink wireless image transmission in 6G networks, specifically within the context of Semantic Non-Orthogonal Multiple Access (NOMA). While semantic communication offers bandwidth efficiency by transmitting semantic features rather than raw bits, multi-user NOMA scenarios introduce significant hurdles:

Dynamic Channel Conditions: Performance degrades under varying conditions such as Additive White Gaussian Noise (AWGN) and Rayleigh fading.
Semantic Interference: In NOMA, signals from multiple users are superimposed, leading to non-orthogonal interference that distorts semantic features and reduces pixel-level fidelity.
Perceptual Quality: Traditional Deep Joint Source-Channel Coding (DeepJSCC) often optimizes for pixel-level distortion (e.g., MSE) but fails to maintain high perceptual quality, especially under poor channel conditions or high compression ratios.
Resource Constraints: Existing multi-user solutions often rely on orthogonal resource allocation, which increases overhead as the number of users grows, or lack robust interference mitigation strategies in non-orthogonal settings.

Methodology

The authors propose KDD-SemNOMA, a two-stage framework designed to enhance both pixel-level accuracy and perceptual quality for multi-user image transmission.

1. Core Architecture: ConvNeXt-based DeepJSCC with Enhanced AF-Module

The foundation is a DeepJSCC network based on the ConvNeXt architecture, modified for semantic transmission.

Enhanced Adaptive Feature (AF) Module: To ensure robustness against dynamic channels, the authors introduce an AF-Module. This module takes channel side information (Signal-to-Noise Ratio $\gamma$ , and Rayleigh fading parameters amplitude $a$ and phase $\phi$ ) as input. It dynamically modulates semantic features using an adaptive mask generated via a fusion network, allowing the system to suppress noise-corrupted components and emphasize robust features.
User Differentiation: At the transmitter, user-specific embeddings are concatenated with input images to enable the model to distinguish between users despite non-orthogonal superposition.

2. Stage One: Knowledge Distillation (KD-SemNOMA)

To mitigate the performance gap caused by non-orthogonal interference, the authors employ a Knowledge Distillation (KD) strategy:

Teacher Model: A model trained on orthogonal transmission (where users do not interfere with each other, only channel noise exists). This model learns "interference-free" semantic representations.
Student Model: The actual SemNOMA model trained on non-orthogonal transmission.
Distillation Mechanism: The student model is guided by the teacher through two specific losses:
- Feature Affinity (FA) Distillation: Aligns the spatial affinity matrices of intermediate feature maps between the teacher and student, ensuring the student learns the structural relationships of interference-free features.
- Cross-Head Prediction (CrossKD) Distillation: Transfers intermediate features from the student's decoder to the teacher's decoder to generate "cross-head" predictions, bridging the gap between supervised and distillation objectives.
Efficiency: During inference, only the lightweight student model is used, incurring no additional computational overhead compared to standard SemNOMA.

3. Stage Two: Diffusion Model-Based Refinement (KDD-SemNOMA)

To further enhance perceptual quality and recover fine details lost during transmission, a second stage utilizes a pre-trained Diffusion Model:

Error Contraction: The initial reconstruction ( $x_{init}$ ) from the KD-SemNOMA stage is treated as a coarse estimate. The framework applies a forward diffusion process to $x_{init}$ , leveraging the mathematical property that the diffusion process contracts the prediction error associated with the initial estimate.
Reverse Diffusion: A reverse diffusion process (using DDIM sampling) iteratively denoises the contracted state, guided by the generative priors of the pre-trained diffusion model. This transforms the initial output into a high-fidelity image ( $x_0$ ) with enhanced texture and semantic coherence.

Key Contributions

Non-Orthogonal Semantic Transmission Framework: The proposal of a ConvNeXt-based DeepJSCC architecture with an enhanced AF-Module that dynamically adapts to both AWGN and Rayleigh fading channels, ensuring robust feature transmission.
Knowledge Distillation for Interference Mitigation: The innovative integration of a teacher-student distillation scheme into the SemNOMA framework. By using an orthogonal teacher to guide the non-orthogonal student via feature affinity and cross-head prediction, the method achieves significant PSNR gains (0.2–0.8 dB) without increasing inference complexity.
Diffusion-Enhanced Image Refinement: The development of a two-stage reconstruction approach (KDD-SemNOMA) that combines the pixel-level accuracy of KD-SemNOMA with the generative priors of diffusion models. This stage specifically targets perceptual quality improvements, effectively recovering details and mitigating artifacts in low-SNR environments.

Experimental Results

Extensive experiments were conducted on CIFAR-10 (low-resolution) and FFHQ-256 (high-resolution face images) datasets under AWGN and Rayleigh fading channels.

Pixel-Level Fidelity (PSNR/SSIM):
- The proposed KD-SemNOMA outperforms baseline DeepJSCC-NOMA (ResNet-based) and traditional BPG+LDPC+QAM+SIC schemes.
- Under Rayleigh fading, the enhanced AF-Module provides PSNR improvements of 1–3 dB over baselines.
- Knowledge distillation yields an additional PSNR gain of approximately 0.2–0.8 dB over the non-distilled SemNOMA.
- The method demonstrates "graceful degradation" at low SNR, avoiding the "cliff effect" seen in traditional separated coding schemes.
Perceptual Quality (LPIPS/FID):
- The full KDD-SemNOMA (including diffusion refinement) significantly outperforms one-stage methods and GAN-based refinement (SemNOMA-CGAN) in terms of LPIPS and FID scores.
- In low-SNR regimes (e.g., 0 dB), KDD-SemNOMA reduces LPIPS and FID values by approximately 23% and 36% compared to GAN-based methods, demonstrating superior ability to recover semantic details and avoid mode collapse.
- Visual comparisons confirm that KDD-SemNOMA preserves facial contours and texture details better than competing methods under severe channel conditions.

Significance and Claims

The paper claims that KDD-SemNOMA represents a significant advancement in multi-user semantic communication by effectively addressing the dual challenges of interference mitigation and perceptual quality enhancement.

Scalability and Efficiency: The framework achieves high performance without increasing inference overhead (due to the distillation strategy) and reduces transmission overhead compared to orthogonal schemes (NOMA vs. OMA).
Robustness: The system is shown to be highly robust across diverse channel conditions (AWGN and Rayleigh) and compression ratios, making it suitable for bandwidth-constrained 6G scenarios.
Perceptual Superiority: By integrating diffusion models, the work moves beyond traditional distortion metrics, delivering high-fidelity image recovery that aligns better with human visual perception, even in extremely poor channel conditions.

The authors conclude that their approach effectively bridges the gap between pixel-level accuracy and perceptual quality, offering a viable solution for scalable, high-quality multi-user image transmission in future 6G networks.