Diffusion-Aided Bandwidth-Efficient Semantic Communication with Adaptive Requests

Technical Summary: Diffusion-Aided Bandwidth-Efficient Semantic Communication with Adaptive Requests

Problem Statement
Traditional communication systems prioritize exact bitwise recovery, whereas semantic communication aims to transmit task-relevant meaning. In image transmission using generative receivers (e.g., latent diffusion models), relying solely on text captions often fails to preserve instance-specific visual details, while transmitting dense latent representations incurs significant bandwidth overhead. A critical challenge arises in wireless settings: the receiver lacks access to the ground-truth source image, making it impossible to determine a priori how much visual evidence (latent blocks) is sufficient for semantic fidelity. Fixed-budget transmission strategies are inefficient because they either waste resources on simple images or fail to provide enough evidence for complex ones. Furthermore, classical Hybrid Automatic Repeat Request (HARQ) mechanisms, which rely on bit-level error detection, are inapplicable here as there is no source image at the receiver to verify bit correctness.

Methodology
The paper proposes a receiver-driven, closed-loop semantic communication scheme that dynamically adjusts the transmitted payload based on semantic consistency rather than bit reliability.

1. System Architecture:

   • Transmitter: Encodes an input image $x$ into a latent embedding $\tilde{z}$ using a VAE encoder. The latent space is partitioned into square blocks. The transmitter sends a short text caption (generated via an image-to-text converter) and an initial sparse subset of latent blocks, masking the rest. Both streams are modulated and transmitted over an AWGN channel.
   • Receiver: Upon receiving the noisy caption $\hat{y}$ and sparse latent blocks $\hat{z}$, the receiver reconstructs the image using caption-conditioned latent diffusion inpainting. The diffusion model fills in the missing (masked) regions while keeping the received blocks fixed.

2. Adaptive Retransmission Loop:

   • Semantic Stopping Rule: Since the source image is unavailable, the receiver evaluates the quality of the reconstruction $\hat{x}$ by generating a new caption $\bar{y}$ from it (using a captioner like BLIP-2) and computing a semantic similarity score (ROUGE-L, denoted as $\Phi$) between $\bar{y}$ and the received caption $\hat{y}$.
   • Decision Logic: If the score $r_t = \Phi(\bar{y}_t, \hat{y})$ meets a predefined threshold $\tau$, transmission stops. If not, the receiver requests a small, random subset of the withheld latent blocks.
   • Refinement: The newly received blocks are merged into the latent representation, and the diffusion inpainting process is re-run with updated parameters.
   • Robustness Mechanism: To handle noise, the receiver generates two candidates per round: one with guided diffusion (using a guidance scale $w$) and one with unguided diffusion ($\gamma=0$). It selects the candidate yielding the higher semantic score.

3. Parameters: The system utilizes a tunable threshold $\tau$ to control the trade-off between bandwidth (number of rounds/blocks) and semantic quality. The inpainting strength is heuristically adjusted based on the Signal-to-Noise Ratio (SNR) and the current fraction of withheld blocks.

Key Contributions

• Receiver-Side Semantic Control: The authors propose a novel trigger and early-stopping rule that enables variable-length transmission without requiring the receiver to access the source image. The stopping criterion is defined entirely in the semantic space (caption consistency).
• Incremental Latent Refinement: The work couples incremental latent-block requests with caption-conditioned latent diffusion inpainting, allowing for progressive reconstruction refinement where missing visual evidence is filled in only when necessary.
• Robustness to Noise: The system improves reliability under noisy links by dynamically selecting between guided and unguided reconstructions based on the same semantic-consistency score, rather than relying on fixed generation parameters.

Experimental Results
Experiments were conducted on the Flickr30k dataset over AWGN channels, comparing the proposed scheme against several baselines:

• No-guidance: Latent-only transmission without text conditioning.
• Full-mask: Caption-only transmission (no latent blocks).
• Fixed-budget: One-shot transmission of a number of latent blocks matching the average payload of the proposed scheme.
• Fixed-round: A high-overhead reference that always runs the maximum number of rounds without early stopping.

Findings:

• Semantic Alignment: The proposed adaptive scheme achieved the highest ROUGE-L scores and the lowest semantic failure probability ($P_{fail}$) compared to all baselines. It outperformed the fixed-budget one-shot transmission, demonstrating that adapting the payload to image difficulty improves semantic alignment.
• Efficiency: While the "fixed-round" scheme achieved better pixel-level fidelity (SSIM/LPIPS) due to transmitting more data, the proposed scheme achieved comparable pixel/perceptual quality to the fixed-budget baseline while using fewer latent blocks on average for easier images.
• Trade-off Control: The threshold $\tau$ effectively controls the rate-quality tradeoff; higher $\tau$ values result in more rounds and higher semantic consistency but increased bandwidth usage.
• Failure Rates: The adaptive scheme consistently maintained the lowest failure rate, as it injects additional evidence specifically when the semantic score is insufficient, whereas fixed-round schemes do not guarantee monotonic improvement in semantic scores due to the stochastic nature of diffusion.

Significance and Claims
The paper claims that receiver-side semantic control can significantly improve reliability for generative receivers operating under a fixed average evidence budget. By shifting the feedback mechanism from bit-level error correction to semantic consistency verification, the system achieves a controllable rate-quality tradeoff. The results suggest that this approach outperforms budget-matched one-shot transmission in semantic alignment and failure rates, while typically requiring fewer latent blocks than continuous retransmission schemes. The authors note that future work will address importance-aware block selection, unreliable captions/feedback, and extensions to video and multi-user scenarios.