⚡ electrical engineering

Differential Privacy in Feature Reconstruction Aided Federated Learning for Agent's Semantic Communication Model Update

This paper proposes a differentially private federated learning framework with semantic feature reconstruction (FedSFR) that enables efficient and stable training of image transmission models under heterogeneous wireless conditions while offering stronger privacy guarantees than gradient-based methods through low-dimensional feature transmission and a model selection mechanism.

Original authors: Yoon Huh, Bumjun Kim, Wan Choi

Published 2026-08-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yoon Huh, Bumjun Kim, Wan Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the next generation of wireless networks, engineers are rethinking how data travels. Instead of sending every single bit of a digital file, the goal is to transmit only the meaning behind it, a concept known as semantic communication. Imagine trying to describe a complex image to a friend over a poor phone connection; you would not list every pixel, but rather the essential shapes and colors that define the picture. This approach allows for faster, more efficient transmission, especially when the connection is weak. However, a major hurdle remains: how to teach these systems to understand different types of images without forcing users to share their private photos with a central server. This is where a method called federated learning comes in. It allows many devices to learn together by sharing only the lessons they have learned, not the raw data itself. Yet, even these shared lessons can sometimes reveal too much about the original private images, creating a need for stronger privacy protections that do not sacrifice the quality of the final result.

Researchers at Seoul National University have developed a new framework that solves this problem by changing how the learning happens. They created a system where devices with weak internet connections do not send their learning updates as complex mathematical adjustments to a massive model. Instead, they send a compressed summary of what they have learned, called a semantic feature vector. Think of this as sending a concise sketch of the lesson rather than the full, detailed textbook. The central server then uses these sketches to rebuild a better understanding of the global picture. The team discovered that by sending these sketches, which are generated by a smaller part of the learning system, they could mathematically prove that the privacy of the user is significantly stronger than if they had sent the full, complex updates. This is because the sketches reveal less about the specific details of the private data than the full model updates would.

To make this work in the real world, the researchers added a layer of noise to the data before it is sent, a standard technique for hiding private information. However, adding noise can sometimes make the learning process messy and less accurate. To fix this, they introduced a smart selection process at the central server. After receiving the noisy updates, the server compares the image reconstruction performance of the intermediate global model (derived from gradient-only updates) and the feature-reconstruction-aided global model. It then selectively retains the update strategy that achieves better task performance. This ensures that the system only keeps the updates that actually help, discarding those that are too distorted by the privacy noise.

The researchers tested this approach using two different sets of images, one with small, colorful pictures and another with high-resolution portraits of faces. They simulated a wireless environment where some devices had strong connections and others had very weak ones. The results showed that their new method, which combines the sketch-based transmission with the smart selection process, consistently produced clearer images and trained more stably than traditional methods. Even when the privacy noise was heavy, their system maintained a high level of performance. Crucially, the study proved that by relying on these semantic sketches, the system offers a strictly stronger guarantee of privacy than sending full model updates, all while using the same amount of data bandwidth. This work demonstrates that it is possible to build wireless networks that are not only efficient and robust but also deeply respectful of user privacy, allowing machines to learn from the world without ever needing to see the private details that make it unique.

Technical Summary: Differentially Private Feature Reconstruction-Aided Federated Learning for Semantic Communication

Problem Statement

In sixth-generation (6G) wireless systems, semantic communication aims to transmit task-relevant information rather than raw bit streams. Deep learning-based Joint Source-Channel Coding (JSCC) frameworks are widely used for robust image transmission. To adapt these systems to dynamic data distributions while preserving privacy, Federated Learning (FL) is employed to update semantic communication modules across distributed agents.

However, existing FL approaches face two primary challenges in this context:

Communication Efficiency: In heterogeneous wireless environments, clients with poor uplink capacity struggle to transmit high-dimensional model updates (gradients), leading to instability or exclusion from training.
Privacy Leakage: While FL avoids sharing raw data, model updates or learned representations can still leak sensitive client information. Previous works have shown that even feature vectors generated by encoders trained on private data may implicitly reflect client-specific information.

The specific problem addressed is how to design a privacy-preserving FL framework for semantic communication that maintains training stability under heterogeneous channel conditions while providing rigorous Differential Privacy (DP) guarantees. A key question investigated is how the choice between gradient-based transmission and feature-based transmission affects the achievable DP guarantees under identical communication budgets.

Methodology

The paper proposes a Differentially Private FedSFR framework, built upon a previous algorithm called FedSFR (Federated Learning with Semantic Feature Reconstruction). The system operates under an honest-but-curious (HBC) parameter server (PS) assumption.

1. FedSFR Framework

The framework partitions participating clients into two sets based on uplink channel conditions:

Set $\mathcal{A}_m$ (Reliable Channels): Clients transmit sparse local model updates (gradients) using top- $S$ sparsification with error feedback.
Set $\mathcal{A}_o$ (Unreliable Channels): Clients transmit low-dimensional semantic feature vectors extracted from their locally updated JSCC encoders using a shared public dataset.

The PS aggregates the sparse gradients from $\mathcal{A}_m$ to form an intermediate global model. It then refines this model using Feature Reconstruction (FR), where the server uses the received feature vectors from $\mathcal{A}_o$ to implicitly integrate their local knowledge without requiring explicit gradient transmission.

2. Privacy Mechanism: Oneshot Laplace (OL)

To protect client privacy, the paper integrates the Oneshot Laplace (OL) mechanism, which combines the Oneshot mechanism (for private index selection) and the Laplace mechanism (for value perturbation). Two transmission strategies are privatized:

Gradient-Based Transmission (Option i): The OL mechanism is applied to the entire JSCC model (encoder and decoder). The server receives perturbed sparse gradients and indices.
Feature-Based Transmission (Option ii): The OL mechanism is applied only to the JSCC encoder parameters prior to feature generation. The client generates features using the privatized encoder and transmits the feature vectors. The server then reconstructs the image.

3. Model Selection (MS) Algorithm

To mitigate performance degradation caused by DP noise (which corrupts the feature generation process), a Model Selection (MS) algorithm is introduced at the PS.

The PS compares the image reconstruction performance (Mean Squared Error on a shared public dataset) of two candidates:
1. The intermediate global model ( $w^{(t+1/2)}$ ) derived solely from gradient aggregation.
2. The FR-aided global model ( $w^{(t+1)}$ ) derived after feature reconstruction.
The model yielding the lower reconstruction error is retained as the updated global model. This adaptive selection ensures that noisy feature reconstructions do not degrade the global training trajectory.

Key Contributions

DP-Aided FedSFR Framework: The authors propose a novel FL framework that incorporates the OL mechanism into both gradient-based and feature-based transmission strategies, enabling privacy-preserving training in heterogeneous uplink channels.
Model Selection Algorithm: A new algorithm is proposed at the server to adaptively select between FR-aided and gradient-only updates, improving training stability and optimization trajectories under DP noise.
Theoretical Privacy Analysis: The paper establishes formal DP guarantees for both transmission strategies. It mathematically proves that feature-based transmission achieves strictly stronger DP guarantees than gradient-based transmission under an identical uplink communication budget. This is because feature-based transmission privatizes only the encoder (approx. 50% of parameters), reducing the sensitivity of the query function compared to privatizing the full model.
Experimental Validation: Extensive experiments on CIFAR-10 and CelebA datasets demonstrate that the proposed DP-aided FedSFR outperforms DP-enabled FedAvg in terms of image reconstruction quality (PSNR), convergence behavior, and robustness across various privacy and channel conditions.

Results

Convergence and Stability: On the CIFAR-10 dataset, DP-aided FedSFR with the MS algorithm (FedSFR+OL+MS) achieves faster convergence and more stable training compared to DP-enabled FedAvg, particularly in heterogeneous wireless settings where FedAvg often fails to train properly.
Privacy-Performance Tradeoff: The study shows that FedSFR+OL+MS (privacy) can match the privacy level of FedAvg while using reduced noise scales, or provide stronger privacy (lower $\epsilon$ ) with the same noise scales, due to the reduced parameter exposure in feature-based transmission.
Robustness to Noise: Even under severe noise injection (e.g., $\sigma_1 = \sigma_2 = 0.01$ ), the proposed method consistently outperforms baselines. The results indicate that heavy noise during index selection ( $\sigma_1$ ) is more detrimental than magnitude perturbation ( $\sigma_2$ ).
Ablation Studies: Experiments on the CelebA dataset (higher resolution) and varying SNR conditions confirm the framework's consistency. The study also highlights that the threshold $S_{th}$ (determining the split between gradient and feature transmitters) significantly impacts performance; an optimal balance (e.g., $S_{th}/N = 0.15$ ) yields the highest PSNR.

Significance and Claims

The paper claims that the proposed DP-aided FedSFR offers a superior solution for updating semantic communication modules in heterogeneous wireless systems. Its primary significance lies in demonstrating that feature-based transmission is not only communication-efficient but also inherently more privacy-preserving than traditional gradient-based transmission.

By leveraging the structural distinction of semantic communication (where only the encoder needs privatization for feature generation), the framework achieves strictly stronger differential privacy guarantees without sacrificing communication bandwidth. Furthermore, the introduction of the Model Selection algorithm effectively addresses the common trade-off between privacy and utility, ensuring that the benefits of feature reconstruction are retained even in the presence of privacy-preserving perturbations. The authors assert that this approach provides a robust, privacy-preserving foundation for next-generation semantic communication networks.