🤖 machine learning

Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacy

To address the utility degradation caused by enforcing uniform privacy constraints in heterogeneous settings, this paper proposes **GDPFed**, a framework that partitions clients into groups based on their privacy budgets and optimizes per-group sampling ratios and model sparsification to achieve superior model utility under client-level differential privacy.

Original authors: Jiahao Xu, Rui Hu, Olivera Kotevska

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiahao Xu, Rui Hu, Olivera Kotevska

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are part of a massive, global group of people trying to train a super-intelligent AI to recognize different types of fruit. To do this, everyone needs to share what they’ve learned from their own private photos of fruit.

However, there is a problem: Privacy.

The Problem: The "One-Size-Fits-All" Security Guard

In the world of AI training (called Federated Learning), we use a security technique called Differential Privacy (DP). Think of DP as a "blurring" tool. Before you send your data to the central AI, you add a layer of digital "static" or "noise" to it. This noise makes it impossible for anyone to see your exact photo, but the AI can still see the general patterns.

The issue is that everyone has different levels of "shyness":

Person A is very private and wants a thick, heavy blur (high security, but the data becomes very messy).
Person B is more relaxed and is okay with a light, subtle blur (low security, but the data stays very clear).

Currently, most systems use a "Strict Security Guard" approach. The guard looks at the shiest person in the room and says, "Since Person A is so shy, EVERYONE must use the thickest, heaviest blur possible!"

This is a disaster for the AI. Because everyone is using massive amounts of "noise," the data becomes a blurry mess, and the AI ends up being quite stupid.

The Solution: GDPFed (The "VIP Lounge" Approach)

The researchers in this paper created a smarter system called GDPFed. Instead of one strict guard for everyone, they create "Privacy Groups."

Imagine a club with different lounges:

The Vault: For the super-shy people. They get a heavy blur.
The Lounge: For the moderately shy. They get a medium blur.
The Terrace: For the relaxed people. They get a light blur.

By grouping people based on their comfort levels, the system doesn't waste "blurring power" on people who don't need it. This keeps the overall data much clearer for the AI to learn from.

The Upgrade: GDPFed+ (The "Smart Filter" & "Perfect Schedule")

The researchers then took it a step further with GDPFed+. They realized that even with groups, there were two ways to make the AI even smarter:

The Smart Filter (Sparsification):
Imagine you are sending a detailed description of a fruit. Instead of sending a 1,000-page book where 900 pages are just useless filler, you only send the most important 100 pages. In AI terms, this is "sparsification." By only sending the most important "bits" of information, the system avoids adding "noise" to the useless parts, making the important parts much sharper.
The Perfect Schedule (Optimal Sampling):
If the AI is trying to learn, it shouldn't just pick people at random. If you have a group of people who are very shy (and thus provide very noisy, blurry data), you shouldn't let them talk too often, or they will drown out the clear information from the relaxed group. GDPFed+ uses math to calculate the perfect schedule—deciding exactly how often each group should participate so the AI learns as much as possible without getting confused by the noise.

The Result

When the researchers tested this, the results were like moving from a blurry, low-resolution TV to a crisp 4K screen.

By treating people differently based on their privacy needs, using "smart filters" to cut out the junk, and managing the "speaking schedule" perfectly, they created an AI that is both highly private and highly intelligent.

Technical Summary: Optimal Client Sampling in Federated Learning with Client-level Heterogeneous Differential Privacy

1. Problem Statement

The paper addresses a critical gap in Federated Learning (FL): how to provide rigorous privacy guarantees when clients have heterogeneous privacy requirements (Heterogeneous Differential Privacy, or HDP).

While existing research has explored record-level HDP, client-level HDP (protecting the entire dataset of a client) remains underexplored. Traditional approaches like DP-FedAvg fail in heterogeneous settings because they must enforce the strictest privacy budget ( $\epsilon$ ) across all clients to ensure safety. This leads to "privacy budget waste," where clients with relaxed privacy needs are forced to undergo excessive noise injection, severely degrading the global model's utility. Furthermore, existing heuristic methods to mitigate this often assume a trusted server, which is unrealistic in many adversarial environments.

2. Methodology

The authors propose a two-stage framework to optimize the privacy-utility trade-off under an honest-but-curious attack model (where both the server and clients may attempt to infer private data).

A. GDPFed (Group-based Differentially Private Federated Learning):

Grouping Strategy: Instead of a global $\epsilon$ , clients are partitioned into $M$ groups based on their privacy budgets.
Group-level DP: Client-level DP is enforced within each group using the minimum privacy budget of that specific group ( $\epsilon_m$ ). This prevents the strictest client from dictating the noise level for everyone.
Secure Aggregation: To protect against a curious server, noisy updates are aggregated using secure aggregation protocols, ensuring the server only sees the sum of updates.

B. GDPFed+ (The Enhanced Framework):
To further minimize the convergence error identified in their theoretical analysis, the authors introduce two optimizations:

Per-group Model Sparsification: Since DP noise scales with model dimensionality ( $d$ ), the authors apply a Top- $k$ sparsifier to the aggregated updates of each group. This removes less significant parameters, effectively reducing the dimensionality of the noise injected into the model.
Optimal Client Sampling Ratios: The authors derive a mathematical formulation (Problem 1) to optimize the sampling ratio ( $q_m$ ) for each group. This balances the need for frequent participation (to improve utility) against the need to limit participation for high-privacy/high-noise groups (to satisfy DP constraints).

3. Key Contributions

Theoretical Foundation: The paper provides a rigorous privacy analysis (using parallel composition) and a convergence analysis for non-convex objectives under HDP settings.
Novel Algorithm (GDPFed): A group-based approach that mitigates privacy budget waste in client-level HDPFL.
Optimization Framework (GDPFed+): The first work to theoretically optimize client sampling ratios and integrate sparsification to enhance the privacy-utility trade-off in client-level HDP.
Complexity Efficiency: The authors demonstrate that the computational overhead of solving the optimization problem and performing sparsification is negligible, making it suitable for resource-constrained IoT devices.

4. Experimental Results

The authors evaluated their methods on four benchmark datasets: FMNIST, SVHN, CIFAR-10, and Shakespeare.

Superior Utility: GDPFed+ consistently outperformed state-of-the-art baselines (IDP-FedAvg and PFA) and the standard DP-FedAvg. On average, GDPFed+ achieved a 5.75% accuracy improvement over the base GDPFed.
Noise Reduction: Empirical analysis showed that GDPFed reduces the total magnitude of injected noise ( $\Lambda$ ) by nearly half compared to DP-FedAvg.
Robustness: The method proved robust across various client privacy distributions, different clipping thresholds, and varying degrees of data heterogeneity (non-IID data).
Optimal Configuration: The experiments validated that assigning more aggressive sparsification to groups with stricter privacy requirements (lower $\epsilon$ ) yields the best global model performance.

5. Significance

This work is significant because it moves Federated Learning toward a more practical and realistic deployment model. In real-world IoT and healthcare ecosystems, privacy is not a "one-size-fits-all" requirement. By providing a mathematically grounded way to accommodate diverse privacy preferences without sacrificing model accuracy, the authors offer a scalable solution for privacy-preserving distributed machine learning in heterogeneous environments.