Federated Learning Architecture: Data Privacy and System Security Approaches

Technical Summary: Federated Learning Architecture with Homomorphic Encryption and Differential Privacy

Problem Statement
While Federated Learning (FL) offers a paradigm for training machine learning models without centralizing raw data, it remains vulnerable to security and privacy threats. Specifically, model updates transmitted between clients and the central server can be intercepted to perform model inference or data reconstruction attacks, potentially leaking sensitive information about local datasets. Furthermore, the central server itself presents a single point of failure. Existing approaches often struggle to balance the rigorous requirements of data privacy with the need for high model accuracy and computational efficiency, particularly in sensitive domains like healthcare and finance.

Methodology
This study proposes a hybrid FL architecture that integrates Homomorphic Encryption (HE) and Differential Privacy (DP) to secure the training process.

• Architecture & Encryption: The system utilizes the CKKS (Cheon-Kim-Kim-Song) homomorphic encryption scheme. CKKS was selected for its ability to perform approximate computations on encrypted floating-point numbers, which is essential for neural network operations. In this workflow, clients perform local training on their data. Instead of sending plaintext weights, clients encrypt their model updates using CKKS before transmission. The central server aggregates these encrypted updates using the Federated Averaging (FedAvg) algorithm. The aggregated result is then decrypted to update the global model, ensuring that the server never accesses plaintext model parameters.

• Differential Privacy: To prevent the extraction of individual information from model updates, the study employs DP mechanisms during the local training phase. Using the PrivacyEngine library, random noise is added to the gradients. The system operates under $(\epsilon, \delta)$-differential privacy constraints, with specific parameters set to $\sigma=5.0$ (noise multiplier), max gradient norm of $0.5$, and $\delta=10^{-5}$.

• Experimental Setup: The proposed architecture was evaluated on three distinct datasets:

  1. Framingham Heart Study: For cardiovascular disease prediction (4,240 samples).
  2. Pima Indians Diabetes (PID): For diabetes prediction (768 samples).
  3. Bank Marketing: For customer subscription prediction (41,188 samples).

  The experiments simulated a decentralized environment with varying numbers of clients (3, 5, and 10). A multi-layer Artificial Neural Network (ANN) with three fully connected layers (256 and 128 neurons in hidden layers) was trained for 5 local epochs per round over 10 global rounds.

Key Contributions

1. Integrated Security Framework: The paper demonstrates a functional FL system that simultaneously applies CKKS-based homomorphic encryption for secure transmission and DP for local gradient protection.
2. Empirical Analysis of Privacy-Accuracy Trade-offs: The study provides a detailed analysis of how increasing the number of clients and the duration of training (number of rounds) impacts the privacy budget ($\epsilon$). It observes that as the number of clients increases or dataset sizes decrease, the privacy budget is consumed more rapidly due to the increased necessity of noise addition per client.
3. Performance Evaluation: The research quantifies the impact of these security measures on model performance (Accuracy, Precision, Recall, F1-score) across different data distributions and client counts.

Experimental Results

• Privacy Budget ($\epsilon$): The study found a direct correlation between the number of clients and the privacy budget. For instance, in the Bank dataset, increasing clients from 3 to 10 resulted in the $\epsilon$ value at Round 10 nearly doubling (from 0.3566 to 0.6785). Similarly, smaller datasets (like PID) exhibited higher $\epsilon$ values compared to larger datasets (like Bank) under the same client configurations, indicating that data heterogeneity and sample size significantly influence privacy consumption.
• Model Accuracy: The integration of DP resulted in a measurable, albeit slight, decrease in model accuracy compared to non-private baselines.
  • Bank Dataset: Showed the highest robustness, achieving 81.85% accuracy with DP (3 clients) compared to 86.20% without DP.
  • PID Dataset: Showed a more pronounced drop, with 72.00% accuracy with DP (3 clients) versus 75.00% without.
  • Framingham Dataset: Achieved 69.92% accuracy with DP (3 clients) versus 71.47% without.
• Conclusion on Trade-offs: The results confirm that while privacy-preserving techniques introduce a performance cost, the system maintains meaningful accuracy levels, particularly in larger datasets like the Bank Marketing data.

Significance and Claims
The authors claim that this architecture demonstrates the feasibility of deploying secure, privacy-preserving AI in sensitive sectors such as healthcare and finance without relying on centralized data collection. The study concludes that while data heterogeneity and the number of clients significantly affect model performance, strategies such as careful selection of DP parameters and the use of larger datasets can mitigate efficiency losses.

The paper modestly acknowledges its limitations, noting that the experiments were conducted in a simulated environment with a small number of clients (3, 5, 10) and relatively large local datasets. The authors state that in real-world scenarios with thousands of clients and sparse local data, performance degradation is anticipated. Consequently, the paper positions its findings as a foundational step, suggesting that future work must address scalability, communication efficiency, and advanced aggregation algorithms to fully realize the potential of privacy-preserving FL in large-scale applications.