Quantization in Federated Learning: Methods, Challenges and Future Directions

Based on the provided manuscript, here is a detailed technical summary of the paper "Quantization in Federated Learning: Methods, Challenges and Future Directions."

Problem Statement

Federated Learning (FL) has emerged as a foundational paradigm for privacy-preserving distributed intelligence, allowing multiple clients to collaboratively train a global model without sharing local data. However, the scalability of FL is fundamentally constrained by three primary bottlenecks:

1. Communication Overhead: The exchange of model updates (parameters, gradients, or activations) between clients and the central server often dominates training time and energy consumption, leading to latency and reduced participation from resource-constrained devices.
2. Device Heterogeneity: FL systems operate across diverse hardware (mobile, IoT, edge) with varying computational capabilities, memory constraints, and energy budgets.
3. Statistical Non-IID Data: Data distributions across clients are often non-independent and identically distributed (non-IID), which complicates model convergence and introduces client drift.

While various communication-efficient techniques exist (e.g., sparsification, pruning, knowledge distillation), many introduce additional overhead or complexity. Quantization, which reduces numerical precision to compress model parameters, gradients, or activations, offers a direct mechanism to mitigate these limitations by reducing both uplink/downlink payloads and on-device computation costs. However, existing literature lacks a dedicated, systematic review of quantization specifically within the FL context, often treating it merely as a sub-component of general model compression.

Methodology

The authors conducted a Systematic Literature Review (SLR) following the PRISMA (Preferred Reporting Items for Systematic Reviews) guidelines.

• Search Process: Articles were retrieved from major databases (IEEE Xplore, arXiv, ScienceDirect, ACM Digital Library, Wiley) covering the period from 2019 to 2026. Search strings combined terms such as "Federated Learning," "Quantization," and "Communication."
• Selection Criteria: The review focused on peer-reviewed English articles where quantization was a primary research focus within FL frameworks, specifically addressing improvements in communication-computation efficiency, energy efficiency, or system performance.
• Scope: From an initial pool of studies, the authors filtered for relevance, resulting in a curated set of works analyzed to address four specific Research Questions (RQs):
  • RQ1: How do different quantization methods impact FL performance, efficiency, and effectiveness?
  • RQ2: What challenges and limitations characterize existing quantization approaches in FL?
  • RQ3: What research gaps persist across algorithmic, statistical, and system dimensions?
  • RQ4: What emerging trends can guide the design of scalable, robust, and privacy-aware quantized FL systems?

Key Contributions

The paper provides the first FL-centric systematic review of quantization, introducing a novel taxonomy and critical analysis across four main dimensions:

1. Novel Taxonomy of Quantization in FL

The authors propose a taxonomy organized around four design dimensions, moving beyond simple categorization to analyze how quantization interacts with core FL behaviors:

• Training Stage: Distinguishes between Post-Training Quantization (PTQ) (applied after training, e.g., Dynamic Range Quantization) and Quantization-Aware Training (QAT) (integrating quantization noise simulation during training).
• Quantized Object: Categorizes methods based on what is compressed: Model Parameters, Gradients, Activations/Intermediate Features, or Model Updates (deltas).
• Encoding Structure: Analyzes Scalar vs. Vector quantization and Uniform vs. Non-Uniform schemes.
• Precision Strategy: Covers Fixed-bit, Variable-bit, and Adaptive Precision strategies.

2. System-Level Property Analysis

The review evaluates quantization methods against six FL-specific system properties:

• Client Heterogeneity: How methods adapt to varying device capabilities.
• Aggregation Consistency: The impact of quantization on the stability of server-side aggregation.
• Communication Scheduling Adaptation: Integration with client selection and scheduling policies.
• Non-IID Robustness: Performance under statistically heterogeneous data.
• Privacy/Security Integration: Interactions with secure aggregation and differential privacy.
• Hardware/Energy Co-optimization: Alignment with specific edge hardware constraints.

3. Critical Evaluation of Trade-offs

The paper identifies key limitations and trade-offs inherent in current approaches:

• Accuracy vs. Efficiency: While quantization reduces bandwidth and energy, it often introduces accuracy degradation, particularly under low bit-widths or non-IID data distributions.
• Convergence Instability: Quantization noise can lead to client drift and unstable convergence, necessitating techniques like bias correction or clipping.
• Calibration Challenges: In FL, global quantization thresholds may be biased due to unique client data distributions, requiring adaptive calibration (e.g., KLD minimization, MSE optimization) or layer-wise clipping.
• Implementation Complexity: Advanced methods like Mixed Precision Quantization (MPQ) or Federated Hybrid QAT offer better trade-offs but increase synchronization and coordination overhead.

4. Identification of Research Gaps and Future Directions

The authors highlight several critical gaps in the current literature:

• Lack of Standardized Benchmarks: There is no unified benchmark for evaluating quantized FL under realistic non-IID and heterogeneous settings.
• Insufficient Adaptive Strategies: Limited exploration of quantization strategies co-designed with aggregation, scheduling, and system constraints.
• Privacy-Utility Trade-offs: The interaction between quantization and privacy-preserving techniques (e.g., Secure Aggregation, Differential Privacy) is underexplored.

Future Directions proposed include:

• Adaptive Precision Mechanisms: Dynamically adjusting bit-widths based on client hardware and network conditions.
• Joint Design: Integrating quantization with aggregation policies and client scheduling.
• Hardware-Aware Optimization: Tailoring quantization for specific mobile, IoT, and edge platforms to maximize energy efficiency.

Results and Findings

The survey synthesizes existing methods to demonstrate that:

• PTQ (e.g., Dynamic Range Quantization) is highly effective for rapid deployment and reducing communication costs but may suffer accuracy drops in highly non-IID settings without adaptive calibration.
• QAT (e.g., QA-FedAvg, FedDQ) provides superior robustness against quantization-induced bias and maintains accuracy closer to full-precision models, though it requires local retraining and higher computational cost.
• Mixed Precision Quantization (MPQ) offers a flexible mechanism to balance accuracy and efficiency across heterogeneous clients but introduces complexity in synchronization and calibration.
• Gradient and Update Quantization are particularly effective for reducing uplink bandwidth, with stochastic quantization ensuring unbiased gradient estimators.

Significance of the Work

The paper asserts that quantization should not be viewed merely as a compression technique but as a fundamental systems component that shapes the performance, robustness, and practicality of modern Federated Learning. By establishing a unified taxonomy and bridging the gap between algorithmic design, system constraints, and security requirements, this survey provides a structured guide for researchers and practitioners. It aims to facilitate the deployment of scalable, communication-efficient, and resource-aware FL systems in real-world edge and mobile environments. The work serves as a foundation for future research aimed at overcoming the current limitations of FL scalability and heterogeneity.