Communication is widely known as the primary bottleneck of federated learning, and quantization of local model updates before uploading to the parameter server is an effective solution to reduce the communication overhead. However, prior literature always assumes homogeneous quantization for all clients, while in reality, devices are heterogeneous and support different levels of quantization precision. This heterogeneity of quantization poses a new challenge: fine-quantized model updates are more accurate than coarse-quantized ones, and how to optimally aggregate them at the server is an open problem. In this paper, we propose FedHQ – Federated Learning with Heterogeneous Quantization – that allocates different aggregation weights to different clients by minimizing the convergence rate upper bound as a function of the heterogeneous quantization errors of all clients, for both strongly convex and non-convex loss functions. To further accelerate the convergence, the instantaneous quantization error is computed and piggybacked when each client uploads the local model update, and the server dynamically calculates the weight accordingly for the current aggregation. Numerical experiment results demonstrate the performance advantages of FedHQ over both vanilla FedAvg with standard equal weights and a heuristic aggregation scheme, which assigns weights linearly proportional to the clients’ quantization precision.

中文翻译：

---

联邦学习中异构量化的动态聚合


通信被广泛认为是联邦学习的主要瓶颈，在上传到参数服务器之前对本地模型更新进行量化是减少通信开销的有效解决方案。然而，现有文献总是假设所有客户端都进行同质量化，而实际上，设备是异构的并且支持不同级别的量化精度。这种量化的异构性提出了新的挑战：精细量化的模型更新比粗量化的模型更新更准确，并且如何在服务器上最佳地聚合它们是一个悬而未决的问题。在本文中，我们提出了 FedHQ（异构量化联合学习），通过最小化收敛速度上限（作为所有客户端异构量化误差的函数），为不同客户端分配不同的聚合权重，无论是强凸损失还是非凸损失功能。为了进一步加速收敛，当每个客户端上传本地模型更新时，计算并搭载瞬时量化误差，并且服务器相应地动态计算当前聚合的权重。数值实验结果证明了 FedHQ 相对于具有标准等权重的普通 FedAvg 和启发式聚合方案（其分配与客户端的量化精度成线性比例的权重）的性能优势。