用单个量化标量代替梯度，降低联邦学习通信开销和能耗。
Communication and Energy Efficient Federated Learning using Zero-Order Optimization Technique
- 每轮仅上传一个量化标量，大幅减少通信量。
- 在非凸场景下证明收敛性并给出上界，支持实际部署。
- 适合低带宽、低功耗设备，如物联网终端。
联邦学习（FL）是一种允许多个用户在保护数据隐私的前提下协同训练模型的热门机器学习技术。其主要挑战在于上行通信瓶颈及由此带来的设备高能耗，这源于模型/梯度尺寸的持续增大。本文提出一种零阶（ZO）优化方法，使每个设备每轮只需上传一个量化后的标量，而非完整的梯度向量。我们证明了该方法的理论收敛性，并在非凸设置下推导出其收敛速率的上界，同时讨论了在实际无线场景中量化误差和包丢失的影响。结果表明，相较于标准梯度基联邦学习方法，本方法在通信开销和能耗方面均具明显优势。
原文摘要 · Abstract (English)
Federated learning (FL) is a popular machine learning technique that enables multiple users to collaboratively train a model while maintaining the user data privacy. A significant challenge in FL is the communication bottleneck in the upload direction, and thus the corresponding energy consumption of the devices, attributed to the increasing size of the model/gradient. In this paper, we address this issue by proposing a zero-order (ZO) optimization method that requires the upload of a quantized single scalar per iteration by each device instead of the whole gradient vector. We prove its theoretical convergence and find an upper bound on its convergence rate in the non-convex setting, and we discuss its implementation in practical scenarios. Our FL method and the corresponding convergence analysis take into account the impact of quantization and packet dropping due to wireless errors. We show also the superiority of our method, in terms of communication overhead and energy consumption, as compared to standard gradient-based FL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。