通过复用历史更新降低联邦学习通信开销，精度几乎不变。
Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning
- 按层评估梯度重要性，筛选关键层复用旧更新
- 在AG News数据集上通信量降至17%仍保持原精度
- 适合通信资源受限的边缘设备联邦学习场景
联邦学习中高昂的通信开销是其在实际应用中的主要瓶颈。现有高效方法多基于梯度大小丢弃部分模型更新。本文发现，与其直接丢弃，不如将历史更新进行复用，能更有效地降低通信成本并维持模型性能。为此提出FedLUAR：一种分层更新聚合与复用机制。该方法定义了一个量化指标，衡量每层聚合梯度对参数的影响程度，并据此选择若干层在服务器端复用其先前更新。大量实验表明，该更新复用策略显著降低了通信成本，同时保持模型精度。例如，在AG News数据集上，本方法达到与FedAvg相近的准确率，但通信成本仅为后者的17%。
原文摘要 · Abstract (English)
Expensive communication cost is a common performance bottleneck in Federated Learning (FL), which makes it less appealing in real-world applications. Many communication-efficient FL methods focus on discarding a part of model updates mostly based on gradient magnitude. In this study, we find that recycling previous updates, rather than simply dropping them, more effectively reduces the communication cost while maintaining FL performance. We propose FedLUAR, a Layer-wise Update Aggregation with Recycling scheme for communication-efficient FL. We first define a useful metric that quantifies the extent to which the aggregated gradients influences the model parameter values in each layer. FedLUAR selects a few layers based on the metric and recycles their previous updates on the server side. Our extensive empirical study demonstrates that the update recycling scheme significantly reduces the communication cost while maintaining model accuracy. For example, our method achieves nearly the same AG News accuracy as FedAvg, while reducing the communication cost to just 17%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。