解决异构设备联邦学习训练慢问题，提升效率与精度。
FedEL: Federated Elastic Learning for Heterogeneous Devices
- 动态滑窗选择关键张量训练，按资源弹性分配任务。
- 相比基线提速3.87倍，最终准确率不降反升。
- 适合边缘计算、设备差异大的真实场景使用。
联邦学习（FL）使分布式设备在保护数据隐私的前提下协同训练模型。然而，设备硬件能力异构常导致显著训练延迟，资源有限的落后客户端会拖慢聚合进程。现有方法如客户端选择、异步联邦学习和部分训练虽部分缓解此问题，但存在准确率下降、更新过时及模型性能受损等缺陷。为此，我们提出FedEL，一种联邦弹性学习框架，可在保持模型精度的同时提升训练效率。FedEL引入基于窗口的训练机制，滑动窗口定位模型训练部分，并在协调运行时预算下动态选择重要张量进行训练，确保所有客户端（包括落后者）实现渐进且均衡的训练。此外，该框架采用张量重要性调整模块，调和局部与全局张量重要性，缓解数据异构带来的偏差。实验表明，FedEL相比基线最高实现3.87倍的时间-精度比提升，同时维持或超越最终测试准确率。
原文摘要 · Abstract (English)
Federated learning (FL) enables distributed devices to collaboratively train machine learning models while maintaining data privacy. However, the heterogeneous hardware capabilities of devices often result in significant training delays, as straggler clients with limited resources prolong the aggregation process. Existing solutions such as client selection, asynchronous FL, and partial training partially address these challenges but encounter issues such as reduced accuracy, stale updates, and compromised model performance due to inconsistent training contributions. To overcome these limitations, we propose FedEL, a federated elastic learning framework that enhances training efficiency while maintaining model accuracy. FedEL introduces a novel window-based training process, sliding the window to locate the training part of the model and dynamically selecting important tensors for training within a coordinated runtime budget. This approach ensures progressive and balanced training across all clients, including stragglers. Additionally, FedEL employs a tensor importance adjustment module, harmonizing local and global tensor importance to mitigate biases caused by data heterogeneity. The experiment results show that FedEL achieves up to 3.87x improvement in time-to-accuracy compared to baselines while maintaining or exceeding final test accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。