arXiv:2411.12244cs.LGcs.DC2024-11被引 3

为联邦学习设计自适应调参机制,提升资源受限设备上的优化效率

Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism

  • 采用分步反馈机制缩小搜索空间,加速超参数调优
  • 在FEMNIST和CIFAR10上验证工具有效性,显著降低训练耗时
  • 适合边缘计算场景下的自动化联邦学习部署

联邦学习(FL)通过本地参数而非原始数据共享来保护隐私,但其超参数调优在大量客户端与服务器间多轮通信下变得耗时且难以在资源受限的边缘设备(如物联网设备)上实现。本文研究了轻量级超参数优化工具Raytune和Optuna在联邦学习中的集成应用,并设计了一种分步反馈机制,结合本地与全局反馈以缩小搜索空间、加快调优过程。同时提出新型客户端选择策略以缓解慢速客户端问题。在两个基准数据集FEMNIST和CIFAR10上进行了评估,结果表明该方法能有效提升调优效率。论文还讨论了成功HPO工具的关键特性、与FL流水线的整合方式,以及分布式异构环境带来的挑战。

原文摘要 · Abstract (English)

Federated Learning (FL) is a decentralized learning approach that protects sensitive information by utilizing local model parameters rather than sharing clients' raw datasets. While this privacy-preserving method is widely employed across various applications, it still requires significant development and optimization. Automated Machine Learning (Auto-ML) has been adapted for reducing the need for manual adjustments. Previous studies have explored the integration of AutoML with different FL algorithms to evaluate their effectiveness in enhancing FL settings. However, Automated FL (Auto-FL) faces additional challenges due to the involvement of a large cohort of clients and global training rounds between clients and the server, rendering the tuning process time-consuming and nearly impossible on resource-constrained edge devices (e.g., IoT devices). This paper investigates the deployment and integration of two lightweight Hyper-Parameter Optimization (HPO) tools, Raytune and Optuna, within the context of FL settings. A step-wise feedback mechanism has also been designed to accelerate the hyper-parameter tuning process and coordinate AutoML toolkits with the FL server. To this end, both local and global feedback mechanisms are integrated to limit the search space and expedite the HPO process. Further, a novel client selection technique is introduced to mitigate the straggler effect in Auto-FL. The selected hyper-parameter tuning tools are evaluated using two benchmark datasets, FEMNIST, and CIFAR10. Further, the paper discusses the essential properties of successful HPO tools, the integration mechanism with the FL pipeline, and the challenges posed by the distributed and heterogeneous nature of FL environments.

联邦学习超参数优化边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。