用粒子群算法优化联邦学习聚合节点部署,提速近三分之一。
Towards a Distributed Federated Learning Aggregation Placement using Particle Swarm Intelligence
- 基于处理延迟的粒子群优化,动态决定聚合节点位置。
- 实测总处理时间比随机部署快43%,比均匀部署快32%。
- 适合资源受限、需低通信开销的分布式联邦学习场景。
联邦学习作为一种保障数据隐私的分布式学习范式,近年来备受关注。其中层次化半去中心化联邦学习(SDFL)通过在多级节点间分摊聚合任务,实现并行化处理。现有方法通常依赖每轮监控节点性能与资源消耗,导致频繁系统数据交换。为在最小化系统通信的前提下实现高效聚合,本文提出Flag-Swap:一种仅依据处理延迟进行优化的粒子群优化(PSO)方法。仿真结果表明,该方法能在多客户端候选场景中快速找到最优聚合部署方案。真实环境下的Docker实现显示,相较黑箱确定性策略,总处理时间比随机部署快约43分钟,比均匀部署快约32分钟。
原文摘要 · Abstract (English)
Federated learning has become a promising distributed learning concept with extra insurance on data privacy. Extensive studies on various models of Federated learning have been done since the coinage of its term. One of the important derivatives of federated learning is hierarchical semi-decentralized federated learning, which distributes the load of the aggregation task over multiple nodes and parallelizes the aggregation workload at the breadth of each level of the hierarchy. Various methods have also been proposed to perform inter-cluster and intra-cluster aggregation optimally. Most of the solutions, nonetheless, require monitoring the nodes' performance and resource consumption at each round, which necessitates frequently exchanging systematic data. To optimally perform distributed aggregation in SDFL with minimal reliance on systematic data, we propose Flag-Swap, a Particle Swarm Optimization (PSO) method that optimizes the aggregation placement according only to the processing delay. Our simulation results show that PSO-based placement can find the optimal placement relatively fast, even in scenarios with many clients as candidates for aggregation. Our real-world docker-based implementation of Flag-Swap over the recently emerged FL framework shows superior performance compared to black-box-based deterministic placement strategies, with about 43% minutes faster than random placement, and 32% minutes faster than uniform placement, in terms of total processing time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。