用预测调度提升联邦学习效率,减少慢节点影响。
RIFLES: Resource-effIcient Federated LEarning via Scheduling
- 将客户端选择建模为调度问题,引入长期可用性预测
- 通过CNN-LSTM模型预测客户端参与时机,提升训练效率10%-50%
- 适合资源受限场景下需优化客户端调度的联邦学习应用
联邦学习(FL)是一种保护隐私的分布式机器学习方法,通过避免原始数据交换,在多个分布式客户端间协同训练模型。其核心是中心服务器在每轮中选择部分客户端参与训练。现有选择策略多基于历史或当前交互,具有短视性,常导致慢节点等问题。本文提出RIFLES方法,构建新的可用性预测层以支持客户端选择。主要贡献包括:(i) 将顺序选择问题形式化并简化为调度问题,证明其为NP完全问题;(ii) 利用客户端心跳消息构建长期可用性预测层;(iii) 提出一种自适应选择策略,提升学习效率与资源利用率。为克服指数级复杂度,设计基于历史可用性数据的启发式算法,采用CNN-LSTM时序预测模型,使服务器可预测客户端最优参与时间,从而做出更优选择。相比其他FL方法,RIFLES在准确率、测试损失等指标上提升10%-50%。据我们所知,这是首个将联邦学习视为调度问题的研究。
原文摘要 · Abstract (English)
Federated Learning (FL) is a privacy-preserving machine learning technique that allows decentralized collaborative model training across a set of distributed clients, by avoiding raw data exchange. A fundamental component of FL is the selection of a subset of clients in each round for model training by a central server. Current selection strategies are myopic in nature in that they are based on past or current interactions, often leading to inefficiency issues such as straggling clients. In this paper, we address this serious shortcoming by proposing the RIFLES approach that builds a novel availability forecasting layer to support the client selection process. We make the following contributions: (i) we formalise the sequential selection problem and reduce it to a scheduling problem and show that the problem is NP-complete, (ii) leveraging heartbeat messages from clients, RIFLES build an availability prediction layer to support (long term) selection decisions, (iii) we propose a novel adaptive selection strategy to support efficient learning and resource usage. To circumvent the inherent exponential complexity, we present RIFLES, a heuristic that leverages clients' historical availability data by using a CNN-LSTM time series forecasting model, allowing the server to predict the optimal participation times of clients, thereby enabling informed selection decisions. By comparing against other FL techniques, we show that RIFLES provide significant improvement by between 10%-50% on a variety of metrics such as accuracy and test loss. To the best of our knowledge, it is the first work to investigate FL as a scheduling problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。