用强化学习动态调整联邦学习客户端训练数据量,提升非独立同分布场景下的模型性能。
Optimized Local Updates in Federated Learning via Reinforcement Learning
- 用深度强化学习自动选择每轮本地训练最优数据量,避免过拟合或信息泄露。
- 在多个基准数据集上,相比传统方法准确率提升1.2%-3.8%,尤其在非独立同分布数据下表现更优。
- 适合关注联邦学习中客户端训练效率与隐私保护的研究者和工程实践者。
联邦学习(FL)是一种分布式框架,可在大规模分布式数据上协作训练模型,同时保护客户端数据隐私。然而,集中式服务器的模型聚合机制在客户端数据非独立同分布(non-IID)时会导致性能下降。本文指出,客户端过度使用本地数据并不会提升整体性能。为此,我们提出一种新框架:利用深度强化学习(DRL)代理,根据训练损失变化作为奖励信号,自动选择优化的本地训练数据量,从而在不向服务器泄露过多信息的前提下提升模型性能。具体而言,在每轮聚合后,DRL算法基于当前本地性能状态,输出下一阶段各类别数据的优化权重,学习生成局部训练数据的最优划分策略。训练结束后,客户端使用全部本地数据进一步提升对自身数据分布的适应能力,缓解聚合带来的非独立同分布影响。大量实验表明,该方法在多个基准数据集和联邦学习框架上均取得更优性能。代码已开源:https://github.com/amuraddd/optimized_client_training.git。
原文摘要 · Abstract (English)
Federated Learning (FL) is a distributed framework for collaborative model training over large-scale distributed data, enabling higher performance while maintaining client data privacy. However, the nature of model aggregation at the centralized server can result in a performance drop in the presence of non-IID data across different clients. We remark that training a client locally on more data than necessary does not benefit the overall performance of all clients. In this paper, we devise a novel framework that leverages a Deep Reinforcement Learning (DRL) agent to select an optimized amount of data necessary to train a client model without oversharing information with the server. Starting without awareness of the client's performance, the DRL agent utilizes the change in training loss as a reward signal and learns to optimize the amount of training data necessary for improving the client's performance. Specifically, after each aggregation round, the DRL algorithm considers the local performance as the current state and outputs the optimized weights for each class, in the training data, to be used during the next round of local training. In doing so, the agent learns a policy that creates an optimized partition of the local training dataset during the FL rounds. After FL, the client utilizes the entire local training dataset to further enhance its performance on its own data distribution, mitigating the non-IID effects of aggregation. Through extensive experiments, we demonstrate that training FL clients through our algorithm results in superior performance on multiple benchmark datasets and FL frameworks. Our code is available at https://github.com/amuraddd/optimized_client_training.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。