提出只更新部分网络层的联邦学习方法，加速收敛并降低开销。
Why Go Full? Elevating Federated Learning Through Partial Network Updates
- 每轮只更新一个或少数几层参数，缓解层间不匹配问题。
- 实验显示收敛更快、精度更高，通信与计算开销更低。
- 适合资源受限场景，尤其对异构设备训练有显著优势。
联邦学习是一种保护用户数据隐私的分布式机器学习范式，已在多种场景成功应用。传统联邦学习在每轮训练中对本地模型的所有参数进行更新和平均，虽能最大化各层知识共享，却导致全局模型各层无法有效协作完成客户端任务，此即层不匹配问题。该问题在每次参数平均后重复出现，阻碍模型收敛并降低性能。为解决此问题，我们提出FedPart方法，每轮仅对单一或少数层进行更新。同时，设计了顺序更新与多轮周期训练等策略以选择可训练层。理论分析与实验证明，相比传统全网更新，FedPart显著提升收敛速度与精度，同时降低通信与计算开销。
原文摘要 · Abstract (English)
Federated learning is a distributed machine learning paradigm designed to protect user data privacy, which has been successfully implemented across various scenarios. In traditional federated learning, the entire parameter set of local models is updated and averaged in each training round. Although this full network update method maximizes knowledge acquisition and sharing for each model layer, it prevents the layers of the global model from cooperating effectively to complete the tasks of each client, a challenge we refer to as layer mismatch. This mismatch problem recurs after every parameter averaging, consequently slowing down model convergence and degrading overall performance. To address the layer mismatch issue, we introduce the FedPart method, which restricts model updates to either a single layer or a few layers during each communication round. Furthermore, to maintain the efficiency of knowledge acquisition and sharing, we develop several strategies to select trainable layers in each round, including sequential updating and multi-round cycle training. Through both theoretical analysis and experiments, our findings demonstrate that the FedPart method significantly surpasses conventional full network update strategies in terms of convergence speed and accuracy, while also reducing communication and computational overheads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。