提出延迟动量聚合机制，让联邦学习在少数参与且有恶意节点时仍能保持稳定
Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation
- 服务器聚合未参与客户端的缓存动量与当前参与者的动量
- 在10%参与率下，20%恶意客户端仍能保持最高准确率
- 适合大规模分布式学习中存在恶意节点的场景
大规模联邦学习依赖部分参与以实现通信效率，但现有拜占庭鲁棒方法通常假设所有客户端参与。在部分参与设置下，采样客户端中可能多数为拜占庭客户端，一旦拜占庭客户端占主导，现有方法立即失效。本文提出延迟动量聚合原则：中央服务器同时聚合未采样客户端的缓存动量和采样客户端的最新动量。该机制确保即使拜占庭客户端主导采样集，从服务器视角看其仍为少数。基于此原则，我们设计了优化器DeMoA。理论分析表明，DeMoA在部分参与下具有拜占庭鲁棒性。实验显示，在20%拜占庭比例、仅10%参与率条件下，DeMoA性能优于现有方法，甚至在其他方法失败时仍能取得最佳准确率。
原文摘要 · Abstract (English)
Partial participation is essential for communication-efficient federated learning at scale, yet existing Byzantine-robust methods typically assume full client participation. In the partial participation setting, a majority of the sampled clients may be Byzantine, once Byzantine clients dominate, existing methods break down immediately. We introduce delayed momentum aggregation, a principle where the central server aggregates cached momentum from non-sampled clients along with fresh momentum from sampled clients. This principle ensures Byzantine clients remain a minority from the server's perspective even when they dominate the sampled set. We instantiate this principle in our optimizer DeMoA. We analyze the convergence rate of DeMoA, showing that DeMoA is Byzantine-robust under partial participation. Experiments show that, with 20% Byzantine ratio and only 10% partial participation rate, DeMoA achieves the best accuracy even when existing methods fail empirically.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。