自适应调整学习率，让异常客户端不拖后腿
Adaptive Decentralized Federated Learning for Robust Optimization
- 根据客户端可信度动态调整学习率，异常者低速、正常者高速
- 无需预知可靠客户端，无严格邻居数量要求，理论保证收敛
- 适合高噪声或被攻击的分布式训练场景，提升模型鲁棒性
在去中心化联邦学习（DFL）中，异常客户端（由噪声或投毒数据导致）会严重干扰学习过程并降低模型整体鲁棒性。现有方法通常需要足够多的正常邻近客户端或已知可靠客户端，限制了实际应用。为此，本文提出一种新型自适应去中心化联邦学习（aDFL）方法，通过自适应调整客户端学习率：对可疑客户端赋予较小学习率，对正常客户端赋予较大学习率，从而以完全自适应方式缓解异常客户端对全局模型的负面影响。理论分析不依赖严格邻接节点条件，也无需先验知识，并提供严格的收敛性证明，确保aDFL具备优良的统计性质。大量数值实验验证了aDFL方法的优越性能。
原文摘要 · Abstract (English)
In decentralized federated learning (DFL), the presence of abnormal clients, often caused by noisy or poisoned data, can significantly disrupt the learning process and degrade the overall robustness of the model. Previous methods on this issue often require a sufficiently large number of normal neighboring clients or prior knowledge of reliable clients, which reduces the practical applicability of DFL. To address these limitations, we develop here a novel adaptive DFL (aDFL) approach for robust estimation. The key idea is to adaptively adjust the learning rates of clients. By assigning smaller rates to suspicious clients and larger rates to normal clients, aDFL mitigates the negative impact of abnormal clients on the global model in a fully adaptive way. Our theory does not put any stringent conditions on neighboring nodes and requires no prior knowledge. A rigorous convergence analysis is provided to guarantee the oracle property of aDFL. Extensive numerical experiments demonstrate the superior performance of the aDFL method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。