arXiv:2602.23827cs.LGcs.AI2026-02被引 15

解决联邦学习中局部与全局平坦性不一致问题,提升模型泛化能力。

FedNSAM:Consistency of Local and Global Flatness for Federated Learning

  • 引入全局Nesterov动量,协调本地与全局的平坦性一致性。
  • 理论证明收敛更快,实验显示在CNN与Transformer上性能更优。
  • 适合数据异构严重的联邦学习场景,尤其关注泛化性能的研究者。

在联邦学习中,多步本地更新和数据异构常导致更尖锐的全局极小值,降低全局模型性能。现有算法虽将尖锐度感知最小化(SAM)融入本地训练以缓解此问题,但在高数据异构情况下,本地平坦性并不保证全局平坦性,因此难以提升全局模型的泛化能力。本文定义了 extbf{平坦距离}来解释该现象。通过重新思考联邦学习中的SAM,并理论分析平坦距离,提出新型 extbf{FedNSAM}算法:在本地更新中引入全局Nesterov动量,用于估计客户端全局扰动方向并进行外推,从而协调局部与全局平坦性。理论上,我们证明其收敛界比FedSAM更紧。实验在CNN与Transformer模型上验证了 extbf{FedNSAM}在性能与效率上的优越性。代码已开源。

原文摘要 · Abstract (English)

In federated learning (FL), multi-step local updates and data heterogeneity usually lead to sharper global minima, which degrades the performance of the global model. Popular FL algorithms integrate sharpness-aware minimization (SAM) into local training to address this issue. However, in the high data heterogeneity setting, the flatness in local training does not imply the flatness of the global model. Therefore, minimizing the sharpness of the local loss surfaces on the client data does not enable the effectiveness of SAM in FL to improve the generalization ability of the global model. We define the \textbf{flatness distance} to explain this phenomenon. By rethinking the SAM in FL and theoretically analyzing the \textbf{flatness distance}, we propose a novel \textbf{FedNSAM} algorithm that accelerates the SAM algorithm by introducing global Nesterov momentum into the local update to harmonize the consistency of global and local flatness. \textbf{FedNSAM} uses the global Nesterov momentum as the direction of local estimation of client global perturbations and extrapolation. Theoretically, we prove a tighter convergence bound than FedSAM by Nesterov extrapolation. Empirically, we conduct comprehensive experiments on CNN and Transformer models to verify the superior performance and efficiency of \textbf{FedNSAM}. The code is available at https://github.com/junkangLiu0/FedNSAM.

联邦学习优化算法泛化能力Nesterov动量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。