提出新方法提升联邦学习全局平坦性，改善模型泛化能力。
Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning
- 在服务器端优化全局平坦性，避免本地尖锐极小值陷阱。
- 仅用上轮全局梯度近似尖锐度，通信开销几乎为零。
- 在多个视觉基准上优于现有方法，泛化性能显著提升。
联邦学习（FL）可在保护隐私的前提下实现协同模型训练。但边缘设备间的数据异质性常导致模型收敛至尖锐极小值，影响泛化与鲁棒性。现有方法采用客户端的尖锐度感知最小化（SAM）以追求更平坦的极小值，但局部与全局损失景观的差异常削弱其效果，因局部平坦不等于全局平坦。本文提出联邦全局服务器端尖锐度优化（FedGloSS），在服务器端使用SAM优化全局平坦性。为降低通信开销，该方法巧妙地利用上一轮全局梯度近似尖锐度，无需额外客户端通信。大量实验表明，与当前最优联邦学习方法相比，FedGloSS能持续收敛到更平坦的极小值，并在多个联邦视觉基准上取得更优性能。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative model training with privacy preservation. Data heterogeneity across edge devices (clients) can cause models to converge to sharp minima, negatively impacting generalization and robustness. Recent approaches use client-side sharpness-aware minimization (SAM) to encourage flatter minima, but the discrepancy between local and global loss landscapes often undermines their effectiveness, as optimizing for local sharpness does not ensure global flatness. This work introduces FedGloSS (Federated Global Server-side Sharpness), a novel FL approach that prioritizes the optimization of global sharpness on the server, using SAM. To reduce communication overhead, FedGloSS cleverly approximates sharpness using the previous global gradient, eliminating the need for additional client communication. Our extensive evaluations demonstrate that FedGloSS consistently reaches flatter minima and better performance compared to state-of-the-art FL methods across various federated vision benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。