针对联邦学习中的数据异构问题,动态调整优化策略提升模型鲁棒性。
FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation): Scam-resistant SAM for Robust Federated Optimization in Heterogeneous Environments
- 按客户端数据异构程度动态调节扰动半径,避免高异构客户端干扰全局训练。
- 在CIFAR-10和Fashion-MNIST上比现有方法更快收敛,最终准确率更优。
- 适合数据分布不均的边缘设备联邦学习场景,尤其对非IID数据敏感的应用
联邦学习(FL)可在保护数据隐私的前提下实现跨分散边缘设备的协作模型训练。然而,客户端间存在的统计异构性(常表现为标签分布非独立同分布)严重影响了模型的收敛性与泛化能力。尽管已有研究将尖锐度感知最小化(SAM)引入联邦学习以寻找更平坦、更鲁棒的极小值,但现有方法通常对所有客户端采用统一的扰动半径,忽略了客户端间的异构差异。本文提出一种新算法——FedSCAM(基于聚类聚合与调制的联邦尖锐度感知最小化),根据每个客户端的异构度量动态调节其扰动半径,并反比例于该得分,从而抑制高异构客户端对全局模型的破坏。同时,设计了一种异构感知加权聚合机制,优先采纳与全局优化方向一致的客户端更新。在基于狄利克雷分布的标签偏斜设置下,于CIFAR-10与Fashion-MNIST上的大量实验表明,FedSCAM在收敛速度与最终测试准确率方面均达到当前先进基线(如FedSAM、FedLESAM等)水平。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, statistical heterogeneity among clients, often manifested as non-IID label distributions, poses significant challenges to convergence and generalization. While Sharpness-Aware Minimization (SAM) has been introduced to FL to seek flatter, more robust minima, existing approaches typically apply a uniform perturbation radius across all clients, ignoring client-specific heterogeneity. In this work, we propose \textbf{FedSCAM} (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation), a novel algorithm that dynamically adjusts the SAM perturbation radius and aggregation weights based on client-specific heterogeneity scores. By calculating a heterogeneity metric for each client and modulating the perturbation radius inversely to this score, FedSCAM prevents clients with high variance from destabilizing the global model. Furthermore, we introduce a heterogeneity-aware weighted aggregation mechanism that prioritizes updates from clients that align with the global optimization direction. Extensive experiments on CIFAR-10 and Fashion-MNIST under various degrees of Dirichlet-based label skew demonstrate that FedSCAM achieves competitive performance among state-of-the-art baselines, including FedSAM, FedLESAM, etc. in terms of convergence speed and final test accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。