通过损失曲面粗糙度感知，提升非独立同分布下的联邦学习稳定性
Roughness-Informed Federated Learning
- 引入粗糙度指数（RI）动态调节本地更新，抑制客户端漂移
- 在非独立同分布数据下，准确率更高且收敛速度更快
- 适用于真实场景中数据异构的联邦学习任务
联邦学习（FL）可在保护数据隐私的前提下实现分布式协作训练，但在非独立同分布（non-IID）设置下易受客户端漂移影响，导致收敛困难。本文提出一种新型算法 RI-FedAvg，通过在本地目标函数中加入基于粗糙度指数（RI）的正则化项，自适应地根据局部损失曲面的波动性惩罚更新，从而缓解客户端漂移问题。该方法利用RI量化高维损失函数的粗糙程度，在异构环境中确保优化鲁棒性。针对非凸目标函数，本文提供了严格的收敛性分析，证明了在标准假设下RI-FedAvg可收敛至平稳点。在MNIST、CIFAR-10和CIFAR-100上的大量实验表明，相比FedAvg、FedProx、FedDyn、SCAFFOLD及DP-FedAvg等先进基线方法，RI-FedAvg在非IID场景下实现了更高的准确率与更快的收敛速度。结果凸显了其在实际异构环境中的鲁棒性与效率潜力。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy, yet faces challenges in non-independent and identically distributed (non-IID) settings due to client drift, which impairs convergence. We propose RI-FedAvg, a novel FL algorithm that mitigates client drift by incorporating a Roughness Index (RI)-based regularization term into the local objective, adaptively penalizing updates based on the fluctuations of local loss landscapes. This paper introduces RI-FedAvg, leveraging the RI to quantify the roughness of high-dimensional loss functions, ensuring robust optimization in heterogeneous settings. We provide a rigorous convergence analysis for non-convex objectives, establishing that RI-FedAvg converges to a stationary point under standard assumptions. Extensive experiments on MNIST, CIFAR-10, and CIFAR-100 demonstrate that RI-FedAvg outperforms state-of-the-art baselines, including FedAvg, FedProx, FedDyn, SCAFFOLD, and DP-FedAvg, achieving higher accuracy and faster convergence in non-IID scenarios. Our results highlight RI-FedAvg's potential to enhance the robustness and efficiency of federated learning in practical, heterogeneous environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。