KARULA通过约束模型差异提升联邦学习个性化效果。
Personalized Federated Learning under Model Dissimilarity Constraints
- 基于1-Wasserstein距离代理度量客户端分布差异，约束模型间相似性。
- 在真实与合成数据集上，显著优于传统聚类方法的个性化性能。
- 适合异构数据场景下的个性化联邦学习，尤其适用于复杂客户关系建模。
联邦学习中客户端间的统计异质性是主要挑战之一。本文提出KARULA，一种正则化的个性化联邦学习策略，通过在客户端之间基于分布差异（以适应联邦设置的1-Wasserstein距离代理衡量）来约束成对模型差异，从而适应客户端间复杂的相互关系，例如聚类方法难以捕捉的情况。我们提出了一个非精确投影随机梯度算法来求解该约束问题，并从理论上证明了在平滑、可能非凸损失下，算法收敛到驻点邻域的速度为O(1/K)。我们在合成和真实联邦数据集上验证了KARULA的有效性。
原文摘要 · Abstract (English)
One of the defining challenges in federated learning is that of statistical heterogeneity among clients. We address this problem with KARULA, a regularized strategy for personalized federated learning, which constrains the pairwise model dissimilarities between clients based on the difference in their distributions, as measured by a surrogate for the 1-Wasserstein distance adapted for the federated setting. This allows the strategy to adapt to highly complex interrelations between clients, that e.g., clustered approaches fail to capture. We propose an inexact projected stochastic gradient algorithm to solve the constrained problem that the strategy defines, and show theoretically that it converges with smooth, possibly non-convex losses to a neighborhood of a stationary point with rate O(1/K). We demonstrate the effectiveness of KARULA on synthetic and real federated data sets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。