无需预设聚类数,自动识别最佳聚类时机,提升联邦学习个性化效率。
One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption
- 基于梯度余弦距离与温度感知,自动判断聚类启动时机。
- 在5个数据集上40+任务中表现稳定,无需调参。
- 验证了基于密度的聚类对不同数据分布的高效区分能力。
联邦学习(FL)是一种去中心化学习范式,允许多个客户端在不直接传输数据的情况下协同训练模型。自2015年提出以来,已发展出多个子领域,如应对数据异构性或资源分配等问题。其中,聚类联邦学习(CFL)旨在将客户端分组以提供个性化模型。尽管已有部分研究,但该问题仍处于探索阶段,因其假设和设置与标准FL略有不同。本文提出一种聚类无关的一次性聚类联邦学习(OCFL)算法,能自动检测最早合适的聚类时机。该方法通过计算客户端梯度间的余弦距离,并结合温度指标来判断联邦模型是否开始收敛。我们在五个基准数据集上对超过四十项任务测试了多种一次性聚类算法。实验表明,该方法可在无需调整超参数的情况下实现自动化聚类,性能优异。我们还重新评估了基于客户端梯度的聚类算法的实际可行性,提供了充分证据:基于密度的聚类方法在区分神经网络在不同分布上训练出的损失曲面方面具有极高效率。此外,通过分析GradCAM生成的本地解释,进一步揭示了个性化与局部预测可解释性之间的关系。
原文摘要 · Abstract (English)
Federated Learning (FL) is a widespread and well-adopted paradigm of decentralised learning that allows training one model from multiple sources without the need to transfer data between participating clients directly. Since its inception in 2015, it has been divided into numerous subfields that deal with application-specific issues, such as data heterogeneity or resource allocation. One such sub-field, Clustered Federated Learning (CFL), deals with the problem of clustering the population of clients into separate cohorts to deliver personalised models. Although a few remarkable works have been published in this domain, the problem remains largely unexplored, as its basic assumptions and settings differ slightly from those of standard FL. In this work, we present One-Shot Clustered Federated Learning (OCFL), a clustering-agnostic algorithm that can automatically detect the earliest suitable moment for clustering. Our algorithm is based on computing the cosine distance between the gradients of the clients and a temperature measure that detects when the federated model starts to converge. We empirically evaluate our methodology by testing various one-shot clustering algorithms for over forty different tasks on five benchmark datasets. Our experiments showcase the good performance of our approach when used to perform CFL in an automated manner without the need to adjust hyperparameters. We also revisit the practical feasibility of CFL algorithms based on the gradients of the clients, providing firm evidence of the high efficiency of density-based clustering methods when used to differentiate between the loss surfaces of neural networks trained on different distributions. Moreover, by inspecting the feasibility of local explanations generated with the help of GradCAM, we can provide more insights into the relationship between personalisation and the explainability of local predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。