通过懒惰影响近似实现高效个性化联邦学习，提升异构数据下模型性能。
Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Data
- 用分布式聚类方法按客户端数据特征分组训练
- 在真实任务中恢复全局模型因非独立同分布损失的性能，如CIFAR100提升17%
- 适合医疗、键盘预测等异构数据场景，计算开销低
在联邦学习中，客户端数据分布的异质性常导致单一全局模型无法为每个客户端提供最佳性能。例如键盘的下一个词预测模型，用户因年龄、方言、语言水平和书写风格不同，形成高度非独立同分布的数据集；类似情况也出现在不同设备拍摄的医学图像或不同车型的驾驶数据中。为此，我们提出一种简单有效的个性化联邦学习框架pFedLIA，采用计算高效的影响力近似方法——'懒惰影响'（Lazy Influence），在模型聚合前以分布式方式对客户端进行聚类。同一聚类内的数据所有者协作训练一个能捕捉本地数据模式的模型。实验表明，该方法在多种合成与真实场景中成功恢复了因非独立同分布导致的全局模型性能下降，包括北欧语言的下一个词预测任务及多个基准任务，其性能接近假设的理想聚类（Oracle clustering），显著优于现有基线，在CIFAR100上提升达17%。
原文摘要 · Abstract (English)
In Federated Learning, heterogeneity in client data distributions often means that a single global model does not have the best performance for individual clients. Consider for example training a next-word prediction model for keyboards: user-specific language patterns due to demographics (dialect, age, etc.), language proficiency, and writing style result in a highly non-IID dataset across clients. Other examples are medical images taken with different machines, or driving data from different vehicle types. To address this, we propose a simple yet effective personalized federated learning framework (pFedLIA) that utilizes a computationally efficient influence approximation, called `Lazy Influence', to cluster clients in a distributed manner before model aggregation. Within each cluster, data owners collaborate to jointly train a model that captures the specific data patterns of the clients. Our method has been shown to successfully recover the global model's performance drop due to the non-IID-ness in various synthetic and real-world settings, specifically a next-word prediction task on the Nordic languages as well as several benchmark tasks. It matches the performance of a hypothetical Oracle clustering, and significantly improves on existing baselines, e.g., an improvement of 17% on CIFAR100.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。