arXiv:2412.20020cs.LGcs.AI2024-12被引 4

解决个性化联邦学习中自监督模型泛化差的问题,提升准确率与公平性。

Calibre: Towards Fair and Accurate Personalized Federated Learning with Self-Supervised Learning

  • 引入客户端原型损失和原型引导聚合算法,校准自监督表示
  • 在非独立同分布设置下,平均准确率和客户端公平性均达领先水平
  • 适合数据异构场景下的个性化联邦学习研究与应用

在个性化联邦学习中,现有方法依赖全局模型提取可迁移表征,使各客户端可用少量数据训练个性化模型。自监督学习因其生成通用表征而被视为有前景的方向,有助于实现公平的个性化。然而,当客户端间数据异构时,仅用自监督学习训练的全局模型会产生模糊的类别边界,导致客户端个性化学习所得模型准确率低下。本文指出,未经修改的自监督学习全局模型产生的表征存在类别边界模糊问题。为此,提出Calibre框架,通过平衡通用与客户端特异性表征,校准自监督表示。该框架基于理论支撑,引入(1)客户端特定原型损失作为辅助目标;(2)基于跨客户端原型的聚合算法。大量非独立同分布设置下的实验表明,Calibre在平均准确率与客户端公平性方面均达到当前最优。代码仓库:https://github.com/TL-System/plato/tree/main/examples/ssl/calibre。

原文摘要 · Abstract (English)

In the context of personalized federated learning, existing approaches train a global model to extract transferable representations, based on which any client could train personalized models with a limited number of data samples. Self-supervised learning is considered a promising direction as the global model it produces is generic and facilitates personalization for all clients fairly. However, when data is heterogeneous across clients, the global model trained using SSL is unable to learn high-quality personalized models. In this paper, we show that when the global model is trained with SSL without modifications, its produced representations have fuzzy class boundaries. As a result, personalized learning within each client produces models with low accuracy. In order to improve SSL towards better accuracy without sacrificing its advantage in fairness, we propose Calibre, a new personalized federated learning framework designed to calibrate SSL representations by maintaining a suitable balance between more generic and more client-specific representations. Calibre is designed based on theoretically-sound properties, and introduces (1) a client-specific prototype loss as an auxiliary training objective; and (2) an aggregation algorithm guided by such prototypes across clients. Our experimental results in an extensive array of non-i.i.d.~settings show that Calibre achieves state-of-the-art performance in terms of both mean accuracy and fairness across clients. Code repo: https://github.com/TL-System/plato/tree/main/examples/ssl/calibre.

联邦学习自监督个性化公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。