用高斯生成模型实现联邦学习中的个性化建模,提升异构数据下的性能。
Personalized Federated Learning via Gaussian Generative Modeling
- 通过高斯生成器建模客户端数据分布差异,分离全局与本地优化目标。
- 在多个数据集上优于或相当主流方法,尤其在类别数量不均时表现更优。
- 适合处理数据异构性强的场景,如医疗、金融等隐私敏感领域。
联邦学习作为一种在保护隐私的前提下协同训练模型的范式,面临数据异质性挑战。个性化联邦学习通过为每个客户端配备专属模型来应对这一问题。现有方法通常将模型拆分为共享特征提取器和个性化分类头,由分类头引导表示学习。然而,这些工作忽略了表示分布中潜在的个性化特征。为此,本文提出 pFedGM,基于高斯生成建模的方法。首先训练一个高斯生成器,通过加权重采样建模客户端异质性;再通过双目标机制平衡全局协作与个性化:全局目标最大化跨客户端类间距离,局部目标最小化客户端内类间距离。为此,将传统高斯分类器解耦为导航器(用于全局优化)与统计提取器(用于捕获分布特征)。借鉴卡尔曼增益思想,在全局与本地层面采用双尺度融合框架,为每个客户端生成个性化分类头。该框架将全局表示分布视为先验,客户端数据视为似然,实现贝叶斯推断以估计类别概率。评估覆盖类别数量差异、环境噪声污染等多种场景,涵盖多个基准数据集与配置。pFedGM 在多数情况下优于或媲美当前最优方法。
原文摘要 · Abstract (English)
Federated learning has emerged as a paradigm to train models collaboratively on inherently distributed client data while safeguarding privacy. In this context, personalized federated learning tackles the challenge of data heterogeneity by equipping each client with a dedicated model. A prevalent strategy decouples the model into a shared feature extractor and a personalized classifier head, where the latter actively guides the representation learning. However, previous works have focused on classifier head-guided personalization, neglecting the potential personalized characteristics in the representation distribution. Building on this insight, we propose pFedGM, a method based on Gaussian generative modeling. The approach begins by training a Gaussian generator that models client heterogeneity via weighted re-sampling. A balance between global collaboration and personalization is then struck by employing a dual objective: a shared objective that maximizes inter-class distance across clients, and a local objective that minimizes intra-class distance within them. To achieve this, we decouple the conventional Gaussian classifier into a navigator for global optimization, and a statistic extractor for capturing distributional statistics. Inspired by the Kalman gain, the algorithm then employs a dual-scale fusion framework at global and local levels to equip each client with a personalized classifier head. In this framework, we model the global representation distribution as a prior and the client-specific data as the likelihood, enabling Bayesian inference for class probability estimation. The evaluation covers a comprehensive range of scenarios: heterogeneity in class counts, environmental corruption, and multiple benchmark datasets and configurations. pFedGM achieves superior or competitive performance compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。