让联邦学习客户端共享私有参数，提升个性化模型效果
PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
- 引入个性化专家混合架构，动态选择其他客户端的私有参数
- 在6个数据集上平均提升性能，且训练开销几乎不变
- 适合数据异构严重的场景，如医疗、金融等隐私敏感领域
联邦学习因其保护隐私和协同学习的能力受到广泛关注。由于数据分布差异大，传统联邦学习难以在不同数据域间泛化共享模型。个性化联邦学习通过将模型分为全局共享部分和本地私有部分，由本地模型纠正全局模型引入的表征偏差。然而，本地收敛的参数更精准地捕捉了领域特异性知识，现有方法却未充分挖掘其价值。为此，本文提出PM-MoE架构，融合个性化专家模块与基于能量的去噪机制，使每个客户端可从其他客户端中选择有益的个性化参数。该架构已应用于9种基于模型分割的个性化联邦学习算法，在6个常用数据集和2种异构设置下均取得显著性能提升，且额外训练成本极低。源代码见 https://github.com/dannis97500/PM-MOE。
原文摘要 · Abstract (English)
Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditional FL struggles to generalize a shared model across diverse data domains. Personalized federated learning addresses this issue by dividing the model into a globally shared part and a locally private part, with the local model correcting representation biases introduced by the global model. Nevertheless, locally converged parameters more accurately capture domain-specific knowledge, and current methods overlook the potential benefits of these parameters. To address these limitations, we propose PM-MoE architecture. This architecture integrates a mixture of personalized modules and an energy-based personalized modules denoising, enabling each client to select beneficial personalized parameters from other clients. We applied the PM-MoE architecture to nine recent model-split-based personalized federated learning algorithms, achieving performance improvements with minimal additional training. Extensive experiments on six widely adopted datasets and two heterogeneity settings validate the effectiveness of our approach. The source code is available at \url{https://github.com/dannis97500/PM-MOE}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。