用超网络实现异构模型个性化联邦学习,无需外部数据。
Hypernetworks for Model-Heterogeneous Personalized Federated Learning
- 服务器端超网络根据客户端特征生成定制参数。
- 多头结构让相似模型的客户端共享计算,提升效率。
- 不依赖外部数据或模型结构信息,隐私性更强。
近期个性化联邦学习研究聚焦于解决客户端模型异构问题,但多数方法仍需外部数据、依赖模型解耦或采用部分学习策略,限制了实用性与可扩展性。本文重新审视基于超网络的方法,利用其强大的泛化能力,设计了一种简单高效的异构个性化联邦学习框架。我们提出MH-pFedHN,通过服务器端超网络接收客户端特定嵌入向量,并输出适配各客户端异构模型的个性化参数。为促进知识共享并降低计算开销,我们在超网络中引入多头结构,使模型规模相近的客户端可共享头部。此外,进一步提出MH-pFedHNGD,集成一个轻量级全局模型以增强泛化能力。本框架不依赖外部数据,也不需披露客户端模型架构,显著提升隐私性与灵活性。在多个基准和模型设置上的大量实验表明,该方法在准确率和泛化性能上表现优异,可作为未来异构个性化联邦学习研究的稳健基线。
原文摘要 · Abstract (English)
Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。