通过自蒸馏实现个性化联邦学习，提升本地模型适应性。
Personalized Federated Learning via Backbone Self-Distillation
- 客户端用全局骨干网络作为教师，蒸馏更新本地骨干。
- 共享骨干与私有头部协同，兼顾全局知识与本地差异。
- 相比12种先进方法，显著提升个性化性能，适合异构数据场景。
在实际应用中，联邦学习常需为每个客户端基于异构数据训练个性化模型。本文提出一种骨干自蒸馏方法，使客户端仅需向服务器发送骨干权重，由服务器聚合生成全局骨干并返回。为解决本地骨干缺乏个性化的问题，客户端进一步以全局骨干为教师，进行自蒸馏，将知识迁移至本地骨干。该过程同时学习共享骨干（用于通用表征）与私有头部（用于本地个性化），实现有效的全局知识传递。大量实验表明，本方法在12种前沿方法中表现优异。
原文摘要 · Abstract (English)
In practical scenarios, federated learning frequently necessitates training personalized models for each client using heterogeneous data. This paper proposes a backbone self-distillation approach to facilitate personalized federated learning. In this approach, each client trains its local model and only sends the backbone weights to the server. These weights are then aggregated to create a global backbone, which is returned to each client for updating. However, the client's local backbone lacks personalization because of the common representation. To solve this problem, each client further performs backbone self-distillation by using the global backbone as a teacher and transferring knowledge to update the local backbone. This process involves learning two components: the shared backbone for common representation and the private head for local personalization, which enables effective global knowledge transfer. Extensive experiments and comparisons with 12 state-of-the-art approaches demonstrate the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。