提出分层知识蒸馏机制，提升联邦聚类学习的个性化与全局性能。
Clustered Federated Learning with Hierarchical Knowledge Distillation
- 采用双层聚合框架，实现边缘聚类模型与云端统一模型协同训练。
- 在多个基准数据集上，相比基线模型准确率提升3.32%至7.57%。
- 适合需要隐私保护与个性化服务的物联网场景，如智能医疗、车联网。
聚类联邦学习（CFL）作为一种应对大规模分布式物联网环境中的数据异构性并保障隐私的有效方法，通过将客户端分组并训练各集群专属模型，实现了面向异构客户端的个性化建模。然而，传统CFL方法因为每个集群独立训练全局模型而造成学习碎片化，未能充分利用各集群间的集体知识。本文倡导采用分层联邦学习范式，在边缘端训练集群特定模型，在云端训练统一全局模型，实现双层聚合。这一转变虽提升了训练效率，但带来了通信挑战。为此，我们提出CFLHKD，一种新型个性化方案，通过多教师知识蒸馏将层级聚类知识融入联邦学习框架。该方法利用双层聚合机制弥合本地与全局学习之间的差距。在标准基准数据集上的大量实验表明，相较于代表性基线方法，CFLHKD在集群特定模型和全局模型的准确性上均表现更优，性能提升达3.32%至7.57%。
原文摘要 · Abstract (English)
Clustered Federated Learning (CFL) has emerged as a powerful approach for addressing data heterogeneity and ensuring privacy in large distributed IoT environments. By clustering clients and training cluster-specific models, CFL enables personalized models tailored to groups of heterogeneous clients. However, conventional CFL approaches suffer from fragmented learning for training independent global models for each cluster and fail to take advantage of collective cluster insights. This paper advocates a shift to hierarchical CFL, allowing bi-level aggregation to train cluster-specific models at the edge and a unified global model at the cloud. This shift improves training efficiency yet might introduce communication challenges. To this end, we propose CFLHKD, a novel personalization scheme for integrating hierarchical cluster knowledge into CFL. Built upon multi-teacher knowledge distillation, CFLHKD enables inter-cluster knowledge sharing while preserving cluster-specific personalization. CFLHKD adopts a bi-level aggregation to bridge the gap between local and global learning. Extensive evaluations of standard benchmark datasets demonstrate that CFLHKD outperforms representative baselines in cluster-specific and global model accuracy and achieves a performance improvement of 3.32-7.57\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。