针对联邦学习中设备差异，提出多策略优化方法提升精度与收敛速度。
Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillation
- 通过模型增量正则化实现高效中心化更新，降低通信开销。
- 结合知识蒸馏与个性化模型，在异构数据下显著提升性能。
- 适用于医疗等隐私敏感领域，尤其适合资源不均的客户端场景。
联邦学习（FL）已成为隐私敏感领域（如医疗）的重要训练范式。然而，客户端在数据分布、计算能力及任务上的异质性带来显著挑战。为此，本文提出一种集成模型增量正则化、个性化模型、联邦知识蒸馏与混池化（mix-pooling）的优化算法。模型增量正则化在服务器端集中优化模型更新，使客户端以最小通信成本完成高效更新。个性化模型与联邦知识蒸馏策略有效应对任务异构性。此外，引入混池化以适应读出操作对不同客户端敏感度的差异。实验表明，模型增量正则化实现了显著更高的准确率和快速收敛；联邦知识蒸馏在多样化数据场景下显著改善了整体性能；混池化读出机制为客户端带来实际收益，验证了所提方法的有效性。
原文摘要 · Abstract (English)
Federated learning (FL) has emerged as a transformative training paradigm, particularly invaluable in privacy-sensitive domains like healthcare. However, client heterogeneity in data, computing power, and tasks poses a significant challenge. To address such a challenge, we propose an FL optimization algorithm that integrates model delta regularization, personalized models, federated knowledge distillation, and mix-pooling. Model delta regularization optimizes model updates centrally on the server, efficiently updating clients with minimal communication costs. Personalized models and federated knowledge distillation strategies are employed to tackle task heterogeneity effectively. Additionally, mix-pooling is introduced to accommodate variations in the sensitivity of readout operations. Experimental results demonstrate the remarkable accuracy and rapid convergence achieved by model delta regularization. Additionally, the federated knowledge distillation algorithm notably improves FL performance, especially in scenarios with diverse data. Moreover, mix-pooling readout operations provide tangible benefits for clients, showing the effectiveness of our proposed methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。