用原型层提升移动端个性化联邦学习的泛化能力
FedAli: Personalized Federated Learning Alignment with Prototype Layers for Generalized Mobile Services
- 引入原型层,通过记忆机制引导模型向个性化与共享原型对齐
- 在异构数据下使客户端泛化能力提升18.3%,同时减少模型漂移
- 适合需要跨设备适应的移动智能应用,如健康监测、语音识别
个性化联邦学习(PFL)支持在边缘设备上分布式训练,使模型在保护数据隐私的前提下协同学习全局模式,并适配各客户端的本地数据。然而,移动端的PFL面临两大挑战:数据异构导致的客户端漂移,以及对客户端泛化能力的忽视——而移动传感动态要求模型不仅适应本地环境,还需具备跨场景适应能力。为此,我们提出联邦对齐(FedAli),一种基于原型的正则化方法,可增强客户端间对齐性,同时强化个性化适应的鲁棒性。核心是受人类记忆启发的原型对齐(ALP)层,在推理时将嵌入导向个性化原型,训练时通过共享原型实现对齐以减少漂移。利用最优传输计划计算原型-嵌入分配,可在无类别标签情况下预训练原型,从而加速收敛并提升性能。大量实验表明,FedAli显著提升了客户端在异构设置下的泛化能力,同时保持强个性化。
原文摘要 · Abstract (English)
Personalized Federated Learning (PFL) enables distributed training on edge devices, allowing models to collaboratively learn global patterns while tailoring their parameters to better fit each client's local data, all while preserving data privacy. However, PFL faces two key challenges in mobile systems: client drift, where heterogeneous data cause model divergence, and the overlooked need for client generalization, as the dynamic of mobile sensing demands adaptation beyond local environments. To overcome these limitations, we introduce Federated Alignment (FedAli), a prototype-based regularization technique that enhances inter-client alignment while strengthening the robustness of personalized adaptations. At its core, FedAli introduces the ALignment with Prototypes (ALP) layer, inspired by human memory, to enhance generalization by guiding inference embeddings toward personalized prototypes while reducing client drift through alignment with shared prototypes during training. By leveraging an optimal transport plan to compute prototype-embedding assignments, our approach allows pre-training the prototypes without any class labels to further accelerate convergence and improve performance. Our extensive experiments show that FedAli significantly enhances client generalization while preserving strong personalization in heterogeneous settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。