arXiv:2605.21264cs.LG2026-05

提出联邦协同双层专家模型,兼顾全局泛化与本地个性化。

FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs

论文配图:FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs
图 1 · 摘自论文原文
  • 用双层专家架构动态分配数据,避免参数漂移和过拟合。
  • 全局准确率78.00%,个性化准确率89.32%,显著优于基线。
  • 新客户端无需微调即可达77.27%准确率,适合冷启动场景。

联邦学习在隐私保护的分布式学习中前景广阔,但面临根本挑战:传统平均方法在非独立同分布条件下易产生参数漂移,个性化方法则过度拟合本地数据且难以泛化至新客户端(冷启动问题)。混合专家模型通过将异构数据路由至专用专家,自然缓解该矛盾。本文提出FedCoE——一种联邦协同双层专家框架,服务器维护多个独立的全局专家模型,使用共享门控网络在聚合过程中动态建模客户端与专家的相关性,有效缓解专家漂移与门控不一致问题。为解决冷启动问题,引入自适应机制,使新客户端可立即利用全局专家池而无需大量本地训练。大量实验表明,FedCoE在平均上实现78.00%的全局准确率和89.32%的个性化准确率,分别优于基线8.82%和29.19%。在冷启动场景下,无需本地微调即可达到77.27%准确率,优于基线超12.54%。

原文摘要 · Abstract (English)

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving distributed learning. However, existing FL methods face a fundamental challenge. Traditional averaging-based approaches suffer from parameter divergence under non-IID conditions, while personalized FL methods overfit to local data and fail to generalize to new clients (cold-start problem). Mixture-of-Experts naturally addresses this by routing heterogeneous data to specialized experts rather than forcing uniform aggregation. In this paper, we propose FedCoE, a Federated Coordinated dual-level mixture-of-Experts framework that effectively balances global generalization with local personalization. FedCoE maintains multiple independent global expert models on the server and employs a shared gating network to dynamically model client-expert correlations during aggregation, effectively mitigating expert drift and gating inconsistency. To address the cold-start challenge, we introduce an adaptive mechanism that enables new clients to immediately leverage the global expert pool without extensive local training. Extensive experiments demonstrate that FedCoE achieves 78.00% global accuracy and 89.32% personalized accuracy on average, outperforming the baseline by 8.82% and 29.19%, respectively. In cold-start scenarios, FedCoE delivers 77.27% accuracy without any local fine-tuning, outperforming baselines by over 12.54%.

联邦学习混合专家个性化冷启动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。