解决边缘计算中专家模型负载不均问题，提升联邦学习效率
FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
- 根据本地数据评估专家适配度，动态分配最适合的专家
- 在资源受限设备上实现高精度与专家使用均衡
- 适合异构边缘设备上的联邦学习应用
混合专家（MoE）模型通过条件计算实现可扩展神经网络，在下一代无线通信中兼具高效性与有效性。然而，在资源受限的无线与物联网边缘网络中部署联邦学习（FL）时面临两大挑战：1）客户端无法存储包含全部专家的大型AI模型；2）非独立同分布（non-IID）数据导致专家负载严重不均，影响模型性能。为此，我们提出FLEX-MoE，一种联合优化专家分配与负载均衡的联邦MoE框架。该方法引入客户端-专家适配度评分，基于训练反馈量化专家对本地数据的适用性，并采用优化算法最大化客户端-专家专业化程度，同时全局保障专家使用均衡。与仅关注个性化的贪心方法不同，FLEX-MoE有效缓解了异构边缘联邦学习中的专家利用偏差。实验结果表明，在多种资源受限场景下，该方法显著提升了准确率并保持了稳定的专家使用均衡。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) models enable scalable neural networks through conditional computation, offering enhanced effectiveness and efficiency for next-generation wireless communications. However, deploying MoE with federated learning (FL) over wireless and IoT edge networks faces two critical challenges: 1) resource-constrained clients cannot store large AI models with full expert sets, and 2) non-IID data distributions cause severe expert load imbalance that degrades model performance. To this end, we propose FLEX-MoE, a federated MoE framework that jointly optimizes expert assignment and load balancing under limited client capacity. Specifically, our approach introduces client-expert fitness scores that quantify expert suitability for local datasets through training feedback, and employs an optimization-based algorithm to maximize client-expert specialization while enforcing balanced expert utilization system-wide. Unlike greedy methods that focus solely on personalization while ignoring load imbalance, FLEX-MoE addresses expert utilization skew, which is particularly severe in heterogeneous edge FL. Our experimental results demonstrate superior accuracy and consistently balanced expert utilization across diverse resource-constrained scenarios for edge computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。