用专家混合模型提升边缘计算持续学习能力,避免任务遗忘。
Theory of Mixture-of-Experts for Mobile Edge Computing
- 将每个边缘服务器视为专家,动态分配任务以实现专精
- 理论推导出最少所需专家数,保证任务与专家匹配
- 实验证明该方法可降低长期泛化误差,适合持续在线任务
在移动边缘计算(MEC)网络中,用户动态生成多样化的机器学习任务,通常被就近卸载到边缘服务器以兼顾通信与计算效率。然而,传统方式未使各服务器专精于特定任务类型,导致严重过拟合或灾难性遗忘。本文首次将专家混合(MoE)理论引入MEC,旨在缓解随时间累积的泛化误差。我们将每个边缘服务器视为一个专家,并设计自适应门控网络,根据数据传输与计算时延动态识别并路由新到达的、分布未知的任务至可用专家,使各专家在收敛后专精于特定任务类型。我们推导出匹配每类任务所需的最小专家数量。实验基于真实数据集,在深度神经网络上验证了理论结果:相较于传统方法,该方案能持续降低整体泛化误差;当专家数量足够保证收敛时,增加更多专家反而延缓收敛并恶化泛化性能。
原文摘要 · Abstract (English)
In mobile edge computing (MEC) networks, mobile users generate diverse machine learning tasks dynamically over time. These tasks are typically offloaded to the nearest available edge server, by considering communication and computational efficiency. However, its operation does not ensure that each server specializes in a specific type of tasks and leads to severe overfitting or catastrophic forgetting of previous tasks. To improve the continual learning (CL) performance of online tasks, we are the first to introduce mixture-of-experts (MoE) theory in MEC networks and save MEC operation from the increasing generalization error over time. Our MoE theory treats each MEC server as an expert and dynamically adapts to changes in server availability by considering data transfer and computation time. Unlike existing MoE models designed for offline tasks, ours is tailored for handling continuous streams of tasks in the MEC environment. We introduce an adaptive gating network in MEC to adaptively identify and route newly arrived tasks of unknown data distributions to available experts, enabling each expert to specialize in a specific type of tasks upon convergence. We derived the minimum number of experts required to match each task with a specialized, available expert. Our MoE approach consistently reduces the overall generalization error over time, unlike the traditional MEC approach. Interestingly, when the number of experts is sufficient to ensure convergence, adding more experts delays the convergence time and worsens the generalization error. Finally, we perform extensive experiments on real datasets in deep neural networks (DNNs) to verify our theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。