CoMoE让边缘设备高效运行大模型,内存降70%、延迟更低。
CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
- 动态协同优化专家聚合与卸载策略,适应实时资源与网络变化。
- 在真实边缘测试中,7.4B模型内存从15.6GB降至4.7GB,推理延迟降10.5%。
- 适合移动边缘部署大模型的开发者,尤其关注资源受限场景。
大型语言模型(LLMs)的普及推动了混合专家(MoE)架构的发展,该架构可在控制计算成本的同时扩展模型容量。然而,在资源受限的移动边缘计算环境中部署MoE模型面临巨大挑战,主要源于其庞大的内存占用和动态专家激活模式。为此,我们提出一种新型动态资源感知协同优化框架CoMoE,联合优化专家聚合粒度与卸载策略,依据设备实时资源状态、网络条件及输入特征进行自适应调整。CoMoE首先系统分析现有专家聚合技术(如参数合并、知识蒸馏、参数共享分解),揭示其在动态移动环境中的局限性;进而研究专家卸载策略(包括预测与预取、缓存与调度、多级存储架构),发现路由决策与卸载性能间的内在关联。CoMoE引入自适应调度机制,响应用户移动性与网络波动,实现异构边缘设备上的高效部署。在真实移动边缘测试平台上开展的大量实验表明,CoMoE相较基线方法内存使用降低约70%,推理延迟比现有卸载技术低10.5%,同时保持模型性能稳定。对于大规模MoE模型(如7.4B参数的Switch-Base-128),CoMoE将内存需求从15.6GB降至4.7GB,使原本仅支持小型模型的资源受限边缘设备也能部署大模型。
原文摘要 · Abstract (English)
The proliferation of large language models (LLMs) has driven the adoption of Mixture-of-Experts (MoE) architectures as a promising solution to scale model capacity while controlling computational costs. However, deploying MoE models in resource-constrained mobile edge computing environments presents significant challenges due to their large memory footprint and dynamic expert activation patterns. To address these challenges, we propose a novel dynamic resource-aware collaborative optimization framework that jointly optimizes expert aggregation granularity and offloading strategies based on real-time device resource states, network conditions, and input characteristics in mobile edge environments, denoted as CoMoE. In CoMoE, we first systematically analyze existing expert aggregation techniques, including expert parameter merging,knowledge distillation,and parameter sharing decomposition, identifying their limitations in dynamic mobile environments.We then investigate expert offloading strategies encompassing expert prediction and prefetching, expert caching and scheduling, and multi-tier storage architectures, revealing the interdependencies between routing decisions and offloading performance.The CoMoE incorporates adaptive scheduling mechanisms that respond to user mobility and varying network conditions, enabling efficient MoE deployment across heterogeneous edge devices. Extensive experiments on real mobile edge testbeds demonstrate that CoMoE achieves approximately 70% reduction in memory usage compared to baseline methods, 10.5% lower inference latency than existing expert offloading techniques, while maintaining model performance stability. For large-scale MoE models (e.g,7.4B-parameter Switch-Base-128), the CoMoE reduces memory requirements from 15.6GB to 4.7GB, enabling deployment on resource-constrained mobile edge devices that previously could only support much smaller models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。