arXiv:2607.17154cs.NIcs.DC2026-07被引 2

通过专家相似性优化边缘MoE推理,降低通信开销与延迟。

OrderMoE: An expert similarity driven distributed edge MoE inference

  • 基于路由器输出构建专家相似性模型,分组部署以提升本地覆盖。
  • 实验显示平均延迟和跨服务器流量下降,远程调用率显著减少。
  • 适合资源受限的边缘部署场景,兼顾性能与推理质量。

尽管混合专家(MoE)模型能以较低计算成本扩展大语言模型,但在资源受限、带宽有限的边缘基础设施上部署仍具挑战。现有分布式MoE服务方法多依赖精确专家放置、缓存、复制或通信调度,忽略了专家间的功能相似性,而这一特性可有效减少跨服务器的令牌传输。为此,本文提出一种基于专家相似性的分布式部署框架OrderMoE,旨在加速边缘MoE推理,同时平衡推理延迟、通信开销、服务器负载与推理质量。OrderMoE首先基于路由生成的逻辑值表示构建专家相似性模型,并将每层MoE中的专家划分为多个相似组;随后设计相似性感知的分组与部署策略,提升各边缘服务器间的本地相似性覆盖。由于减少远程专家调用与保持精确推理质量存在冲突,OrderMoE进一步提出一种质量感知且轨迹感知的运行时服务器-专家选择算法,决定令牌是否调用远程目标专家或使用可行的本地替代专家。在真实分布式边缘测试平台上进行的实验表明,OrderMoE显著降低了平均延迟、尾部延迟、跨服务器流量及远程专家调用比例,仅引入微小且可控的推理质量下降。

原文摘要 · Abstract (English)

Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it remains challenging to deploy MoE inference over resource-constrained and bandwidth-limited edge infrastructures. Existing distributed MoE serving methods mainly rely on exact expert placement, caching, replication, or communication scheduling, while overlooking the functional similarity among experts, which provides an opportunity to reduce cross-server token transmission. Therefore, this paper introduces a similarity-aware expert allocation and distributed deployment framework, dubbed OrderMoE, which aims to accelerate edge MoE inference while balancing inference latency, communication overhead, server workload, and inference quality. OrderMoE first constructs an expert similarity model based on router-induced logits representations and partitions experts in each MoE layer into multiple similarity groups. Then, it develops a similarity-aware expert grouping and deployment strategy to improve local similarity coverage across edge servers. Since reducing remote expert invocation and preserving exact inference quality are conflicting objectives, OrderMoE further designs a quality-aware and trajectory-aware runtime server-expert selection algorithm to decide whether a token should invoke its remote target expert or use a feasible local substitute expert. Experimental results on a real distributed edge testbed show that OrderMoE significantly reduces average latency, tail latency, cross-server traffic, and remote expert invocation ratio, while introducing only small and controllable inference quality degradation.

MoE边缘计算专家系统分布式推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。