arXiv:2608.00573cs.NIcs.CL2026-08

通过动态跳层与提前退出,显著降低边缘部署MoE模型的通信开销。

TrimMoE A communication aware and adaptive depth framework for distributed edge inference

  • 根据通信成本动态决定是否执行某层及后续层。
  • 实测平均延迟降低62.8%,跨服务器流量减少,质量损失控制在2%内。
  • 适合资源受限的分布式边缘推理场景,尤其适用于MoE大模型部署。

在分布式边缘服务器上服务混合专家(MoE)大语言模型时,跨服务器专家传输成为瓶颈。现有方法主要关注如何更快获取远程专家,而本文提出一种通信感知的自适应深度框架TrimMoE,核心思想是判断当前层及后续层是否需要执行。在线下阶段,固定主干网络,训练轻量级逐层退出头,校准各层重要性阈值,并基于跳层/退出感知的冗余收益分配专家副本;在线上阶段,采用前瞻式预测令牌移动,使深度缩减针对最昂贵的传输,同时通过两条反馈规则动态调整延迟-质量权重和退出阈值。理论上证明替换与跳层导致的性能退化不超过设定预算,且提前退出仅在经校准置信度门控下允许。在包含Switch-Base-8E、Qwen-MoE-A2.7B和Mixtral-8x7B的异构10服务器测试平台上,TrimMoE将平均延迟降低最高达62.8%,减少跨服务器流量与远程执行比例,在负载下保持高吞吐量,任务质量下降控制在2%以内。

原文摘要 · Abstract (English)

Serving Mixture-of-Experts (MoE) large language models across distributed edge servers is bottlenecked by the cross-server expert transmission. The existing approaches mainly focus on how to reach a remote expert faster. However, in this paper, we instead consider whether a given layer, and the layers after it, need to be executed at all. To this end, a communication-aware adaptive-depth framework is proposed in this paper, termed TrimMoE, which couples layer skipping and confidence-based early exit with substitute execution and server-expert selection under a unified quality budget. Specifically, in the offline stage, TrimMoE freezes the backbone, trains the lightweight per-layer exit heads, calibrates the per-layer importance thresholds, and allocates the expert replicas by a skip/exit-aware redundancy benefit. In the online stage, a transition-aware look-ahead anticipates the token movement, so that the depth reduction targets the costliest transmissions, and besides, two feedback rules adapt the delay-quality weights and the exit threshold. Moreover, we prove that the substitution-and-skipping proxy degradation never exceeds the configured budget, and that the early exit is admitted only under a calibrated confidence gate. On a heterogeneous 10-server testbed with Switch-Base-8E, Qwen-MoE-A2.7B, and Mixtral-8x7B, TrimMoE reduces the average latency by up to 62.8%, lowers the cross-server traffic and the remote-execution ratio, and sustains high throughput under load, while keeping the task-quality degradation within a 2% bound.

MoE边缘计算通信优化自适应推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。