arXiv:2602.07616cs.LGcs.AI2026-02被引 1

通过相似性重路由减少MoE模型批处理时的专家激活,提升推理速度。

SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models

  • 根据输入相似性动态重路由令牌,减少次要专家调用。
  • 在复杂推理任务上实现最高2.0倍加速,质量损失极小。
  • 无需静态剪枝,适配vLLM,部署简单,适合高并发场景。

混合专家(MoE)架构通过稀疏激活实现比密集大模型更快的训练与推理。但在生产服务中,为提升硬件效率需进行批处理推理,这可能导致过多专家被激活,从而拖慢内存受限的解码阶段。针对批处理与专家稀疏性之间的根本矛盾,我们提出基于相似性的专家重路由方法SERE,通过将次级专家的令牌动态重路由至最相似的主专家,以自适应方式减少活跃专家数量。同时利用相似性模式识别并保留关键专家,防止能力损失。SERE不依赖静态专家剪枝或合并,而是基于批级冗余实现动态跳过。此外,我们提供了高效的自定义CUDA内核,可在vLLM中仅通过一行代码修改即可集成。在多个复杂推理基准上的实验表明,SERE在几乎无质量损失的情况下实现最高2.0倍加速,为大规模低成本、低延迟的MoE部署提供实用解决方案。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) architectures employ sparse activation to deliver faster training and inference with higher accuracy than dense LLMs. However, in production serving, MoE models require batch inference to optimize hardware efficiency, which may cause excessive expert activation and thus slow the memory-bound decoding stage. To address the fundamental tension between batch decoding and expert sparsity, we present SERE, a Similarity-based Expert Re-routing method for Efficient batch decoding in MoE models. SERE dynamically reduces the number of active experts in an input-aware manner by re-routing tokens from secondary experts to their most similar primary counterparts. It also leverages similarity patterns to identify and preserve critical experts, thereby preventing capability loss. Notably, SERE avoids static expert pruning or merging, instead enabling dynamic expert skipping based on batch-level expert redundancy. Additionally, we provide an efficient custom CUDA kernel for SERE, enabling plug-and-play use in vLLM with only a single-line code change. Extensive experiments on various complex reasoning benchmarks demonstrate that SERE achieves up to 2.0x speedup with minimal quality loss, providing a practical solution for cost-efficient and latency-sensitive large-scale MoE deployment. Code implementation of SERE can be found in https://github.com/JL-Cheng/SERE.

MoE推理加速专家路由vLLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。