XShare通过动态共享专家提升MoE模型推理速度，降低显存占用。
XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
- 基于批量感知优化，动态选择最优专家组合。
- 批量推理下专家激活减少30%，峰值显存降为1/3。
- 无需重训练，适用于异构请求的快速生成场景。
Mixture-of-Experts（MoE）架构被广泛用于高效扩展大语言模型。然而在生产推理中，请求批处理和推测解码显著增加了专家激活，削弱了效率优势。本文将批处理感知的专家选择建模为模块化优化问题，设计了适配不同部署场景的高效贪心算法。提出的XShare方法无需重训练，能根据每批请求动态最大化所选专家的门控分数总和。在标准批处理下，专家激活最多减少30%；在专家并行部署中，峰值GPU负载降低至原来的1/3；在推测解码场景中，通过分层、相关性感知的专家选择，即使请求来自异构数据集，也能实现最高14%的吞吐量提升。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding significantly amplify expert activation, eroding these efficiency benefits. We address this issue by modeling batch-aware expert selection as a modular optimization problem and designing efficient greedy algorithms for different deployment settings. The proposed method, namely XShare, requires no retraining and dynamically adapts to each batch by maximizing the total gating score of selected experts. It reduces expert activation by up to 30% under standard batching, cuts peak GPU load by up to 3x in expert-parallel deployments, and achieves up to 14% throughput gains in speculative decoding via hierarchical, correlation-aware expert selection even if requests in a batch drawn from heterogeneous datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。