根据路由分布动态选最优内核，提升MoE推理速度1.4倍
RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts
- 基于硬件特性分析性能区域，预测最佳内核配置
- 用10-24分钟预采样，实现99.07%最优解覆盖率
- 无需修改代码，适配多种模型和部署系统
MoE推理的最优内核配置取决于批大小与专家路由分布，但现有系统仅依据批大小调度，导致内核吞吐率损失10%-70%。我们提出RaMP，一种路由感知调度框架。通过性能区域分析，仅基于硬件常数即可准确预测8种已测架构（含3种未见）的优化收益。采用四参数波浪代价模型，从运行时专家直方图中选取最快配置，相比穷举搜索平均仅0.93%误差，且每模型仅需10-24分钟一次性的预标定。该模型仅依赖CTA网格几何，具备内核无关性：应用于Alpha-MoE，无需源码修改即达1.14倍加速。结合定制的CuTe DSL内核（支持134-268种多态配置），RaMP相较静态调度实现1.22倍内核加速，在vLLM服务中较Triton快1.30倍、较DeepGEMM快1.41倍、较FlashInfer CUTLASS快1.13倍。
原文摘要 · Abstract (English)
The optimal kernel configuration for Mixture-of-Experts (MoE) inference depends on both batch size and the expert routing distribution, yet production systems dispatch from batch size alone, leaving 10-70% of kernel throughput unrealized. We present RaMP, a routing-aware dispatch framework. A performance-region analysis derives, from hardware constants alone, when each optimization helps, correctly predicting all 8 tested architectures, including 3 unseen. A four-parameter wave cost model selects the fastest configuration from the runtime expert histogram, achieving 0.93% mean regret versus exhaustive search, fitted from just 10-24 minutes of one-time profiling per model. Because the model depends only on CTA grid geometry, it is kernel-agnostic: applied to Alpha-MoE, it delivers 1.14x with no source modification. Paired with a co-designed CuTe DSL kernel exposing 134-268 polymorphic configurations, RaMP delivers 1.22x kernel speedup over static dispatch and 1.30x end-to-end speedup in vLLM serving over Triton, 1.41x over DeepGEMM, and 1.13x over FlashInfer CUTLASS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。