通过动态调用多个专家提升生成质量，不需训练即可提速降耗。
MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
- 在推理时随机选取多个专家并行计算，再融合输出结果。
- 70亿参数模型性能逼近105亿模型，计算量减少30%。
- 适合追求高效高质推理的部署场景，无需修改模型参数。
大语言模型的生成质量常通过推理阶段的序列级扩展方法（如思维链）提升。本文提出超并行扩展，一种在词元级别提升预测质量的互补框架。该方法为单个词元生成多个输出候选，并进行聚合。我们在混合专家（MoE）模型中实现此思想，称为专家名单（RoE）。RoE是一种无需训练的推理算法，可将单个MoE变为动态集成的MoE集合。通过向专家路由机制引入可控随机性，使每个词元能采样多个多样化专家，并融合其输出以获得更准确的最终预测。为降低计算开销，我们设计了高效的批处理策略和专用的键值缓存机制，显著减少计算与内存消耗。例如，RoE使一个70亿参数的MoE模型达到105亿参数模型的性能，同时推理计算量减少30%。这些提升无需对模型参数进行任何微调。
原文摘要 · Abstract (English)
The generation quality of large language models (LLMs) is often improved by utilizing inference-time sequence-level scaling methods (e.g., Chain-of-Thought). We introduce hyper-parallel scaling, a complementary framework that improves prediction quality at the token level. Hyper-parallel scaling computes and aggregates multiple output proposals for a single token from the model. We implement this concept in Mixture-of-Experts (MoE) models, which we refer to as Roster of Experts (RoE). RoE is a training-free inference algorithm that turns a single MoE into a dynamic ensemble of MoEs. RoE injects controlled stochasticity into the expert routing mechanism, enabling it to sample multiple diverse experts for each token and aggregate their outputs for a more accurate final prediction. To overcome the computational cost, we introduce an efficient batching strategy and a specialized KV-caching mechanism that minimizes compute and memory overhead. For example, RoE enables a 7B MoE model to match the performance of a 10.5B MoE model while using 30% less compute for inference. These gains are achieved without any fine-tuning of model parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。