arXiv:2607.24787cs.AIcs.LG2026-07被引 1

提出轻量级预取框架,让稀疏专家模型更快加载专家,节省内存。

SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models

论文配图:SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models
图 1 · 摘自论文原文
  • 用轻量适配器提前预测下一阶段专家,异步预加载避免等待
  • 在10个测试场景中9次实现最高专家召回率,参数量少于基线方法
  • 适合移动端和存储受限设备部署,实测推理速度提升20%

稀疏混合专家(MoE)模型通过条件性激活专家扩展基础模型容量,但其完整专家池在加速器内存有限时难以部署。尽管专家卸载通过将非活跃专家移至主机内存或存储缓解内存压力,却引入依赖路由的传输瓶颈:所需专家仅在原生 top-K 路由后才知悉,导致路由、专家加载与执行串行进行。为此,我们提出 SpecPrefetch,一种针对离线 MoE 推理的参数高效预取框架。SpecPrefetch 使用共享轻量适配器仅对下一层专家候选进行预测,用于异步传输,而冻结的原生路由器仍决定最终执行的专家。通过分离传输预测与执行路由,SpecPrefetch 在不改变预训练路由语义的前提下降低暴露的专家加载延迟,预测错误仅影响传输效率而非模型输出。此外,窗口感知调度器在缓存与带宽约束下优先可行传输。在 Qwen3-VL-30B-A3B 与 DeepSeek-VL2-Tiny 上,SpecPrefetch 在 10 个模型-基准组合中 9 次达到最高平均专家召回率,且可训练参数显著少于学习型预测基线。在 Snapdragon 8 Elite 设备上,相比计算优化的卸载运行时,解码吞吐率提升高达 20%,验证了其在存储受限场景下的实际价值。代码与模型权重见 https://github.com/wei390/SpecPrefetch。

原文摘要 · Abstract (English)

Sparse Mixture-of-Experts (MoE) models expand foundation model capacity through conditional expert activation, but their full expert pools remain difficult to deploy under limited accelerator memory. Although expert offloading alleviates memory pressure by moving inactive experts to host memory or storage, it introduces a routing-dependent transfer bottleneck: required experts are known only after native top-\(K\) routing, which serializes routing, expert loading, and expert execution during inference. To address this bottleneck, we propose SpecPrefetch, a parameter-efficient prefetching framework for offloaded MoE inference. SpecPrefetch uses a shared lightweight adapter to predict next-layer expert candidates only for asynchronous transfer, while the frozen native router still determines the final executed experts. By separating transfer prediction from execution routing, SpecPrefetch reduces exposed expert-loading latency without changing pretrained routing semantics, so prediction errors affect transfer efficiency rather than model outputs. In addition, a window-aware scheduler prioritizes feasible transfers under cache and bandwidth constraints. Across Qwen3-VL-30B-A3B and DeepSeek-VL2-Tiny, SpecPrefetch achieves the best average expert recall in 9 out of 10 model-benchmark settings with substantially fewer trainable parameters than learned predictor baselines. On a Snapdragon 8 Elite device, SpecPrefetch further improves decoding throughput by up to \(20\%\) over a compute-optimized offloading runtime, demonstrating practical benefits for storage-constrained MoE deployment. The code and model weights are available at https://github.com/wei390/SpecPrefetch.

MoE模型专家预取轻量级移动端部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。