arXiv:2605.20179cs.CL2026-05被引 1

通过动态调度专家模块,实现扩散模型推理的高效无损加速。

TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload

论文配图:TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload
图 1 · 摘自论文原文
  • 基于专家激活的时序稳定性,按区间刷新专家分配以减少输入输出开销。
  • 在单卡系统上使LLaDA2.0-mini和LLaDA2.0-flash的吞吐分别提升1.4倍和1.5倍。
  • 无需训练即可部署,适合资源受限设备上的扩散语言模型推理场景。

扩散大语言模型(dLLMs)作为自回归模型的有力替代,通过块级并行解码实现了更好的硬件利用率和双向上下文建模。然而,随着dLLMs采用混合专家(MoE)架构持续扩展,其在资源受限设备上的部署仍面临挑战。现有基于自回归的方法往往导致极高的输入输出开销或显著的计算瓶颈。本文提出TIDE,一种新型高效的推理系统,利用扩散过程中块内专家激活的时序稳定性,引入基于区间的专家刷新策略,以输入输出感知的方式更新专家放置。为确保最优性能,将推理调度建模为数学规划问题,求解最小化输入输出流量与CPU计算量的最佳区间。最重要的是,TIDE是无损优化,无需模型训练,为dLLM推理提供“免费加速”。在单GPU-CPU系统中,TIDE在LLaDA2.0-mini和LLaDA2.0-flash模型上分别实现最高1.4×和1.5×的吞吐提升。

原文摘要 · Abstract (English)

Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive (AR) models, offering better hardware utilization and bidirectional context through parallel block-level decoding. However, as dLLMs continue to scale up with mixture-of-experts (MoE) architectures, their deployment on resource-constrained devices remains an open challenge. Existing AR-based methods often incur either prohibitive I/O overhead or significant compute bottlenecks. In this work, we propose TIDE, a novel resource-efficient inference system that leverages the temporal stability of expert activations during the diffusion process within the block. Specifically, we leverage the temporal stability of expert activations during the diffusion process within the block and introduce an interval-based expert refresh strategy that updates the expert placement in an I/O-aware fashion. To ensure optimal performance, we formulate the inference scheduling as a mathematical programming problem, solving for the optimal interval that minimizes I/O traffic and CPU computation. Most importantly, TIDE is a lossless optimization that requires no model training, providing a "free lunch" acceleration for dLLM inference. In a single GPU-CPU system, we demonstrate that TIDE achieves up to 1.4$\times$ and 1.5$\times$ throughput improvements over prior baselines on LLaDA2.0-mini and LLaDA2.0-flash models, respectively.

扩散模型MoE推理优化无损加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。