实测发现扩散语言模型服务需按去噪步级优化,而非传统逐词生成。
Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
- 发现请求难度分11个离散去噪步级,无法提前预测。
- 单请求仅24%为GPU计算,其余为调度开销,批量可提升16倍吞吐。
- 适合追求高并发、低延迟文本生成的系统设计者参考。
掩码扩散语言模型(dLLMs)理论上可比自回归(AR)模型更快生成文本,因能同时去噪多个词元。现有系统虽开始构建dLLM服务基础设施,但尚未在真实并发负载下测量其行为。缺乏此类数据可能导致沿用AR模型的服务假设,不适用于dLLMs。本文使用LLaDA-8B-Instruct搭配D2F LoRA适配器,在单张NVIDIA H200 GPU上评估GSM8K和HumanEval。结果表明:第一,请求难度(所需去噪步数)为离散而非连续,分为11个固定步级(178至29,000步),测试信号均无法在生成前预测(最佳R²=0.150);第二,生成预算低于320词元的基准会低估服务方差,因请求被截断前延迟差异未显现;第三,单请求中仅有24%为GPU计算时间,其余为CPU调度开销;批量处理主要通过摊销该开销提升性能:每步共享一次前向传播,批大小16时吞吐提升16.0倍。我们还论证输出质量不应随批大小下降,并实测单请求下获得74%~76% GSM8K准确率。最后,提出泊松到达下固定填充同步批处理的批超时规则。结果表明,服务扩散模型需在每个去噪步层面实现并行,与AR模型在准入/驱逐与共享前向传播间的交互机制不同。
原文摘要 · Abstract (English)
Masked diffusion language models (dLLMs) can in principle generate text faster than autoregressive (AR) models, since they denoise many tokens at once. Recent systems have begun building serving infrastructure for dLLMs, but none first measure how these models behave under real, concurrent serving load. Serving systems built without this grounding risk carrying over assumptions from AR serving that may not hold for dLLMs. We characterize dLLM serving to close this gap, using LLaDA-8B-Instruct with a D2F (Discrete Diffusion Forcing) LoRA adapter on a single NVIDIA H200 GPU, evaluated on GSM8K and HumanEval. We report three findings. First, request difficulty, the number of denoising steps a request needs, is discrete rather than continuous: requests fall into 11 fixed step-count levels (178 + 29k), and no signal we test predicts the level before generation starts (best R2 = 0.150). Second, benchmarks with short generation budgets below 320 tokens understate serving variance, since requests are cut off before the latency spread appears. Third, only 24% of single-request wall-clock time is GPU computation; the rest is CPU-side dispatch overhead. Batching mainly helps by amortizing this overhead: sharing one forward pass per denoising step improves throughput by 16.0x at batch size 16 over a per-request-dispatch baseline. We also argue structurally that output quality should not degrade with batch size, stating three assumptions this rests on; we measure 74 to 76% GSM8K accuracy at single-request scale. Finally, we derive a batch-timeout rule for fixed-fill synchronized batching under Poisson arrivals. Together, these results show that serving diffusion language models needs parallelism at the level of each denoising step, which differs from AR serving in how admission and eviction interact with an already shared forward pass.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。