通过块级调度提升扩散型大模型推理吞吐量
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
- 按块粒度即时清理完成请求，避免慢请求拖累整体
- 在五个基准上相比Fast-dLLM吞吐提升1.9至10.6倍
- 适合需要高并发推理的生成式AI服务场景
高效部署扩散型大语言模型（dLLMs）受限于收敛异质性：批量处理多个请求时，不同序列完成速度不一，导致快请求被迫等待慢请求，产生计算空洞和尾部延迟。我们提出BlockServe，一种连续批处理框架，结合块粒度调度——在块边界立即清除已完成请求——与混合状态执行，通过收集-散射索引将双缓存和并行解码扩展至异构批次。此外，一个计算感知的准入控制器通过基于令牌预算的重填机制扩大有效批处理容量。在Dream和LLaDA上，跨五个基准测试，BlockServe相比Fast-dLLM实现1.9至10.6倍的吞吐提升，且生成质量相当，确立了块粒度调度作为高吞吐离线dLLM推理的基础。
原文摘要 · Abstract (English)
Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency. We present BlockServe, a continuous batching framework that integrates block-grained scheduling -- immediately evicting completed requests at block boundaries -- with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing. Furthermore, a compute-aware admission controller expands effective batch capacity through token-budgeted refill. On Dream and LLaDA across five benchmarks, BlockServe achieves 1.9--10.6$\times$ throughput over Fast-dLLM with comparable generation quality, establishing block-grained scheduling as a foundation for high-throughput offline dLLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。