为扩散语言模型设计高吞吐低延迟的部署系统
DiLaServe: High SLO Attainment Serving for Diffusion Language Models
- 通过动态调整置信度阈值实现任务调度与负载自适应
- 在真实负载下提升服务达标率56.6个百分点，延迟降低46%
- 适用于需要高并发低延迟生成的AI应用开发
扩散语言模型（DLMs）作为传统自回归模型的替代方案，通过在每一步去噪中并行生成多个词元，实现了更高的推理吞吐量且保持良好质量。然而，在服务系统中同时实现高吞吐与满足延迟SLO，需应对由DLM特性带来的挑战：基于置信度的去噪引发的速度-质量权衡、负载波动下的模型实例并行度选择，以及引入非均匀每步开销的近似KV缓存机制。为此，我们提出DiLaServe，一个面向DLM的集群级服务系统。该系统通过置信度阈值调节实现截止时间感知调度与自适应负载控制，并通过求解质量感知优化问题动态重构集群，显式建模近似KV缓存带来的步骤级异构性。在多个基准测试与真实负载轨迹上，DiLaServe将SLO达标率提升最高达56.6个百分点，端到端请求延迟降低最多46%，同时精度损失小于1%。
原文摘要 · Abstract (English)
Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive language models. By generating multiple tokens in parallel during each denoising step, they offer higher inference throughput while maintaining competitive quality. However, realizing these throughput gains while meeting latency SLOs in a serving system requires addressing challenges introduced by DLMs' unique characteristics. These include navigating the speed-quality tradeoff created by confidence-based denoising, choosing appropriate parallelization levels across model instances under fluctuating load, and coordinating approximate KV caching mechanisms that introduce non-uniform per-step costs. To address these challenges, we present DiLaServe, a cluster-level serving system for DLMs. DiLaServe enables deadline-aware scheduling and adaptive load control through confidence-threshold adjustment, and dynamically reconfigures the cluster by solving a quality-aware optimization problem, while explicitly modeling the step-level heterogeneity introduced by approximate KV caching. Across multiple benchmarks and real-world traces, DiLaServe improves SLO attainment by up to 56.6 percentage points and reduces end-to-end request latency by up to 46\% while incurring less than 1\% accuracy drop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。