提出一套精准分配预填充与解码资源的方法，保障大模型推理性能和响应速度。
SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
- 结合理论建模与实测数据，推导预填充和解码阶段所需资源数。
- 在满足请求长度与延迟要求下，实现吞吐量最优的资源配置。
- 适合需要稳定服务等级的大模型部署团队使用。
预填充-解码（Prefill-Decode, P/D）解耦已成为大语言模型（LLM）推理的主流优化策略。然而，当前尚无成熟方法确定P/D硬件资源的最佳数量，尤其在总吞吐量、服务等级目标（SLO）、请求输入输出长度等约束条件下。为此，本文提出一种融合理论建模与实测验证的混合方法：首先建立基于总吞吐量需求、请求输入输出长度以及预填充和解码吞吐量的理论模型；其次，利用M/M/1队列模型分析预填充过程，结合基准测试的最大预填充吞吐量与首次生成时间（TTFT），推导实际预填充吞吐量；对于解码阶段，通过确定满足每输出令牌耗时（TPOT）要求的批处理大小，并经实测获取对应解码吞吐量。实验结果表明，该方法能准确预测真实场景下大模型推理所需的最优资源分配方案。
原文摘要 · Abstract (English)
Prefill-Decode (P/D) disaggregation has emerged as a widely adopted optimization strategy for Large Language Model (LLM) inference. However, there currently exists no well-established methodology for determining the optimal number of P/D hardware resources, subject to constraints on total throughput, service level objectives (SLOs), and request characteristics - specifically input and output lengths. To address this gap, we propose a hybrid approach that combines theoretical modeling with empirical benchmarking. First, we present a theoretical model for calculating P/D resource counts, which is based on total throughput requirements, request input and output lengths, as well as prefill and decode throughput. Then, to obtain the actual prefill and decode throughput under SLO constraints, we model the prefill process using M/M/1 queuing theory, deriving the achieved prefill throughput from the benchmarked maximum prefill throughput and Time-To-First-Token (TTFT). For the decode phase, we determine the decode batch sizes that meet Time-Per-Output-Token (TPOT) requirements and obtain the corresponding decode throughput through empirical measurements. Our experimental results demonstrate that the proposed method can accurately predict optimal P/D resource allocation in real-world LLM inference scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。