PASCAL通过感知推理阶段，显著降低大模型响应延迟。
PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
- 区分推理与回答阶段，动态调度优化资源分配
- 尾部响应延迟降低72%，回答阶段服务等级达标
- 适合高并发推理类大模型部署场景
基于思维链（Chain-of-Thought, CoT）推理的大型语言模型在服务中面临新挑战：推理阶段延长用户可见输出时间，增加首次令牌生成时间（TTFT）。现有服务框架无法区分推理与回答阶段，导致在GPU内存受限时性能下降。我们提出PASCAL，一种感知阶段的调度算法，优先处理推理阶段以减少TTFT，同时在回答阶段采用可控抢占和令牌速率控制，保障用户体验质量（QoE）。其分层调度器结合实例级放置与实例内执行，并支持阶段边界动态迁移，有效平衡负载、减少干扰。在DeepSeek-R1-Distill-Qwen-32B等模型上的基准测试显示，PASCAL将尾部TTFT降低高达72%，同时维持回答阶段的服务水平目标（SLO）达成，凸显了阶段感知调度对推理型大模型部署的重要性。
原文摘要 · Abstract (English)
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output and inflate Time-To-First-Token (TTFT). Existing LLM serving frameworks fail to distinguish between reasoning and answering phases, leading to performance degradation under GPU memory constraints. We present PASCAL, a phase-aware scheduling algorithm that prioritizes reasoning to reduce TTFT while using controlled preemption and token pacing during answering to preserve Quality-of-Experience (QoE). Our hierarchical scheduler combines instance-level placement with intra-instance execution and enables dynamic migration at phase boundaries to balance load and reduce interference. Across benchmarks using DeepSeek-R1-Distill-Qwen-32B, PASCAL reduces tail TTFT by up to 72% while maintaining answering phase SLO attainment, demonstrating the importance of phase-aware scheduling for reasoning-based LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。