首次系统分析大模型多请求工作流中的性能-能耗权衡,揭示关键优化策略。
Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflows
- 构建四类典型多请求工作流,模拟真实应用模式。
- 批处理大小影响最大,但效果因任务类型而异,共享提示场景收益显著。
- vLLM引擎效率更高,Parrot调度在严苛功耗下更省电,适配不同需求。
大型语言模型(LLMs)正广泛应用于文档摘要、基于搜索的智能助手和多智能体编程等多请求工作流中。这类工作流虽增强功能,却也加剧了推理时延与能耗。现有研究或聚焦单请求评估,或仅关注系统性能,忽视了多请求间的依赖关系与交互特性,且对跨请求能耗缺乏深入探索。本文首次系统刻画多请求场景下大模型的性能-能耗权衡。我们设计了四类代表性工作负载,涵盖顺序、交互、代理及复合模式。基于NVIDIA A100测试平台与vLLM、Parrot先进服务系统,分析关键能效控制参数对延迟、吞吐量及组件级能耗的影响。结果表明:批处理大小是最具影响力的调控因子,但其效果依赖于工作负载类型——在共享提示场景中收益明显,而在顺序摘要中无效,多智能体编码中仅部分有效。GPU功耗限制可带来适度且可预测的节能,输出长度则导致能量线性增长,效率提升有限。此外,vLLM的引擎级优化在解码密集型任务中维持更高GPU利用率与效率;而Parrot的流程感知调度在严格功耗约束下实现更低能耗。这些发现为开发者与系统运维人员提供了可落地的性能与能耗协同优化指导。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used in applications forming multi-request workflows like document summarization, search-based copilots, and multi-agent programming. While these workflows unlock richer functionality, they also amplify latency and energy demand during inference. Existing measurement and benchmarking efforts either focus on assessing LLM inference systems or consider single-request evaluations, overlooking workflow dependencies and cross-request interactions unique to multi-request workflows. Moreover, the energy usage of such interdependent LLM calls remains underexplored. To address these gaps, this paper presents the first systematic characterization of performance-energy trade-offs in multi-request LLM inference. We develop four representative workloads capturing sequential, interactive, agentic, and composite patterns common in modern deployments. Using an NVIDIA A100 testbed with state-of-the-art serving systems (vLLM and Parrot), we analyze how key energy knobs affect latency, throughput, and component-level energy use. Our findings reveal batch size as the most impactful lever, though benefits are workload dependent. While optimal batching benefits workloads with large shared prompts, it is ineffective for sequential summarization and only partially effective for multi-agent coding. GPU power capping provides modest but predictable savings, while output length induces linear energy scaling with limited efficiency gains. We further show that engine-level optimizations in vLLM maintain higher GPU utilization and efficiency, especially for decode-heavy workloads, while Parrot's workflow-aware scheduling achieves lower energy consumption under strict power constraints. These findings offer actionable guidelines for developers and system operators designing performance- and energy-aware LLM serving systems in emerging multi-request workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。