让大模型推理自动调功率，省电26.3%还更稳
PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
- 把GPU功耗当可调参数，联动批大小等软件设置
- 多卡下能效提升26.3%，功耗约束下服务降级减少7倍
- 无需改模型或接口，适合部署在云数据中心
大语言模型推理已成为现代数据中心的主要负载，带来显著的GPU利用率和能耗。现有系统虽通过批处理、调度与并行优化吞吐与延迟，但普遍将GPU功耗视为静态限制，而非可控资源。本文提出面向LLM服务的功耗感知运行时PALS，将GPU功耗上限作为核心控制变量，与批大小等软件参数协同优化。系统结合轻量级离线功耗-性能模型与反馈驱动控制器，动态选择满足吞吐目标的同时最大化能效的配置。我们在vLLM框架中实现PALS，无需模型重训练或API变更。在多GPU系统及稠密模型与混合专家（MoE）模型上，PALS能提升能效最高达26.3%，在功耗约束下服务质量违规降低4至7倍，并支持动态功耗预算跟踪。结果表明，将功耗控制直接集成到LLM推理运行时具有潜力，可实现按需供电与电网互动的AI系统。
原文摘要 · Abstract (English)
Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。