通过动态调频与智能路由，让大模型推理省电36%还保延迟达标。
VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
- 按推理阶段动态调节GPU频率，每轮迭代自适应选择能效最佳点。
- 实测在真实负载下端到端能耗降低36.3%，且满足严格延迟要求。
- 适合关注绿色AI部署、算力成本优化的研究者与工程师。
大型语言模型（LLM）推理的能耗正成为可持续、可扩展部署的主要障碍。尽管现代服务架构呈现出预填充（prefill）与解码（decode）阶段的显著差异，现有系统未能利用这些阶段特性实现节能服务。本文提出VoltanaLLM，首个针对预填充-解码（P/D）解耦式LLM服务中能量浪费问题的系统。基于控制理论视角，该系统分离两个调控维度：实例级运行点选择（每轮迭代的GPU频率）与系统级状态空间请求路由。我们实证发现LLM推理存在典型的U型能效-频率曲线，其“甜点”位置依赖于阶段行为和负载。VoltanaLLM结合阶段特异性、迭代级频率选择（由轻量级在线预测器驱动），以及解码状态空间引导的路由器，有效规避了架构粒度带来的效率损失，全程满足预期服务等级目标（SLO）。我们在SGLang上实现并评估了多种模型与真实工作负载。结果表明，相比静态最大频率基线，其端到端能耗最高降低36.3%，同时保持高SLO达成率，并具备对新型GPU的泛化能力。这表明，通过感知阶段特性的迭代级频率调节与架构感知路由，可实现可持续的LLM服务。源代码已开源：https://github.com/Supercomputing-System-AI-Lab/VoltanaLLM。
原文摘要 · Abstract (English)
The energy cost of Large Language Model (LLM) inference is rapidly becoming a barrier to sustainable and scalable deployment. Although modern serving architectures expose distinct prefill and decode behaviors, existing systems fail to exploit these phase differences for energy-efficient serving under strict latency SLOs. This paper introduces VoltanaLLM, the first system that explicitly targets and reduces the energy bloat in modern prefill-decode (P/D) disaggregated LLM serving. Guided by a control-theory perspective, VoltanaLLM separates two levers: per-instance operating-point selection (GPU frequency per iteration) and system-level state-space routing of requests. We empirically observe that LLM inference exhibits a U-shaped energy-frequency curve creating "sweet spots" that depend on phase behavior and load. VoltanaLLM exploits this by combining phase-specific, iteration-level frequency selection driven by a lightweight, online-adaptive latency predictor, with a decode state-space guided router that avoids architectural granularity-induced inefficiencies, all while meeting desired SLOs. We implement VoltanaLLM using SGLang and evaluate it across multiple models and real-world workloads. Our results show VoltanaLLM reduces end-to-end energy by up to 36.3% versus a static max-frequency baseline while maintaining high SLO attainment, and generalizes to newer GPUs. These results point to sustainable LLM serving via phase-aware, iteration-level frequency selection coupled with architecture-aware routing. Source code is available in https://github.com/Supercomputing-System-AI-Lab/VoltanaLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。