通过动态调节各阶段频率与缓存,实现高负载下的稳定大模型推理性能。
PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response

- 按预填充、思考、生成三阶段分别调控频率和缓存,精准分配电力
- 在30%功率限制下保持78.3%吞吐,优于最佳基线1.64倍
- 适合电网紧急调度场景,保障关键任务延迟不超1.3倍
AI推理集群正受瞬时功率限制,而非总能耗:电网运营商要求需求响应,施加随时间变化的功率上限。现有大模型服务系统仅优化静态能耗或固定优先级丢弃;一旦功率上限变动,吞吐量急剧下降。大模型流水线并非均匀负载:计算密集型预填充阶段吞吐随GPU频率近乎线性下降,内存密集型回答解码可维持至0.57倍标称频率,而推理的思考阶段将键值缓存容量与调度耦合——因此应将功率约束引导至每瓦性能损失最小处。PowerSlider引入新弹性SLO协议,将用户容忍度转为优化约束,通过预填充-思考-回答拆分暴露各阶段频率与KV缓存控制,并使用7.7毫秒内重求解的KKT在线求解器应对每次功率变化,辅以统一熔断机制,在静态功耗耗尽时断电保护实例。在SGLang与生产流量数据上,该系统在功率降低30%时仍保持78.3%在线吞吐,优于五种基线中最佳者(1.64倍),延迟关键尾部保持在1.3倍标称水平(基线2.3–6倍,最高达12倍),并在再现加州独立系统运营商(CAISO)电网紧急日时,实现均值92%吞吐,最低降至0.41倍(基线均低于7%)。
原文摘要 · Abstract (English)
AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power caps. Existing LLM serving systems optimize a static energy objective or shed fixed priority tiers under load; either way, goodput collapses when the power envelope moves. An LLM pipeline is not a uniform load: compute-bound prefill loses throughput almost linearly with GPU frequency, memory-bound answer decode sustains it down to $0.57\times$ nominal, and reasoning's thinking phase couples KV-cache capacity to scheduling -- so a cap should be steered to where each watt costs the least performance. PowerSlider does so with a new Flex SLO contract that turns bounded user slack into an optimization constraint, prefill--think--answer disaggregation exposing per-stage frequency and KV control, and a Karush--Kuhn--Tucker (KKT) online solver re-solving within 7.7 ms of every cap change, backed by a consolidated fail-safe that power-gates drained instances when DVFS bottoms out on static power. On SGLang with production traces, \sys{} sustains 78.3\% online goodput at a 30\% cap reduction versus 47.6\% for the best of five baselines ($1.64\times$), holds latency-critical tails within $1.3\times$ of nominal (baselines: $2.3$--$6\times$, up to $12\times$), and delivers 92\% mean goodput through a replayed CAISO grid-emergency day bottoming at $0.41\times$ (54\% at the trough; every baseline below 7\%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。