arXiv:2608.01891cs.DCcs.AI2026-08

通过分离注意力与前馈网络并动态调频,显著降低大模型推理能耗。

Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

论文配图:Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
图 1 · 摘自论文原文
  • 分离注意力与前馈网络,按需独立调节运行频率。
  • 实测每令牌能耗降低49%,且满足响应延迟要求。
  • 适合高并发、低功耗的生产级大模型服务场景。

大型语言模型(LLM)服务面临严格的性能保障需求,常需将GPU运行在最高频率,导致能耗上升。现有节能方法仅在请求或推理阶段调整频率,忽视了注意力层(Attention)与前馈网络(FFN)在频率敏感性上的差异。我们发现,注意力与前馈网络的能效最优频率不同,且随推理阶段、负载和系统配置变化。但运行时波动及独立频率控制带来巨大搜索空间和通信开销。为此,我们提出AFlex框架,联合优化资源分配与GPU频率调控,支持拆分后的注意力与前馈网络异步执行。AFlex引入全局调度器与局部操作级动态电压频率调节(DVFS)控制器,实现精准频率控制;并通过交错式处理管道与动态微批次深度、自适应请求批处理减少流水线空洞。我们在SGLang中实现AFlex,基于NVIDIA A800 GPU,使用Qwen3-32B与Mixtral-8×7B模型,在生产环境对话与编码任务数据集上评估。结果表明,相比当前最优的拆分式服务,AFlex在满足TTFT与TPOT服务等级目标的前提下,每令牌能耗降低高达49%;相比频率调节系统,亦降低48%。

原文摘要 · Abstract (English)

Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing energy consumption. Existing energy-management approaches adapt GPU frequencies only at the request or inference-phase level, overlooking operator-level differences in frequency sensitivity between Attention and feed-forward networks (FFNs). We find that the energy-optimal frequencies of Attention and FFN (A/F) differ and vary with the inference phase, workload, and system configurations. However, runtime variability and independent A/F frequency control create a large search space and high communication overhead. To address these challenges, we present AFlex, a framework that jointly optimizes resource provisioning and GPU frequency scaling for disaggregated A/F serving. AFlex introduces a global scheduler and a local operator-level dynamic voltage and frequency scaling (DVFS) controller to determine A/F resource allocations and frequencies. It further introduces an interleaved A/F pipeline with dynamic microbatch depth and adaptive request batching to reduce pipeline bubbles. We implement AFlex in SGLang and evaluate it on NVIDIA A800 GPUs using Qwen3-32B and Mixtral-8$\times$7B under production Conversation and Coding traces. \AFlex reduces energy per token by up to 49\% over state-of-the-art disaggregated serving and 48\% over frequency-scaling systems while satisfying TTFT and TPOT SLOs.

大模型推理能耗优化动态调频异构计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。