研究大模型推理的能耗与性能权衡，发现降频可省42%能但延迟仅增1-6%。
Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling
- 用轻量语义特征预测推理难度，比输入长度更准。
- 解码阶段占总时间77%-91%，调低显卡频率能省42%电，延迟仅升1-6%。
- 适合关注节能推理系统、模型部署优化的研究者和工程师。
大模型推理在不同查询和执行阶段表现出显著差异，但配置常被统一应用。本文通过测量分析，在GPU动态电压频率调节（DVFS）下对五种Decoder-only LLM（1B-32B参数）在四个NLP基准上的工作负载异质性与能效行为进行了表征。结果表明，轻量语义特征比输入长度更能预测推理难度，44.5%的查询在不同模型尺寸下可达到相近质量。硬件层面，解码阶段占据77%-91%的推理时间，且对显卡频率变化不敏感。将显卡频率从2842 MHz降至180 MHz，平均实现42%的能耗降低，延迟仅增加1%-6%。此外，我们通过上限分析展示了结合工作负载感知模型选择与阶段感知DVFS的潜在收益，为未来能效优化的大模型推理系统提供依据。
原文摘要 · Abstract (English)
LLM inference exhibits substantial variability across queries and execution phases, yet inference configurations are often applied uniformly. We present a measurement-driven characterization of workload heterogeneity and energy-performance behavior of LLM inference under GPU dynamic voltage and frequency scaling (DVFS). We evaluate five decoder-only LLMs (1B-32B parameters) across four NLP benchmarks using a controlled offline setup. We show that lightweight semantic features predict inference difficulty better than input length, with 44.5% of queries achieving comparable quality across model sizes. At the hardware level, the decode phase dominates inference time (77-91%) and is largely insensitive to GPU frequency. Consequently, reducing GPU frequency from 2842 MHz to 180 MHz achieves an average of 42% energy savings with only a 1-6% latency increase. We further provide a use case with an upper-bound analysis of the potential benefits of combining workload-aware model selection with phase-aware DVFS, motivating future energy-efficient LLM inference systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。