提出可预测大模型推理能耗峰值的分析模型,优化生成效率
SweetSpot: An Analytical Model for Predicting Energy Efficiency of LLM Inference
- 基于Transformer计算与访存复杂度构建能量效率模型
- 实测均方百分比误差仅1.79%,发现输入短中等、输出中长时能效最优
- 适用于生产系统中动态截断与自适应生成策略设计
大语言模型(LLM)推理是现代AI应用的核心,主导全球数据中心负载,预测其能耗至关重要。现有方法将能耗简单建模为输入输出序列长度的线性函数,但通过分析Transformer的自回归结构,我们揭示了能耗与序列长度之间存在根本非线性关系,并发现生成能耗最小化的“甜点”区域:短至中等输入搭配中等输出时能效最高,而长输入或极短输出会导致效率急剧下降。为此,我们提出SweetSpot,一个从Transformer计算与内存访问复杂度推导出的解析模型,可精确刻画能耗效率随输入输出长度变化的曲线。我们在NVIDIA H100 GPU上使用TensorRT-LLM,对1B至9B参数量的OPT、LLaMA、Gemma、Falcon、Qwen2和Granite等多种模型进行测试,输入输出长度覆盖64至4096个标记,平均绝对百分比误差(MAPE)仅为1.79%。结果表明,将序列长度对齐至能效甜点区,可使能耗降低最多达33.41倍,支持生产环境中智能截断、摘要与自适应生成策略的部署。
原文摘要 · Abstract (English)
Large Language Models (LLMs) inference is central to modern AI applications, dominating worldwide datacenter workloads, making it critical to predict its energy footprint. Existing approaches estimate energy consumption as a simple linear function of input and output sequence. However, by analyzing the autoregressive structure of Transformers, which implies a fundamentally non-linear relationship between input and output sequence lengths and energy consumption, we demonstrate the existence of a generation energy minima. Peak efficiency occurs with short-to-moderate inputs and medium-length outputs, while efficiency drops sharply for long inputs or very short outputs. Consequently, we propose SweetSpot, an analytical model derived from the computational and memory-access complexity of the Transformer architecture, which accurately characterizes the efficiency curve as a function of input and output lengths. To assess accuracy, we measure energy consumption using TensorRT-LLM on NVIDIA H100 GPUs across a diverse set of LLMs ranging from 1B to 9B parameters, including OPT, LLaMA, Gemma, Falcon, Qwen2, and Granite. We test input and output lengths from 64 to 4096 tokens and achieve a mean MAPE of 1.79%. Our results show that aligning sequence lengths with these efficiency "sweet spots" reduce energy usage, up to 33.41x, enabling informed truncation, summarization, and adaptive generation strategies in production systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。