压缩大模型输出长度，能省25%到60%推理能耗。
Brevity is the soul of sustainability: Characterizing LLM response lengths
- 用提示工程控制输出长度和信息量，实现节能。
- 实测12个模型在5个数据集上响应过长，冗余率高。
- 适合关注大模型能效、部署优化的研究者与工程师。
大型语言模型（LLMs）推理过程消耗大量能源，因此提升推理效率至关重要。尽管已有多种优化方法，但输出压缩仍较少被研究。本文对12个解码器仅有的LLM在5个数据集上进行基准测试，发现其生成的响应远超必要长度。我们进一步对响应质量进行全面评估，正式定义了六类信息类别，并表明模型常包含冗余或额外信息。通过探索几种简单直观的提示工程策略，实验证明：针对长度缩减和信息内容控制的提示可使响应长度减少，同时保持输出质量，实现25%-60%的能源优化。
原文摘要 · Abstract (English)
A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for inference optimization, output compression remains relatively unexplored, with only a few preliminary efforts addressing this aspect. In this work, we first benchmark 12 decoder-only LLMs across 5 datasets, revealing that these models often produce responses that are substantially longer than necessary. We then conduct a comprehensive quality assessment of LLM responses, formally defining six information categories present in LLM responses. We show that LLMs often tend to include redundant or additional information besides the minimal answer. To address this issue of long responses by LLMs, we explore several simple and intuitive prompt-engineering strategies. Empirical evaluation shows that appropriate prompts targeting length reduction and controlling information content can achieve significant energy optimization between 25-60\% by reducing the response length while preserving the quality of LLM responses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。